borgbackup-1.0.13/0000755000175000017500000000000013431514213013666 5ustar useruser00000000000000borgbackup-1.0.13/AUTHORS0000644000175000017500000000141113431513133014733 0ustar useruser00000000000000Borg authors ("The Borg Collective") ------------------------------------ - Thomas Waldmann - Antoine Beaupré - Radek Podgorny - Yuri D'Elia - Michael Hanselmann - Teemu Toivanen - Marian Beermann - Martin Hostettler Borg is a fork of Attic. Attic authors ------------- Attic is written and maintained by Jonas Borgström and various contributors: Attic Development Lead `````````````````````` - Jonas Borgström Attic Patches and Suggestions ````````````````````````````` - Brian Johnson - Cyril Roussillon - Dan Christensen - Jeremy Maitin-Shepard - Johann Klähn - Petros Moisiadis - Thomas Waldmann borgbackup-1.0.13/CHANGES.rst0000644000175000017500000024516413431513133015504 0ustar useruser00000000000000Important notes =============== This section provides information about security and corruption issues. .. _tam_vuln: Pre-1.0.9 manifest spoofing vulnerability (CVE-2016-10099) ---------------------------------------------------------- A flaw in the cryptographic authentication scheme in Borg allowed an attacker to spoof the manifest. The attack requires an attacker to be able to 1. insert files (with no additional headers) into backups 2. gain write access to the repository This vulnerability does not disclose plaintext to the attacker, nor does it affect the authenticity of existing archives. The vulnerability allows an attacker to create a spoofed manifest (the list of archives). Creating plausible fake archives may be feasible for small archives, but is unlikely for large archives. The fix adds a separate authentication tag to the manifest. For compatibility with prior versions this authentication tag is *not* required by default for existing repositories. Repositories created with 1.0.9 and later require it. Steps you should take: 1. Upgrade all clients to 1.0.9 or later. 2. Run ``borg upgrade --tam `` *on every client* for *each* repository. 3. This will list all archives, including archive IDs, for easy comparison with your logs. 4. Done. Prior versions can access and modify repositories with this measure enabled, however, to 1.0.9 or later their modifications are indiscernible from an attack and will raise an error until the below procedure is followed. We are aware that this can be be annoying in some circumstances, but don't see a way to fix the vulnerability otherwise. In case a version prior to 1.0.9 is used to modify a repository where above procedure was completed, and now you get an error message from other clients: 1. ``borg upgrade --tam --force `` once with *any* client suffices. This attack is mitigated by: - Noting/logging ``borg list``, ``borg info``, or ``borg create --stats``, which contain the archive IDs. We are not aware of others having discovered, disclosed or exploited this vulnerability. Vulnerability time line: * 2016-11-14: Vulnerability and fix discovered during review of cryptography by Marian Beermann (@enkore) * 2016-11-20: First patch * 2016-12-20: Released fixed version 1.0.9 * 2017-01-02: CVE was assigned * 2017-01-15: Released fixed version 1.1.0b3 (fix was previously only available from source) .. _attic013_check_corruption: Pre-1.0.9 potential data loss ----------------------------- If you have archives in your repository that were made with attic <= 0.13 (and later migrated to borg), running borg check would report errors in these archives. See issue #1837. The reason for this is a invalid (and useless) metadata key that was always added due to a bug in these old attic versions. If you run borg check --repair, things escalate quickly: all archive items with invalid metadata will be killed. Due to that attic bug, that means all items in all archives made with these old attic versions. Pre-1.0.4 potential repo corruption ----------------------------------- Some external errors (like network or disk I/O errors) could lead to corruption of the backup repository due to issue #1138. A sign that this happened is if "E" status was reported for a file that can not be explained by problems with the source file. If you still have logs from "borg create -v --list", you can check for "E" status. Here is what could cause corruption and what you can do now: 1) I/O errors (e.g. repo disk errors) while writing data to repo. This could lead to corrupted segment files. Fix:: # check for corrupt chunks / segments: borg check -v --repository-only REPO # repair the repo: borg check -v --repository-only --repair REPO # make sure everything is fixed: borg check -v --repository-only REPO 2) Unreliable network / unreliable connection to the repo. This could lead to archive metadata corruption. Fix:: # check for corrupt archives: borg check -v --archives-only REPO # delete the corrupt archives: borg delete --force REPO::CORRUPT_ARCHIVE # make sure everything is fixed: borg check -v --archives-only REPO 3) In case you want to do more intensive checking. The best check that everything is ok is to run a dry-run extraction:: borg extract -v --dry-run REPO::ARCHIVE .. _changelog: Changelog ========= Version 1.0.13 (2019-02-15) --------------------------- Please note: this is very likely the last 1.0.x release, please upgrade to 1.1.x. Bug fixes: - security fix: configure FUSE with "default_permissions", #3903. "default_permissions" is now enforced by borg by default to let the kernel check uid/gid/mode based permissions. "ignore_permissions" can be given to not enforce "default_permissions". - xattrs: fix borg exception handling on ENOSPC error, #3808. New features: - Read a passphrase from a file descriptor specified in the BORG_PASSPHRASE_FD environment variable. Other changes: - acl platform code: fix acl set return type - xattr: - add linux {list,get,set}xattr ctypes prototypes - fix darwin flistxattr ctypes prototype - testing / travis-ci: - fix the homebrew 1.9 issues on travis-ci, #4254 - travis OS X: use xcode 8.3 (not broken) - tox.ini: lock requirements - unbreak 1.0-maint on travis, fixes #4123 - vagrant: - misc. fixes - FUSE for macOS: upgrade 3.7.1 to 3.8.3 - Python: upgrade 3.5.5 to 3.5.6 - docs: - Update installation instructions for macOS - update release workflow using twine (docs, scripts), #4213 Version 1.0.12 (2018-04-08) --------------------------- Bug fixes: - repository: cleanup/write: invalidate cached FDs, tests - serve: fix exitcode, #2910 - extract: set bsdflags last (include immutable flag), #3263 - create --timestamp: set start time, #2957 - create: show excluded dir with "x" for tagged dirs / caches, #3189 - migrate locks to child PID when daemonize is used - Buffer: fix wrong thread-local storage use, #2951 - fix detection of non-local path, #3108 - fix LDLP restoration for subprocesses, #3077 - fix subprocess environments (xattr module's fakeroot version check, borg umount, BORG_PASSCOMMAND), #3050 - remote: deal with partial lines, #2637 - get rid of datetime.isoformat, use safe parse_timestamp to parse timestamps, #2994 - build: do .h file content checks in binary mode, fixes build issue for non-ascii header files on pure-ascii locale platforms, #3544 #3639 - remove platform.uname() call which caused library mismatch issues, #3732 - add exception handler around deprecated platform.linux_distribution() call Other changes: - require msgpack-python >= 0.4.6 and < 0.5.0, see #3753 - add parens for C preprocessor macro argument usages (did not cause malfunction) - ignore corrupt files cache, #2939 - replace "modulo" with "if" to check for wraparound in hashmap - keymanager: don't depend on optional readline module, #2980 - exclude broken pytest 3.3.0 release - exclude broken Cython 0.27(.0) release, #3066 - flake8: add some ignores - docs: - create: document exclusion through nodump - document good and problematic option placements, fix examples, #3356 - update docs about hardlinked symlinks limitation - faq: we do not implement futile attempts of ETA / progress displays - simplified rate limiting wrapper in FAQ - twitter account @borgbackup, #2948 - add note about metadata dedup and --no[ac]time, #2518 - change-passphrase only changes the passphrase, #2990 - clarify encrypted key format for borg key export, #3296 - document sshfs rename workaround, #3315 - update release checklist about security fixes - docs about how to verify a signed release, #3634 - chunk seed is generated per /repository/ - vagrant: - use FUSE for macOS 3.7.1 to build the macOS binary - use python 3.5.5 to build the binaries - add exe location to PATH when we build an exe - use https pypi url for wheezy - netbsd: bash is already installed - netbsd: fix netbsd version in PKG_PATH - use self-made FreeBSD 10.3 box, #3022 - backport fs_init (including related updates) from 1.1 - the boxcutter wheezy boxes are 404, use local ones - travis: - don't perform full Travis build on docs-only changes, #2531 - only short-circuit docs-only changes for pull requests Version 1.0.11 (2017-07-21) --------------------------- Bug fixes: - use limited unpacker for outer key (security precaution), #2174 - fix paperkey import bug Other changes: - change --checkpoint-interval default from 600s to 1800s, #2841. this improves efficiency for big repositories a lot. - docs: fix OpenSUSE command and add OpenSUSE section - tests: add tests for split_lstring and paperkey - vagrant: - fix openbsd shell - backport cpu/ram setup from master - add stretch64 VM Version 1.0.11rc1 (2017-06-27) ------------------------------ Bug fixes: - performance: rebuild hashtable if we have too few empty buckets, #2246. this fixes some sporadic, but severe performance breakdowns. - Archive: allocate zeros when needed, #2308 fixes huge memory usage of mount (8 MiB × number of archives) - IPv6 address support also: Location: more informative exception when parsing fails - borg single-file binary: use pyinstaller v3.2.1, #2396 this fixes that the prelink cronjob on some distros kills the borg binary by stripping away parts of it. - extract: - warning for unextracted big extended attributes, #2258 - also create parent dir for device files, if needed. - don't write to disk with --stdout, #2645 - archive check: detect and fix missing all-zero replacement chunks, #2180 - fix (de)compression exceptions, #2224 #2221 - files cache: update inode number, #2226 - borg rpc: use limited msgpack.Unpacker (security precaution), #2139 - Manifest: use limited msgpack.Unpacker (security precaution), #2175 - Location: accept //servername/share/path - fix ChunkIndex.__contains__ assertion for big-endian archs (harmless) - create: handle BackupOSError on a per-path level in one spot - fix error msg, there is no --keep-last in borg 1.0.x, #2282 - clamp (nano)second values to unproblematic range, #2304 - fuse / borg mount: - fix st_blocks to be an integer (not float) value - fix negative uid/gid crash (they could come into archives e.g. when backing up external drives under cygwin), #2674 - fix crash if empty (None) xattr is read - do pre-mount checks before opening repository - check llfuse is installed before asking for passphrase - borg rename: expand placeholders, #2386 - borg serve: fix forced command lines containing BORG_ env vars - fix error msg, it is --keep-within, not --within - fix borg key/debug/benchmark crashing without subcommand, #2240 - chunker: fix invalid use of types, don't do uint32_t >> 32 - document follow_symlinks requirements, check libc, #2507 New features: - added BORG_PASSCOMMAND environment variable, #2573 - add minimal version of in repository mandatory feature flags, #2134 This should allow us to make sure older borg versions can be cleanly prevented from doing operations that are no longer safe because of repository format evolution. This allows more fine grained control than just incrementing the manifest version. So for example a change that still allows new archives to be created but would corrupt the repository when an old version tries to delete an archive or check the repository would add the new feature to the check and delete set but leave it out of the write set. - borg delete --force --force to delete severely corrupted archives, #1975 Other changes: - embrace y2038 issue to support 32bit platforms - be more clear that this is a "beyond repair" case, #2427 - key file names: limit to 100 characters and remove colons from host name - upgrade FUSE for macOS to 3.5.8, #2346 - split up parsing and filtering for --keep-within, better error message, #2610 - docs: - fix caskroom link, #2299 - address SSH batch mode, #2202 #2270 - improve remote-path description - document snapshot usage, #2178 - document relative path usage, #1868 - one link per distro in the installation page - development: new branching model in git repository - kill api page - added FAQ section about backing up root partition - add bountysource badge, #2558 - create empty docs.txt reequirements, #2694 - README: how to help the project - note -v/--verbose requirement on affected options, #2542 - document borg init behaviour via append-only borg serve, #2440 - be clear about what buzhash is used for (chunking) and want it is not used for (deduplication)- also say already in the readme that we use a cryptohash for dedupe, so people don't worry, #2390 - add hint about chunker params to borg upgrade docs, #2421 - clarify borg upgrade docs, #2436 - quickstart: delete problematic BORG_PASSPRHASE use, #2623 - faq: specify "using inline shell scripts" - document pattern denial of service, #2624 - tests: - remove attic dependency of the tests, #2505 - travis: - enhance travis setuptools_scm situation - install fakeroot for Linux - add test for borg delete --force - enable remote tests on cygwin (the cygwin issue that caused these tests to break was fixed in cygwin at least since cygwin 2.8, maybe even since 2.7.0). - remove skipping the noatime tests on GNU/Hurd, #2710 - fix borg import issue, add comment, #2718 - include attic.tar.gz when installing the package also: add include_package_data=True Version 1.0.10 (2017-02-13) --------------------------- Bug fixes: - Manifest timestamps are now monotonically increasing, this fixes issues when the system clock jumps backwards or is set inconsistently across computers accessing the same repository, #2115 - Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on py.test >= 3.0, #2112 New features: - "key export" can now generate a printable HTML page with both a QR code and a human-readable "paperkey" representation (and custom text) through the ``--qr-html`` option. The same functionality is also available through `paperkey.html `_, which is the same HTML page generated by ``--qr-html``. It works with existing "key export" files and key files. Other changes: - docs: - language clarification - "borg create --one-file-system" option does not respect mount points, but considers different file systems instead, #2141 - setup.py: build_api: sort file list for determinism Version 1.0.10rc1 (2017-01-29) ------------------------------ Bug fixes: - borg serve: fix transmission data loss of pipe writes, #1268 This affects only the cygwin platform (not Linux, BSD, OS X). - Avoid triggering an ObjectiveFS bug in xattr retrieval, #1992 - When running out of buffer memory when reading xattrs, only skip the current file, #1993 - Fixed "borg upgrade --tam" crashing with unencrypted repositories. Since :ref:`the issue ` is not relevant for unencrypted repositories, it now does nothing and prints an error, #1981. - Fixed change-passphrase crashing with unencrypted repositories, #1978 - Fixed "borg check repo::archive" indicating success if "archive" does not exist, #1997 - borg check: print non-exit-code warning if --last or --prefix aren't fulfilled - fix bad parsing of wrong repo location syntax - create: don't create hard link refs to failed files, mount: handle invalid hard link refs, #2092 - detect mingw byte order, #2073 - creating a new segment: use "xb" mode, #2099 - mount: umount on SIGINT/^C when in foreground, #2082 Other changes: - binary: use fixed AND freshly compiled pyinstaller bootloader, #2002 - xattr: ignore empty names returned by llistxattr(2) et al - Enable the fault handler: install handlers for the SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL signals to dump the Python traceback. - Also print a traceback on SIGUSR2. - borg change-passphrase: print key location (simplify making a backup of it) - officially support Python 3.6 (setup.py: add Python 3.6 qualifier) - tests: - vagrant / travis / tox: add Python 3.6 based testing - vagrant: fix openbsd repo, #2042 - vagrant: fix the freebsd64 machine, #2037 #2067 - vagrant: use python 3.5.3 to build binaries, #2078 - vagrant: use osxfuse 3.5.4 for tests / to build binaries vagrant: improve darwin64 VM settings - travis: fix osxfuse install (fixes OS X testing on Travis CI) - travis: require succeeding OS X tests, #2028 - travis: use latest pythons for OS X based testing - use pytest-xdist to parallelize testing - fix xattr test race condition, #2047 - setup.cfg: fix pytest deprecation warning, #2050 - docs: - language clarification - VM backup FAQ - borg create: document how to backup stdin, #2013 - borg upgrade: fix incorrect title levels - add CVE numbers for issues fixed in 1.0.9, #2106 - fix typos (taken from Debian package patch) - remote: include data hexdump in "unexpected RPC data" error message - remote: log SSH command line at debug level - API_VERSION: use numberspaces, #2023 - remove .github from pypi package, #2051 - add pip and setuptools to requirements file, #2030 - SyncFile: fix use of fd object after close (cosmetic) - Manifest.in: simplify, exclude \*.{so,dll,orig}, #2066 - ignore posix_fadvise errors in repository.py, #2095 (works around issues with docker on ARM) - make LoggedIO.close_segment reentrant, avoid reentrance Version 1.0.9 (2016-12-20) -------------------------- Security fixes: - A flaw in the cryptographic authentication scheme in Borg allowed an attacker to spoof the manifest. See :ref:`tam_vuln` above for the steps you should take. CVE-2016-10099 was assigned to this vulnerability. - borg check: When rebuilding the manifest (which should only be needed very rarely) duplicate archive names would be handled on a "first come first serve" basis, allowing an attacker to apparently replace archives. CVE-2016-10100 was assigned to this vulnerability. Bug fixes: - borg check: - rebuild manifest if it's corrupted - skip corrupted chunks during manifest rebuild - fix TypeError in integrity error handler, #1903, #1894 - fix location parser for archives with @ char (regression introduced in 1.0.8), #1930 - fix wrong duration/timestamps if system clock jumped during a create - fix progress display not updating if system clock jumps backwards - fix checkpoint interval being incorrect if system clock jumps Other changes: - docs: - add python3-devel as a dependency for cygwin-based installation - clarify extract is relative to current directory - FAQ: fix link to changelog - markup fixes - tests: - test_get\_(cache|keys)_dir: clean env state, #1897 - get back pytest's pretty assertion failures, #1938 - setup.py build_usage: - fixed build_usage not processing all commands - fixed build_usage not generating includes for debug commands Version 1.0.9rc1 (2016-11-27) ----------------------------- Bug fixes: - files cache: fix determination of newest mtime in backup set (which is used in cache cleanup and led to wrong "A" [added] status for unchanged files in next backup), #1860. - borg check: - fix incorrectly reporting attic 0.13 and earlier archives as corrupt - handle repo w/o objects gracefully and also bail out early if repo is *completely* empty, #1815. - fix tox/pybuild in 1.0-maint - at xattr module import time, loggers are not initialized yet New features: - borg umount exposed already existing umount code via the CLI api, so users can use it, which is more consistent than using borg to mount and fusermount -u (or umount) to un-mount, #1855. - implement borg create --noatime --noctime, fixes #1853 Other changes: - docs: - display README correctly on PyPI - improve cache / index docs, esp. files cache docs, fixes #1825 - different pattern matching for --exclude, #1779 - datetime formatting examples for {now} placeholder, #1822 - clarify passphrase mode attic repo upgrade, #1854 - clarify --umask usage, #1859 - clarify how to choose PR target branch - clarify prune behavior for different archive contents, #1824 - fix PDF issues, add logo, fix authors, headings, TOC - move security verification to support section - fix links in standalone README (:ref: tags) - add link to security contact in README - add FAQ about security - move fork differences to FAQ - add more details about resource usage - tests: skip remote tests on cygwin, #1268 - travis: - allow OS X failures until the brew cask osxfuse issue is fixed - caskroom osxfuse-beta gone, it's osxfuse now (3.5.3) - vagrant: - upgrade OSXfuse / FUSE for macOS to 3.5.3 - remove llfuse from tox.ini at a central place - do not try to install llfuse on centos6 - fix fuse test for darwin, #1546 - add windows virtual machine with cygwin - Vagrantfile cleanup / code deduplication Version 1.0.8 (2016-10-29) -------------------------- Bug fixes: - RemoteRepository: Fix busy wait in call_many, #940 New features: - implement borgmajor/borgminor/borgpatch placeholders, #1694 {borgversion} was already there (full version string). With the new placeholders you can now also get e.g. 1 or 1.0 or 1.0.8. Other changes: - avoid previous_location mismatch, #1741 due to the changed canonicalization for relative pathes in PR #1711 / #1655 (implement /./ relpath hack), there would be a changed repo location warning and the user would be asked if this is ok. this would break automation and require manual intervention, which is unwanted. thus, we automatically fix the previous_location config entry, if it only changed in the expected way, but still means the same location. - docs: - deployment.rst: do not use bare variables in ansible snippet - add clarification about append-only mode, #1689 - setup.py: add comment about requiring llfuse, #1726 - update usage.rst / api.rst - repo url / archive location docs + typo fix - quickstart: add a comment about other (remote) filesystems - vagrant / tests: - no chown when rsyncing (fixes boxes w/o vagrant group) - fix fuse permission issues on linux/freebsd, #1544 - skip fuse test for borg binary + fakeroot - ignore security.selinux xattrs, fixes tests on centos, #1735 Version 1.0.8rc1 (2016-10-17) ----------------------------- Bug fixes: - fix signal handling (SIGINT, SIGTERM, SIGHUP), #1620 #1593 Fixes e.g. leftover lock files for quickly repeated signals (e.g. Ctrl-C Ctrl-C) or lost connections or systemd sending SIGHUP. - progress display: adapt formatting to narrow screens, do not crash, #1628 - borg create --read-special - fix crash on broken symlink, #1584. also correctly processes broken symlinks. before this regressed to a crash (5b45385) a broken symlink would've been skipped. - process_symlink: fix missing backup_io() Fixes a chmod/chown/chgrp/unlink/rename/... crash race between getting dirents and dispatching to process_symlink. - yes(): abort on wrong answers, saying so, #1622 - fixed exception borg serve raised when connection was closed before reposiory was openend. add an error message for this. - fix read-from-closed-FD issue, #1551 (this seems not to get triggered in 1.0.x, but was discovered in master) - hashindex: fix iterators (always raise StopIteration when exhausted) (this seems not to get triggered in 1.0.x, but was discovered in master) - enable relative pathes in ssh:// repo URLs, via /./relpath hack, #1655 - allow repo pathes with colons, #1705 - update changed repo location immediately after acceptance, #1524 - fix debug get-obj / delete-obj crash if object not found and remote repo, #1684 - pyinstaller: use a spec file to build borg.exe binary, exclude osxfuse dylib on Mac OS X (avoids mismatch lib <-> driver), #1619 New features: - add "borg key export" / "borg key import" commands, #1555, so users are able to backup / restore their encryption keys more easily. Supported formats are the keyfile format used by borg internally and a special "paper" format with by line checksums for printed backups. For the paper format, the import is an interactive process which checks each line as soon as it is input. - add "borg debug-refcount-obj" to determine a repo objects' referrer counts, #1352 Other changes: - add "borg debug ..." subcommands (borg debug-* still works, but will be removed in borg 1.1) - setup.py: Add subcommand support to build_usage. - remote: change exception message for unexpected RPC data format to indicate dataflow direction. - improved messages / error reporting: - IntegrityError: add placeholder for message, so that the message we give appears not only in the traceback, but also in the (short) error message, #1572 - borg.key: include chunk id in exception msgs, #1571 - better messages for cache newer than repo, #1700 - vagrant (testing/build VMs): - upgrade OSXfuse / FUSE for macOS to 3.5.2 - update Debian Wheezy boxes, #1686 - openbsd / netbsd: use own boxes, fixes misc rsync installation and fuse/llfuse related testing issues, #1695 #1696 #1670 #1671 #1728 - docs: - add docs for "key export" and "key import" commands, #1641 - fix inconsistency in FAQ (pv-wrapper). - fix second block in "Easy to use" section not showing on GitHub, #1576 - add bestpractices badge - link reference docs and faq about BORG_FILES_CACHE_TTL, #1561 - improve borg info --help, explain size infos, #1532 - add release signing key / security contact to README, #1560 - add contribution guidelines for developers - development.rst: add sphinx_rtd_theme to the sphinx install command - adjust border color in borg.css - add debug-info usage help file - internals.rst: fix typos - setup.py: fix build_usage to always process all commands - added docs explaining multiple --restrict-to-path flags, #1602 - add more specific warning about write-access debug commands, #1587 - clarify FAQ regarding backup of virtual machines, #1672 - tests: - work around fuse xattr test issue with recent fakeroot - simplify repo/hashindex tests - travis: test fuse-enabled borg, use trusty to have a recent FUSE - re-enable fuse tests for RemoteArchiver (no deadlocks any more) - clean env for pytest based tests, #1714 - fuse_mount contextmanager: accept any options Version 1.0.7 (2016-08-19) -------------------------- Security fixes: - borg serve: fix security issue with remote repository access, #1428 If you used e.g. --restrict-to-path /path/client1/ (with or without trailing slash does not make a difference), it acted like a path prefix match using /path/client1 (note the missing trailing slash) - the code then also allowed working in e.g. /path/client13 or /path/client1000. As this could accidentally lead to major security/privacy issues depending on the pathes you use, the behaviour was changed to be a strict directory match. That means --restrict-to-path /path/client1 (with or without trailing slash does not make a difference) now uses /path/client1/ internally (note the trailing slash here!) for matching and allows precisely that path AND any path below it. So, /path/client1 is allowed, /path/client1/repo1 is allowed, but not /path/client13 or /path/client1000. If you willingly used the undocumented (dangerous) previous behaviour, you may need to rearrange your --restrict-to-path pathes now. We are sorry if that causes work for you, but we did not want a potentially dangerous behaviour in the software (not even using a for-backwards-compat option). Bug fixes: - fixed repeated LockTimeout exceptions when borg serve tried to write into a already write-locked repo (e.g. by a borg mount), #502 part b) This was solved by the fix for #1220 in 1.0.7rc1 already. - fix cosmetics + file leftover for "not a valid borg repository", #1490 - Cache: release lock if cache is invalid, #1501 - borg extract --strip-components: fix leak of preloaded chunk contents - Repository, when a InvalidRepository exception happens: - fix spurious, empty lock.roster - fix repo not closed cleanly New features: - implement borg debug-info, fixes #1122 (just calls already existing code via cli, same output as below tracebacks) Other changes: - skip the O_NOATIME test on GNU Hurd, fixes #1315 (this is a very minor issue and the GNU Hurd project knows the bug) - document using a clean repo to test / build the release Version 1.0.7rc2 (2016-08-13) ----------------------------- Bug fixes: - do not write objects to repository that are bigger than the allowed size, borg will reject reading them, #1451. Important: if you created archives with many millions of files or directories, please verify if you can open them successfully, e.g. try a "borg list REPO::ARCHIVE". - lz4 compression: dynamically enlarge the (de)compression buffer, the static buffer was not big enough for archives with extremely many items, #1453 - larger item metadata stream chunks, raise archive item limit by 8x, #1452 - fix untracked segments made by moved DELETEs, #1442 Impact: Previously (metadata) segments could become untracked when deleting data, these would never be cleaned up. - extended attributes (xattrs) related fixes: - fixed a race condition in xattrs querying that led to the entire file not being backed up (while logging the error, exit code = 1), #1469 - fixed a race condition in xattrs querying that led to a crash, #1462 - raise OSError including the error message derived from errno, deal with path being a integer FD Other changes: - print active env var override by default, #1467 - xattr module: refactor code, deduplicate, clean up - repository: split object size check into too small and too big - add a transaction_id assertion, so borg init on a broken (inconsistent) filesystem does not look like a coding error in borg, but points to the real problem. - explain confusing TypeError caused by compat support for old servers, #1456 - add forgotten usage help file from build_usage - refactor/unify buffer code into helpers.Buffer class, add tests - docs: - document archive limitation, #1452 - improve prune examples Version 1.0.7rc1 (2016-08-05) ----------------------------- Bug fixes: - fix repo lock deadlocks (related to lock upgrade), #1220 - catch unpacker exceptions, resync, #1351 - fix borg break-lock ignoring BORG_REPO env var, #1324 - files cache performance fixes (fixes unneccessary re-reading/chunking/ hashing of unmodified files for some use cases): - fix unintended file cache eviction, #1430 - implement BORG_FILES_CACHE_TTL, update FAQ, raise default TTL from 10 to 20, #1338 - FUSE: - cache partially read data chunks (performance), #965, #966 - always create a root dir, #1125 - use an OrderedDict for helptext, making the build reproducible, #1346 - RemoteRepository init: always call close on exceptions, #1370 (cosmetic) - ignore stdout/stderr broken pipe errors (cosmetic), #1116 New features: - better borg versions management support (useful esp. for borg servers wanting to offer multiple borg versions and for clients wanting to choose a specific server borg version), #1392: - add BORG_VERSION environment variable before executing "borg serve" via ssh - add new placeholder {borgversion} - substitute placeholders in --remote-path - borg init --append-only option (makes using the more secure append-only mode more convenient. when used remotely, this requires 1.0.7+ also on the borg server), #1291. Other changes: - Vagrantfile: - darwin64: upgrade to FUSE for macOS 3.4.1 (aka osxfuse), #1378 - xenial64: use user "ubuntu", not "vagrant" (as usual), #1331 - tests: - fix fuse tests on OS X, #1433 - docs: - FAQ: add backup using stable filesystem names recommendation - FAQ about glibc compatibility added, #491, glibc-check improved - FAQ: 'A' unchanged file; remove ambiguous entry age sentence. - OS X: install pkg-config to build with FUSE support, fixes #1400 - add notes about shell/sudo pitfalls with env. vars, #1380 - added platform feature matrix - implement borg debug-dump-repo-objs Version 1.0.6 (2016-07-12) -------------------------- Bug fixes: - Linux: handle multiple LD_PRELOAD entries correctly, #1314, #1111 - Fix crash with unclear message if the libc is not found, #1314, #1111 Other changes: - tests: - Fixed O_NOATIME tests for Solaris and GNU Hurd, #1315 - Fixed sparse file tests for (file) systems not supporting it, #1310 - docs: - Fixed syntax highlighting, #1313 - misc docs: added data processing overview picture Version 1.0.6rc1 (2016-07-10) ----------------------------- New features: - borg check --repair: heal damaged files if missing chunks re-appear (e.g. if the previously missing chunk was added again in a later backup archive), #148. (*) Also improved logging. Bug fixes: - sync_dir: silence fsync() failing with EINVAL, #1287 Some network filesystems (like smbfs) don't support this and we use this in repository code. - borg mount (FUSE): - fix directories being shadowed when contained paths were also specified, #1295 - raise I/O Error (EIO) on damaged files (unless -o allow_damaged_files is used), #1302. (*) - borg extract: warn if a damaged file is extracted, #1299. (*) - Added some missing return code checks (ChunkIndex._add, hashindex_resize). - borg check: fix/optimize initial hash table size, avoids resize of the table. Other changes: - tests: - add more FUSE tests, #1284 - deduplicate fuse (u)mount code - fix borg binary test issues, #862 - docs: - changelog: added release dates to older borg releases - fix some sphinx (docs generator) warnings, #881 Notes: (*) Some features depend on information (chunks_healthy list) added to item metadata when a file with missing chunks was "repaired" using all-zero replacement chunks. The chunks_healthy list is generated since borg 1.0.4, thus borg can't recognize such "repaired" (but content-damaged) files if the repair was done with an older borg version. Version 1.0.5 (2016-07-07) -------------------------- Bug fixes: - borg mount: fix FUSE crash in xattr code on Linux introduced in 1.0.4, #1282 Other changes: - backport some FAQ entries from master branch - add release helper scripts - Vagrantfile: - centos6: no FUSE, don't build binary - add xz for redhat-like dists Version 1.0.4 (2016-07-07) -------------------------- New features: - borg serve --append-only, #1168 This was included because it was a simple change (append-only functionality was already present via repository config file) and makes better security now practically usable. - BORG_REMOTE_PATH environment variable, #1258 This was included because it was a simple change (--remote-path cli option was already present) and makes borg much easier to use if you need it. - Repository: cleanup incomplete transaction on "no space left" condition. In many cases, this can avoid a 100% full repo filesystem (which is very problematic as borg always needs free space - even to delete archives). Bug fixes: - Fix wrong handling and reporting of OSErrors in borg create, #1138. This was a serious issue: in the context of "borg create", errors like repository I/O errors (e.g. disk I/O errors, ssh repo connection errors) were handled badly and did not lead to a crash (which would be good for this case, because the repo transaction would be incomplete and trigger a transaction rollback to clean up). Now, error handling for source files is cleanly separated from every other error handling, so only problematic input files are logged and skipped. - Implement fail-safe error handling for borg extract. Note that this isn't nearly as critical as the borg create error handling bug, since nothing is written to the repo. So this was "merely" misleading error reporting. - Add missing error handler in directory attr restore loop. - repo: make sure write data hits disk before the commit tag (#1236) and also sync the containing directory. - FUSE: getxattr fail must use errno.ENOATTR, #1126 (fixes Mac OS X Finder malfunction: "zero bytes" file length, access denied) - borg check --repair: do not lose information about the good/original chunks. If we do not lose the original chunk IDs list when "repairing" a file (replacing missing chunks with all-zero chunks), we have a chance to "heal" the file back into its original state later, in case the chunks re-appear (e.g. in a fresh backup). Healing is not implemented yet, see #148. - fixes for --read-special mode: - ignore known files cache, #1241 - fake regular file mode, #1214 - improve symlinks handling, #1215 - remove passphrase from subprocess environment, #1105 - Ignore empty index file (will trigger index rebuild), #1195 - add missing placeholder support for --prefix, #1027 - improve exception handling for placeholder replacement - catch and format exceptions in arg parsing - helpers: fix "undefined name 'e'" in exception handler - better error handling for missing repo manifest, #1043 - borg delete: - make it possible to delete a repo without manifest - borg delete --forced allows to delete corrupted archives, #1139 - borg check: - make borg check work for empty repo - fix resync and msgpacked item qualifier, #1135 - rebuild_manifest: fix crash if 'name' or 'time' key were missing. - better validation of item metadata dicts, #1130 - better validation of archive metadata dicts - close the repo on exit - even if rollback did not work, #1197. This is rather cosmetic, it avoids repo closing in the destructor. - tests: - fix sparse file test, #1170 - flake8: ignore new F405, #1185 - catch "invalid argument" on cygwin, #257 - fix sparseness assertion in test prep, #1264 Other changes: - make borg build/work on OpenSSL 1.0 and 1.1, #1187 - docs / help: - fix / clarify prune help, #1143 - fix "patterns" help formatting - add missing docs / help about placeholders - resources: rename atticmatic to borgmatic - document sshd settings, #545 - more details about checkpoints, add split trick, #1171 - support docs: add freenode web chat link, #1175 - add prune visualization / example, #723 - add note that Fnmatch is default, #1247 - make clear that lzma levels > 6 are a waste of cpu cycles - add a "do not edit" note to auto-generated files, #1250 - update cygwin installation docs - repository interoperability with borg master (1.1dev) branch: - borg check: read item metadata keys from manifest, #1147 - read v2 hints files, #1235 - fix hints file "unknown version" error handling bug - tests: add tests for format_line - llfuse: update version requirement for freebsd - Vagrantfile: - use openbsd 5.9, #716 - do not install llfuse on netbsd (broken) - update OSXfuse to version 3.3.3 - use Python 3.5.2 to build the binaries - glibc compatibility checker: scripts/glibc_check.py - add .eggs to .gitignore Version 1.0.3 (2016-05-20) -------------------------- Bug fixes: - prune: avoid that checkpoints are kept and completed archives are deleted in a prune run), #997 - prune: fix commandline argument validation - some valid command lines were considered invalid (annoying, but harmless), #942 - fix capabilities extraction on Linux (set xattrs last, after chown()), #1069 - repository: fix commit tags being seen in data - when probing key files, do binary reads. avoids crash when non-borg binary files are located in borg's key files directory. - handle SIGTERM and make a clean exit - avoids orphan lock files. - repository cache: don't cache large objects (avoid using lots of temp. disk space), #1063 Other changes: - Vagrantfile: OS X: update osxfuse / install lzma package, #933 - setup.py: add check for platform_darwin.c - setup.py: on freebsd, use a llfuse release that builds ok - docs / help: - update readthedocs URLs, #991 - add missing docs for "borg break-lock", #992 - borg create help: add some words to about the archive name - borg create help: document format tags, #894 Version 1.0.2 (2016-04-16) -------------------------- Bug fixes: - fix malfunction and potential corruption on (nowadays rather rare) big-endian architectures or bi-endian archs in (rare) BE mode. #886, #889 cache resync / index merge was malfunctioning due to this, potentially leading to data loss. borg info had cosmetic issues (displayed wrong values). note: all (widespread) little-endian archs (like x86/x64) or bi-endian archs in (widespread) LE mode (like ARMEL, MIPSEL, ...) were NOT affected. - add overflow and range checks for 1st (special) uint32 of the hashindex values, switch from int32 to uint32. - fix so that refcount will never overflow, but just stick to max. value after a overflow would have occured. - borg delete: fix --cache-only for broken caches, #874 Makes --cache-only idempotent: it won't fail if the cache is already deleted. - fixed borg create --one-file-system erroneously traversing into other filesystems (if starting fs device number was 0), #873 - workround a bug in Linux fadvise FADV_DONTNEED, #907 Other changes: - better test coverage for hashindex, incl. overflow testing, checking correct computations so endianness issues would be discovered. - reproducible doc for ProgressIndicator*, make the build reproducible. - use latest llfuse for vagrant machines - docs: - use /path/to/repo in examples, fixes #901 - fix confusing usage of "repo" as archive name (use "arch") Version 1.0.1 (2016-04-08) -------------------------- New features: Usually there are no new features in a bugfix release, but these were added due to their high impact on security/safety/speed or because they are fixes also: - append-only mode for repositories, #809, #36 (see docs) - borg create: add --ignore-inode option to make borg detect unmodified files even if your filesystem does not have stable inode numbers (like sshfs and possibly CIFS). - add options --warning, --error, --critical for missing log levels, #826. it's not recommended to suppress warnings or errors, but the user may decide this on his own. note: --warning is not given to borg serve so a <= 1.0.0 borg will still work as server (it is not needed as it is the default). do not use --error or --critical when using a <= 1.0.0 borg server. Bug fixes: - fix silently skipping EIO, #748 - add context manager for Repository (avoid orphan repository locks), #285 - do not sleep for >60s while waiting for lock, #773 - unpack file stats before passing to FUSE - fix build on illumos - don't try to backup doors or event ports (Solaris and derivates) - remove useless/misleading libc version display, #738 - test suite: reset exit code of persistent archiver, #844 - RemoteRepository: clean up pipe if remote open() fails - Remote: don't print tracebacks for Error exceptions handled downstream, #792 - if BORG_PASSPHRASE is present but wrong, don't prompt for password, but fail instead, #791 - ArchiveChecker: move "orphaned objects check skipped" to INFO log level, #826 - fix capitalization, add ellipses, change log level to debug for 2 messages, #798 Other changes: - update llfuse requirement, llfuse 1.0 works - update OS / dist packages on build machines, #717 - prefer showing --info over -v in usage help, #859 - docs: - fix cygwin requirements (gcc-g++) - document how to debug / file filesystem issues, #664 - fix reproducible build of api docs - RTD theme: CSS !important overwrite, #727 - Document logo font. Recreate logo png. Remove GIMP logo file. Version 1.0.0 (2016-03-05) -------------------------- The major release number change (0.x -> 1.x) indicates bigger incompatible changes, please read the compatibility notes, adapt / test your scripts and check your backup logs. Compatibility notes: - drop support for python 3.2 and 3.3, require 3.4 or 3.5, #221 #65 #490 note: we provide binaries that include python 3.5.1 and everything else needed. they are an option in case you are stuck with < 3.4 otherwise. - change encryption to be on by default (using "repokey" mode) - moved keyfile keys from ~/.borg/keys to ~/.config/borg/keys, you can either move them manually or run "borg upgrade " - remove support for --encryption=passphrase, use borg migrate-to-repokey to switch to repokey mode, #97 - remove deprecated --compression , use --compression zlib, instead in case of 0, you could also use --compression none - remove deprecated --hourly/daily/weekly/monthly/yearly use --keep-hourly/daily/weekly/monthly/yearly instead - remove deprecated --do-not-cross-mountpoints, use --one-file-system instead - disambiguate -p option, #563: - -p now is same as --progress - -P now is same as --prefix - remove deprecated "borg verify", use "borg extract --dry-run" instead - cleanup environment variable semantics, #355 the environment variables used to be "yes sayers" when set, this was conceptually generalized to "automatic answerers" and they just give their value as answer (as if you typed in that value when being asked). See the "usage" / "Environment Variables" section of the docs for details. - change the builtin default for --chunker-params, create 2MiB chunks, #343 --chunker-params new default: 19,23,21,4095 - old default: 10,23,16,4095 one of the biggest issues with borg < 1.0 (and also attic) was that it had a default target chunk size of 64kiB, thus it created a lot of chunks and thus also a huge chunk management overhead (high RAM and disk usage). please note that the new default won't change the chunks that you already have in your repository. the new big chunks do not deduplicate with the old small chunks, so expect your repo to grow at least by the size of every changed file and in the worst case (e.g. if your files cache was lost / is not used) by the size of every file (minus any compression you might use). in case you want to immediately see a much lower resource usage (RAM / disk) for chunks management, it might be better to start with a new repo than continuing in the existing repo (with an existing repo, you'ld have to wait until all archives with small chunks got pruned to see a lower resource usage). if you used the old --chunker-params default value (or if you did not use --chunker-params option at all) and you'ld like to continue using small chunks (and you accept the huge resource usage that comes with that), just explicitly use borg create --chunker-params=10,23,16,4095. - archive timestamps: the 'time' timestamp now refers to archive creation start time (was: end time), the new 'time_end' timestamp refers to archive creation end time. This might affect prune if your backups take rather long. if you give a timestamp via cli this is stored into 'time', therefore it now needs to mean archive creation start time. New features: - implement password roundtrip, #695 Bug fixes: - remote end does not need cache nor keys directories, do not create them, #701 - added retry counter for passwords, #703 Other changes: - fix compiler warnings, #697 - docs: - update README.rst to new changelog location in docs/changes.rst - add Teemu to AUTHORS - changes.rst: fix old chunker params, #698 - FAQ: how to limit bandwidth Version 1.0.0rc2 (2016-02-28) ----------------------------- New features: - format options for location: user, pid, fqdn, hostname, now, utcnow, user - borg list --list-format - borg prune -v --list enables the keep/prune list output, #658 Bug fixes: - fix _open_rb noatime handling, #657 - add a simple archivename validator, #680 - borg create --stats: show timestamps in localtime, use same labels/formatting as borg info, #651 - llfuse compatibility fixes (now compatible with: 0.40, 0.41, 0.42) Other changes: - it is now possible to use "pip install borgbackup[fuse]" to automatically install the llfuse dependency using the correct version requirement for it. you still need to care about having installed the FUSE / build related OS package first, though, so that building llfuse can succeed. - Vagrant: drop Ubuntu Precise (12.04) - does not have Python >= 3.4 - Vagrant: use pyinstaller v3.1.1 to build binaries - docs: - borg upgrade: add to docs that only LOCAL repos are supported - borg upgrade also handles borg 0.xx -> 1.0 - use pip extras or requirements file to install llfuse - fix order in release process - updated usage docs and other minor / cosmetic fixes - verified borg examples in docs, #644 - freebsd dependency installation and fuse configuration, #649 - add example how to restore a raw device, #671 - add a hint about the dev headers needed when installing from source - add examples for delete (and handle delete after list, before prune), #656 - update example for borg create -v --stats (use iso datetime format), #663 - added example to BORG_RSH docs - "connection closed by remote": add FAQ entry and point to issue #636 Version 1.0.0rc1 (2016-02-07) ----------------------------- New features: - borg migrate-to-repokey ("passphrase" -> "repokey" encryption key mode) - implement --short for borg list REPO, #611 - implement --list for borg extract (consistency with borg create) - borg serve: overwrite client's --restrict-to-path with ssh forced command's option value (but keep everything else from the client commandline), #544 - use $XDG_CONFIG_HOME/keys for keyfile keys (~/.config/borg/keys), #515 - "borg upgrade" moves the keyfile keys to the new location - display both archive creation start and end time in "borg info", #627 Bug fixes: - normalize trailing slashes for the repository path, #606 - Cache: fix exception handling in __init__, release lock, #610 Other changes: - suppress unneeded exception context (PEP 409), simpler tracebacks - removed special code needed to deal with imperfections / incompatibilities / missing stuff in py 3.2/3.3, simplify code that can be done simpler in 3.4 - removed some version requirements that were kept on old versions because newer did not support py 3.2 any more - use some py 3.4+ stdlib code instead of own/openssl/pypi code: - use os.urandom instead of own cython openssl RAND_bytes wrapper, #493 - use hashlib.pbkdf2_hmac from py stdlib instead of own openssl wrapper - use hmac.compare_digest instead of == operator (constant time comparison) - use stat.filemode instead of homegrown code - use "mock" library from stdlib, #145 - remove borg.support (with non-broken argparse copy), it is ok in 3.4+, #358 - Vagrant: copy CHANGES.rst as symlink, #592 - cosmetic code cleanups, add flake8 to tox/travis, #4 - docs / help: - make "borg -h" output prettier, #591 - slightly rephrase prune help - add missing example for --list option of borg create - quote exclude line that includes an asterisk to prevent shell expansion - fix dead link to license - delete Ubuntu Vivid, it is not supported anymore (EOL) - OS X binary does not work for older OS X releases, #629 - borg serve's special support for forced/original ssh commands, #544 - misc. updates and fixes Version 0.30.0 (2016-01-23) --------------------------- Compatibility notes: - you may need to use -v (or --info) more often to actually see output emitted at INFO log level (because it is suppressed at the default WARNING log level). See the "general" section in the usage docs. - for borg create, you need --list (additionally to -v) to see the long file list (was needed so you can have e.g. --stats alone without the long list) - see below about BORG_DELETE_I_KNOW_WHAT_I_AM_DOING (was: BORG_CHECK_I_KNOW_WHAT_I_AM_DOING) Bug fixes: - fix crash when using borg create --dry-run --keep-tag-files, #570 - make sure teardown with cleanup happens for Cache and RepositoryCache, avoiding leftover locks and TEMP dir contents, #285 (partially), #548 - fix locking KeyError, partial fix for #502 - log stats consistently, #526 - add abbreviated weekday to timestamp format, fixes #496 - strip whitespace when loading exclusions from file - unset LD_LIBRARY_PATH before invoking ssh, fixes strange OpenSSL library version warning when using the borg binary, #514 - add some error handling/fallback for C library loading, #494 - added BORG_DELETE_I_KNOW_WHAT_I_AM_DOING for check in "borg delete", #503 - remove unused "repair" rpc method name New features: - borg create: implement exclusions using regular expression patterns. - borg create: implement inclusions using patterns. - borg extract: support patterns, #361 - support different styles for patterns: - fnmatch (`fm:` prefix, default when omitted), like borg <= 0.29. - shell (`sh:` prefix) with `*` not matching directory separators and `**/` matching 0..n directories - path prefix (`pp:` prefix, for unifying borg create pp1 pp2 into the patterns system), semantics like in borg <= 0.29 - regular expression (`re:`), new! - --progress option for borg upgrade (#291) and borg delete - update progress indication more often (e.g. for borg create within big files or for borg check repo), #500 - finer chunker granularity for items metadata stream, #547, #487 - borg create --list now used (additionally to -v) to enable the verbose file list output - display borg version below tracebacks, #532 Other changes: - hashtable size (and thus: RAM and disk consumption) follows a growth policy: grows fast while small, grows slower when getting bigger, #527 - Vagrantfile: use pyinstaller 3.1 to build binaries, freebsd sqlite3 fix, fixes #569 - no separate binaries for centos6 any more because the generic linux binaries also work on centos6 (or in general: on systems with a slightly older glibc than debian7 - dev environment: require virtualenv<14.0 so we get a py32 compatible pip - docs: - add space-saving chunks.archive.d trick to FAQ - important: clarify -v and log levels in usage -> general, please read! - sphinx configuration: create a simple man page from usage docs - add a repo server setup example - disable unneeded SSH features in authorized_keys examples for security. - borg prune only knows "--keep-within" and not "--within" - add gource video to resources docs, #507 - add netbsd install instructions - authors: make it more clear what refers to borg and what to attic - document standalone binary requirements, #499 - rephrase the mailing list section - development docs: run build_api and build_usage before tagging release - internals docs: hash table max. load factor is 0.75 now - markup, typo, grammar, phrasing, clarifications and other fixes. - add gcc gcc-c++ to redhat/fedora/corora install docs, fixes #583 Version 0.29.0 (2015-12-13) --------------------------- Compatibility notes: - when upgrading to 0.29.0 you need to upgrade client as well as server installations due to the locking and commandline interface changes otherwise you'll get an error msg about a RPC protocol mismatch or a wrong commandline option. if you run a server that needs to support both old and new clients, it is suggested that you have a "borg-0.28.2" and a "borg-0.29.0" command. clients then can choose via e.g. "borg --remote-path=borg-0.29.0 ...". - the default waiting time for a lock changed from infinity to 1 second for a better interactive user experience. if the repo you want to access is currently locked, borg will now terminate after 1s with an error message. if you have scripts that shall wait for the lock for a longer time, use --lock-wait N (with N being the maximum wait time in seconds). Bug fixes: - hash table tuning (better chosen hashtable load factor 0.75 and prime initial size of 1031 gave ~1000x speedup in some scenarios) - avoid creation of an orphan lock for one case, #285 - --keep-tag-files: fix file mode and multiple tag files in one directory, #432 - fixes for "borg upgrade" (attic repo converter), #466 - remove --progress isatty magic (and also --no-progress option) again, #476 - borg init: display proper repo URL - fix format of umask in help pages, #463 New features: - implement --lock-wait, support timeout for UpgradableLock, #210 - implement borg break-lock command, #157 - include system info below traceback, #324 - sane remote logging, remote stderr, #461: - remote log output: intercept it and log it via local logging system, with "Remote: " prefixed to message. log remote tracebacks. - remote stderr: output it to local stderr with "Remote: " prefixed. - add --debug and --info (same as --verbose) to set the log level of the builtin logging configuration (which otherwise defaults to warning), #426 note: there are few messages emitted at DEBUG level currently. - optionally configure logging via env var BORG_LOGGING_CONF - add --filter option for status characters: e.g. to show only the added or modified files (and also errors), use "borg create -v --filter=AME ...". - more progress indicators, #394 - use ISO-8601 date and time format, #375 - "borg check --prefix" to restrict archive checking to that name prefix, #206 Other changes: - hashindex_add C implementation (speed up cache re-sync for new archives) - increase FUSE read_size to 1024 (speed up metadata operations) - check/delete/prune --save-space: free unused segments quickly, #239 - increase rpc protocol version to 2 (see also Compatibility notes), #458 - silence borg by default (via default log level WARNING) - get rid of C compiler warnings, #391 - upgrade OS X FUSE to 3.0.9 on the OS X binary build system - use python 3.5.1 to build binaries - docs: - new mailing list borgbackup@python.org, #468 - readthedocs: color and logo improvements - load coverage icons over SSL (avoids mixed content) - more precise binary installation steps - update release procedure docs about OS X FUSE - FAQ entry about unexpected 'A' status for unchanged file(s), #403 - add docs about 'E' file status - add "borg upgrade" docs, #464 - add developer docs about output and logging - clarify encryption, add note about client-side encryption - add resources section, with videos, talks, presentations, #149 - Borg moved to Arch Linux [community] - fix wrong installation instructions for archlinux Version 0.28.2 (2015-11-15) --------------------------- New features: - borg create --exclude-if-present TAGFILE - exclude directories that have the given file from the backup. You can additionally give --keep-tag-files to preserve just the directory roots and the tag-files (but not backup other directory contents), #395, attic #128, attic #142 Other changes: - do not create docs sources at build time (just have them in the repo), completely remove have_cython() hack, do not use the "mock" library at build time, #384 - avoid hidden import, make it easier for PyInstaller, easier fix for #218 - docs: - add description of item flags / status output, fixes #402 - explain how to regenerate usage and API files (build_api or build_usage) and when to commit usage files directly into git, #384 - minor install docs improvements Version 0.28.1 (2015-11-08) --------------------------- Bug fixes: - do not try to build api / usage docs for production install, fixes unexpected "mock" build dependency, #384 Other changes: - avoid using msgpack.packb at import time - fix formatting issue in changes.rst - fix build on readthedocs Version 0.28.0 (2015-11-08) --------------------------- Compatibility notes: - changed return codes (exit codes), see docs. in short: old: 0 = ok, 1 = error. now: 0 = ok, 1 = warning, 2 = error New features: - refactor return codes (exit codes), fixes #61 - add --show-rc option enable "terminating with X status, rc N" output, fixes 58, #351 - borg create backups atime and ctime additionally to mtime, fixes #317 - extract: support atime additionally to mtime - FUSE: support ctime and atime additionally to mtime - support borg --version - emit a warning if we have a slow msgpack installed - borg list --prefix=thishostname- REPO, fixes #205 - Debug commands (do not use except if you know what you do: debug-get-obj, debug-put-obj, debug-delete-obj, debug-dump-archive-items. Bug fixes: - setup.py: fix bug related to BORG_LZ4_PREFIX processing - fix "check" for repos that have incomplete chunks, fixes #364 - borg mount: fix unlocking of repository at umount time, fixes #331 - fix reading files without touching their atime, #334 - non-ascii ACL fixes for Linux, FreeBSD and OS X, #277 - fix acl_use_local_uid_gid() and add a test for it, attic #359 - borg upgrade: do not upgrade repositories in place by default, #299 - fix cascading failure with the index conversion code, #269 - borg check: implement 'cmdline' archive metadata value decoding, #311 - fix RobustUnpacker, it missed some metadata keys (new atime and ctime keys were missing, but also bsdflags). add check for unknown metadata keys. - create from stdin: also save atime, ctime (cosmetic) - use default_notty=False for confirmations, fixes #345 - vagrant: fix msgpack installation on centos, fixes #342 - deal with unicode errors for symlinks in same way as for regular files and have a helpful warning message about how to fix wrong locale setup, fixes #382 - add ACL keys the RobustUnpacker must know about Other changes: - improve file size displays, more flexible size formatters - explicitly commit to the units standard, #289 - archiver: add E status (means that an error occurred when processing this (single) item - do binary releases via "github releases", closes #214 - create: use -x and --one-file-system (was: --do-not-cross-mountpoints), #296 - a lot of changes related to using "logging" module and screen output, #233 - show progress display if on a tty, output more progress information, #303 - factor out status output so it is consistent, fix surrogates removal, maybe fixes #309 - move away from RawConfigParser to ConfigParser - archive checker: better error logging, give chunk_id and sequence numbers (can be used together with borg debug-dump-archive-items). - do not mention the deprecated passphrase mode - emit a deprecation warning for --compression N (giving a just a number) - misc .coverragerc fixes (and coverage measurement improvements), fixes #319 - refactor confirmation code, reduce code duplication, add tests - prettier error messages, fixes #307, #57 - tests: - add a test to find disk-full issues, #327 - travis: also run tests on Python 3.5 - travis: use tox -r so it rebuilds the tox environments - test the generated pyinstaller-based binary by archiver unit tests, #215 - vagrant: tests: announce whether fakeroot is used or not - vagrant: add vagrant user to fuse group for debianoid systems also - vagrant: llfuse install on darwin needs pkgconfig installed - vagrant: use pyinstaller from develop branch, fixes #336 - benchmarks: test create, extract, list, delete, info, check, help, fixes #146 - benchmarks: test with both the binary and the python code - archiver tests: test with both the binary and the python code, fixes #215 - make basic test more robust - docs: - moved docs to borgbackup.readthedocs.org, #155 - a lot of fixes and improvements, use mobile-friendly RTD standard theme - use zlib,6 compression in some examples, fixes #275 - add missing rename usage to docs, closes #279 - include the help offered by borg help in the usage docs, fixes #293 - include a list of major changes compared to attic into README, fixes #224 - add OS X install instructions, #197 - more details about the release process, #260 - fix linux glibc requirement (binaries built on debian7 now) - build: move usage and API generation to setup.py - update docs about return codes, #61 - remove api docs (too much breakage on rtd) - borgbackup install + basics presentation (asciinema) - describe the current style guide in documentation - add section about debug commands - warn about not running out of space - add example for rename - improve chunker params docs, fixes #362 - minor development docs update Version 0.27.0 (2015-10-07) --------------------------- New features: - "borg upgrade" command - attic -> borg one time converter / migration, #21 - temporary hack to avoid using lots of disk space for chunks.archive.d, #235: To use it: rm -rf chunks.archive.d ; touch chunks.archive.d - respect XDG_CACHE_HOME, attic #181 - add support for arbitrary SSH commands, attic #99 - borg delete --cache-only REPO (only delete cache, not REPO), attic #123 Bug fixes: - use Debian 7 (wheezy) to build pyinstaller borgbackup binaries, fixes slow down observed when running the Centos6-built binary on Ubuntu, #222 - do not crash on empty lock.roster, fixes #232 - fix multiple issues with the cache config version check, #234 - fix segment entry header size check, attic #352 plus other error handling improvements / code deduplication there. - always give segment and offset in repo IntegrityErrors Other changes: - stop producing binary wheels, remove docs about it, #147 - docs: - add warning about prune - generate usage include files only as needed - development docs: add Vagrant section - update / improve / reformat FAQ - hint to single-file pyinstaller binaries from README Version 0.26.1 (2015-09-28) --------------------------- This is a minor update, just docs and new pyinstaller binaries. - docs update about python and binary requirements - better docs for --read-special, fix #220 - re-built the binaries, fix #218 and #213 (glibc version issue) - update web site about single-file pyinstaller binaries Note: if you did a python-based installation, there is no need to upgrade. Version 0.26.0 (2015-09-19) --------------------------- New features: - Faster cache sync (do all in one pass, remove tar/compression stuff), #163 - BORG_REPO env var to specify the default repo, #168 - read special files as if they were regular files, #79 - implement borg create --dry-run, attic issue #267 - Normalize paths before pattern matching on OS X, #143 - support OpenBSD and NetBSD (except xattrs/ACLs) - support / run tests on Python 3.5 Bug fixes: - borg mount repo: use absolute path, attic #200, attic #137 - chunker: use off_t to get 64bit on 32bit platform, #178 - initialize chunker fd to -1, so it's not equal to STDIN_FILENO (0) - fix reaction to "no" answer at delete repo prompt, #182 - setup.py: detect lz4.h header file location - to support python < 3.2.4, add less buggy argparse lib from 3.2.6 (#194) - fix for obtaining ``char *`` from temporary Python value (old code causes a compile error on Mint 17.2) - llfuse 0.41 install troubles on some platforms, require < 0.41 (UnicodeDecodeError exception due to non-ascii llfuse setup.py) - cython code: add some int types to get rid of unspecific python add / subtract operations (avoid ``undefined symbol FPE_``... error on some platforms) - fix verbose mode display of stdin backup - extract: warn if a include pattern never matched, fixes #209, implement counters for Include/ExcludePatterns - archive names with slashes are invalid, attic issue #180 - chunker: add a check whether the POSIX_FADV_DONTNEED constant is defined - fixes building on OpenBSD. Other changes: - detect inconsistency / corruption / hash collision, #170 - replace versioneer with setuptools_scm, #106 - docs: - pkg-config is needed for llfuse installation - be more clear about pruning, attic issue #132 - unit tests: - xattr: ignore security.selinux attribute showing up - ext3 seems to need a bit more space for a sparse file - do not test lzma level 9 compression (avoid MemoryError) - work around strange mtime granularity issue on netbsd, fixes #204 - ignore st_rdev if file is not a block/char device, fixes #203 - stay away from the setgid and sticky mode bits - use Vagrant to do easy cross-platform testing (#196), currently: - Debian 7 "wheezy" 32bit, Debian 8 "jessie" 64bit - Ubuntu 12.04 32bit, Ubuntu 14.04 64bit - Centos 7 64bit - FreeBSD 10.2 64bit - OpenBSD 5.7 64bit - NetBSD 6.1.5 64bit - Darwin (OS X Yosemite) Version 0.25.0 (2015-08-29) --------------------------- Compatibility notes: - lz4 compression library (liblz4) is a new requirement (#156) - the new compression code is very compatible: as long as you stay with zlib compression, older borg releases will still be able to read data from a repo/archive made with the new code (note: this is not the case for the default "none" compression, use "zlib,0" if you want a "no compression" mode that can be read by older borg). Also the new code is able to read repos and archives made with older borg versions (for all zlib levels 0..9). Deprecations: - --compression N (with N being a number, as in 0.24) is deprecated. We keep the --compression 0..9 for now to not break scripts, but it is deprecated and will be removed later, so better fix your scripts now: --compression 0 (as in 0.24) is the same as --compression zlib,0 (now). BUT: if you do not want compression, you rather want --compression none (which is the default). --compression 1 (in 0.24) is the same as --compression zlib,1 (now) --compression 9 (in 0.24) is the same as --compression zlib,9 (now) New features: - create --compression none (default, means: do not compress, just pass through data "as is". this is more efficient than zlib level 0 as used in borg 0.24) - create --compression lz4 (super-fast, but not very high compression) - create --compression zlib,N (slower, higher compression, default for N is 6) - create --compression lzma,N (slowest, highest compression, default N is 6) - honor the nodump flag (UF_NODUMP) and do not backup such items - list --short just outputs a simple list of the files/directories in an archive Bug fixes: - fixed --chunker-params parameter order confusion / malfunction, fixes #154 - close fds of segments we delete (during compaction) - close files which fell out the lrucache - fadvise DONTNEED now is only called for the byte range actually read, not for the whole file, fixes #158. - fix issue with negative "all archives" size, fixes #165 - restore_xattrs: ignore if setxattr fails with EACCES, fixes #162 Other changes: - remove fakeroot requirement for tests, tests run faster without fakeroot (test setup does not fail any more without fakeroot, so you can run with or without fakeroot), fixes #151 and #91. - more tests for archiver - recover_segment(): don't assume we have an fd for segment - lrucache refactoring / cleanup, add dispose function, py.test tests - generalize hashindex code for any key length (less hardcoding) - lock roster: catch file not found in remove() method and ignore it - travis CI: use requirements file - improved docs: - replace hack for llfuse with proper solution (install libfuse-dev) - update docs about compression - update development docs about fakeroot - internals: add some words about lock files / locking system - support: mention BountySource and for what it can be used - theme: use a lighter green - add pypi, wheel, dist package based install docs - split install docs into system-specific preparations and generic instructions Version 0.24.0 (2015-08-09) --------------------------- Incompatible changes (compared to 0.23): - borg now always issues --umask NNN option when invoking another borg via ssh on the repository server. By that, it's making sure it uses the same umask for remote repos as for local ones. Because of this, you must upgrade both server and client(s) to 0.24. - the default umask is 077 now (if you do not specify via --umask) which might be a different one as you used previously. The default umask avoids that you accidentally give access permissions for group and/or others to files created by borg (e.g. the repository). Deprecations: - "--encryption passphrase" mode is deprecated, see #85 and #97. See the new "--encryption repokey" mode for a replacement. New features: - borg create --chunker-params ... to configure the chunker, fixes #16 (attic #302, attic #300, and somehow also #41). This can be used to reduce memory usage caused by chunk management overhead, so borg does not create a huge chunks index/repo index and eats all your RAM if you back up lots of data in huge files (like VM disk images). See docs/misc/create_chunker-params.txt for more information. - borg info now reports chunk counts in the chunk index. - borg create --compression 0..9 to select zlib compression level, fixes #66 (attic #295). - borg init --encryption repokey (to store the encryption key into the repo), fixes #85 - improve at-end error logging, always log exceptions and set exit_code=1 - LoggedIO: better error checks / exceptions / exception handling - implement --remote-path to allow non-default-path borg locations, #125 - implement --umask M and use 077 as default umask for better security, #117 - borg check: give a named single archive to it, fixes #139 - cache sync: show progress indication - cache sync: reimplement the chunk index merging in C Bug fixes: - fix segfault that happened for unreadable files (chunker: n needs to be a signed size_t), #116 - fix the repair mode, #144 - repo delete: add destroy to allowed rpc methods, fixes issue #114 - more compatible repository locking code (based on mkdir), maybe fixes #92 (attic #317, attic #201). - better Exception msg if no Borg is installed on the remote repo server, #56 - create a RepositoryCache implementation that can cope with >2GiB, fixes attic #326. - fix Traceback when running check --repair, attic #232 - clarify help text, fixes #73. - add help string for --no-files-cache, fixes #140 Other changes: - improved docs: - added docs/misc directory for misc. writeups that won't be included "as is" into the html docs. - document environment variables and return codes (attic #324, attic #52) - web site: add related projects, fix web site url, IRC #borgbackup - Fedora/Fedora-based install instructions added to docs - Cygwin-based install instructions added to docs - updated AUTHORS - add FAQ entries about redundancy / integrity - clarify that borg extract uses the cwd as extraction target - update internals doc about chunker params, memory usage and compression - added docs about development - add some words about resource usage in general - document how to backup a raw disk - add note about how to run borg from virtual env - add solutions for (ll)fuse installation problems - document what borg check does, fixes #138 - reorganize borgbackup.github.io sidebar, prev/next at top - deduplicate and refactor the docs / README.rst - use borg-tmp as prefix for temporary files / directories - short prune options without "keep-" are deprecated, do not suggest them - improved tox configuration - remove usage of unittest.mock, always use mock from pypi - use entrypoints instead of scripts, for better use of the wheel format and modern installs - add requirements.d/development.txt and modify tox.ini - use travis-ci for testing based on Linux and (new) OS X - use coverage.py, pytest-cov and codecov.io for test coverage support I forgot to list some stuff already implemented in 0.23.0, here they are: New features: - efficient archive list from manifest, meaning a big speedup for slow repo connections and "list ", "delete ", "prune" (attic #242, attic #167) - big speedup for chunks cache sync (esp. for slow repo connections), fixes #18 - hashindex: improve error messages Other changes: - explicitly specify binary mode to open binary files - some easy micro optimizations Version 0.23.0 (2015-06-11) --------------------------- Incompatible changes (compared to attic, fork related): - changed sw name and cli command to "borg", updated docs - package name (and name in urls) uses "borgbackup" to have less collisions - changed repo / cache internal magic strings from ATTIC* to BORG*, changed cache location to .cache/borg/ - this means that it currently won't accept attic repos (see issue #21 about improving that) Bug fixes: - avoid defect python-msgpack releases, fixes attic #171, fixes attic #185 - fix traceback when trying to do unsupported passphrase change, fixes attic #189 - datetime does not like the year 10.000, fixes attic #139 - fix "info" all archives stats, fixes attic #183 - fix parsing with missing microseconds, fixes attic #282 - fix misleading hint the fuse ImportError handler gave, fixes attic #237 - check unpacked data from RPC for tuple type and correct length, fixes attic #127 - fix Repository._active_txn state when lock upgrade fails - give specific path to xattr.is_enabled(), disable symlink setattr call that always fails - fix test setup for 32bit platforms, partial fix for attic #196 - upgraded versioneer, PEP440 compliance, fixes attic #257 New features: - less memory usage: add global option --no-cache-files - check --last N (only check the last N archives) - check: sort archives in reverse time order - rename repo::oldname newname (rename repository) - create -v output more informative - create --progress (backup progress indicator) - create --timestamp (utc string or reference file/dir) - create: if "-" is given as path, read binary from stdin - extract: if --stdout is given, write all extracted binary data to stdout - extract --sparse (simple sparse file support) - extra debug information for 'fread failed' - delete (deletes whole repo + local cache) - FUSE: reflect deduplication in allocated blocks - only allow whitelisted RPC calls in server mode - normalize source/exclude paths before matching - use posix_fadvise to not spoil the OS cache, fixes attic #252 - toplevel error handler: show tracebacks for better error analysis - sigusr1 / sigint handler to print current file infos - attic PR #286 - RPCError: include the exception args we get from remote Other changes: - source: misc. cleanups, pep8, style - docs and faq improvements, fixes, updates - cleanup crypto.pyx, make it easier to adapt to other AES modes - do os.fsync like recommended in the python docs - source: Let chunker optionally work with os-level file descriptor. - source: Linux: remove duplicate os.fsencode calls - source: refactor _open_rb code a bit, so it is more consistent / regular - source: refactor indicator (status) and item processing - source: use py.test for better testing, flake8 for code style checks - source: fix tox >=2.0 compatibility (test runner) - pypi package: add python version classifiers, add FreeBSD to platforms Attic Changelog --------------- Here you can see the full list of changes between each Attic release until Borg forked from Attic: Version 0.17 ~~~~~~~~~~~~ (bugfix release, released on X) - Fix hashindex ARM memory alignment issue (#309) - Improve hashindex error messages (#298) Version 0.16 ~~~~~~~~~~~~ (bugfix release, released on May 16, 2015) - Fix typo preventing the security confirmation prompt from working (#303) - Improve handling of systems with improperly configured file system encoding (#289) - Fix "All archives" output for attic info. (#183) - More user friendly error message when repository key file is not found (#236) - Fix parsing of iso 8601 timestamps with zero microseconds (#282) Version 0.15 ~~~~~~~~~~~~ (bugfix release, released on Apr 15, 2015) - xattr: Be less strict about unknown/unsupported platforms (#239) - Reduce repository listing memory usage (#163). - Fix BrokenPipeError for remote repositories (#233) - Fix incorrect behavior with two character directory names (#265, #268) - Require approval before accessing relocated/moved repository (#271) - Require approval before accessing previously unknown unencrypted repositories (#271) - Fix issue with hash index files larger than 2GB. - Fix Python 3.2 compatibility issue with noatime open() (#164) - Include missing pyx files in dist files (#168) Version 0.14 ~~~~~~~~~~~~ (feature release, released on Dec 17, 2014) - Added support for stripping leading path segments (#95) "attic extract --strip-segments X" - Add workaround for old Linux systems without acl_extended_file_no_follow (#96) - Add MacPorts' path to the default openssl search path (#101) - HashIndex improvements, eliminates unnecessary IO on low memory systems. - Fix "Number of files" output for attic info. (#124) - limit create file permissions so files aren't read while restoring - Fix issue with empty xattr values (#106) Version 0.13 ~~~~~~~~~~~~ (feature release, released on Jun 29, 2014) - Fix sporadic "Resource temporarily unavailable" when using remote repositories - Reduce file cache memory usage (#90) - Faster AES encryption (utilizing AES-NI when available) - Experimental Linux, OS X and FreeBSD ACL support (#66) - Added support for backup and restore of BSDFlags (OSX, FreeBSD) (#56) - Fix bug where xattrs on symlinks were not correctly restored - Added cachedir support. CACHEDIR.TAG compatible cache directories can now be excluded using ``--exclude-caches`` (#74) - Fix crash on extreme mtime timestamps (year 2400+) (#81) - Fix Python 3.2 specific lockf issue (EDEADLK) Version 0.12 ~~~~~~~~~~~~ (feature release, released on April 7, 2014) - Python 3.4 support (#62) - Various documentation improvements a new style - ``attic mount`` now supports mounting an entire repository not only individual archives (#59) - Added option to restrict remote repository access to specific path(s): ``attic serve --restrict-to-path X`` (#51) - Include "all archives" size information in "--stats" output. (#54) - Added ``--stats`` option to ``attic delete`` and ``attic prune`` - Fixed bug where ``attic prune`` used UTC instead of the local time zone when determining which archives to keep. - Switch to SI units (Power of 1000 instead 1024) when printing file sizes Version 0.11 ~~~~~~~~~~~~ (feature release, released on March 7, 2014) - New "check" command for repository consistency checking (#24) - Documentation improvements - Fix exception during "attic create" with repeated files (#39) - New "--exclude-from" option for attic create/extract/verify. - Improved archive metadata deduplication. - "attic verify" has been deprecated. Use "attic extract --dry-run" instead. - "attic prune --hourly|daily|..." has been deprecated. Use "attic prune --keep-hourly|daily|..." instead. - Ignore xattr errors during "extract" if not supported by the filesystem. (#46) Version 0.10 ~~~~~~~~~~~~ (bugfix release, released on Jan 30, 2014) - Fix deadlock when extracting 0 sized files from remote repositories - "--exclude" wildcard patterns are now properly applied to the full path not just the file name part (#5). - Make source code endianness agnostic (#1) Version 0.9 ~~~~~~~~~~~ (feature release, released on Jan 23, 2014) - Remote repository speed and reliability improvements. - Fix sorting of segment names to ignore NFS left over files. (#17) - Fix incorrect display of time (#13) - Improved error handling / reporting. (#12) - Use fcntl() instead of flock() when locking repository/cache. (#15) - Let ssh figure out port/user if not specified so we don't override .ssh/config (#9) - Improved libcrypto path detection (#23). Version 0.8.1 ~~~~~~~~~~~~~ (bugfix release, released on Oct 4, 2013) - Fix segmentation fault issue. Version 0.8 ~~~~~~~~~~~ (feature release, released on Oct 3, 2013) - Fix xattr issue when backing up sshfs filesystems (#4) - Fix issue with excessive index file size (#6) - Support access of read only repositories. - New syntax to enable repository encryption: attic init --encryption="none|passphrase|keyfile". - Detect and abort if repository is older than the cache. Version 0.7 ~~~~~~~~~~~ (feature release, released on Aug 5, 2013) - Ported to FreeBSD - Improved documentation - Experimental: Archives mountable as fuse filesystems. - The "user." prefix is no longer stripped from xattrs on Linux Version 0.6.1 ~~~~~~~~~~~~~ (bugfix release, released on July 19, 2013) - Fixed an issue where mtime was not always correctly restored. Version 0.6 ~~~~~~~~~~~ First public release on July 9, 2013 borgbackup-1.0.13/LICENSE0000644000175000017500000000273413431513133014701 0ustar useruser00000000000000Copyright (C) 2015-2019 The Borg Collective (see AUTHORS file) Copyright (C) 2010-2014 Jonas Borgström All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. borgbackup-1.0.13/MANIFEST.in0000644000175000017500000000035713431513133015431 0ustar useruser00000000000000include README.rst AUTHORS LICENSE CHANGES.rst MANIFEST.in exclude .coveragerc .gitattributes .gitignore .travis.yml Vagrantfile prune .travis prune .github graft borg graft docs prune docs/_build global-exclude *.py[co] *.orig *.so *.dll borgbackup-1.0.13/PKG-INFO0000644000175000017500000002063313431514213014767 0ustar useruser00000000000000Metadata-Version: 2.1 Name: borgbackup Version: 1.0.13 Summary: Deduplicated, encrypted, authenticated and compressed backups Home-page: https://borgbackup.readthedocs.io/ Author: The Borg Collective (see AUTHORS file) Author-email: borgbackup@python.org License: BSD Description: What is BorgBackup? ------------------- BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets. See the `installation manual`_ or, if you have already downloaded Borg, ``docs/installation.rst`` to get started with Borg. .. _installation manual: https://borgbackup.readthedocs.org/en/stable/installation.html Main features ~~~~~~~~~~~~~ **Space efficient storage** Deduplication based on content-defined chunking is used to reduce the number of bytes stored: each file is split into a number of variable length chunks and only chunks that have never been seen before are added to the repository. A chunk is considered duplicate if its id_hash value is identical. A cryptographically strong hash or MAC function is used as id_hash, e.g. (hmac-)sha256. To deduplicate, all the chunks in the same repository are considered, no matter whether they come from different machines, from previous backups, from the same backup or even from the same single file. Compared to other deduplication approaches, this method does NOT depend on: * file/directory names staying the same: So you can move your stuff around without killing the deduplication, even between machines sharing a repo. * complete files or time stamps staying the same: If a big file changes a little, only a few new chunks need to be stored - this is great for VMs or raw disks. * The absolute position of a data chunk inside a file: Stuff may get shifted and will still be found by the deduplication algorithm. **Speed** * performance critical code (chunking, compression, encryption) is implemented in C/Cython * local caching of files/chunks index data * quick detection of unmodified files **Data encryption** All data can be protected using 256-bit AES encryption, data integrity and authenticity is verified using HMAC-SHA256. Data is encrypted clientside. **Compression** All data can be compressed by lz4 (super fast, low compression), zlib (medium speed and compression) or lzma (low speed, high compression). **Off-site backups** Borg can store data on any remote host accessible over SSH. If Borg is installed on the remote host, big performance gains can be achieved compared to using a network filesystem (sshfs, nfs, ...). **Backups mountable as filesystems** Backup archives are mountable as userspace filesystems for easy interactive backup examination and restores (e.g. by using a regular file manager). **Easy installation on multiple platforms** We offer single-file binaries that do not require installing anything - you can just run them on these platforms: * Linux * Mac OS X * FreeBSD * OpenBSD and NetBSD (no xattrs/ACLs support or binaries yet) * Cygwin (not supported, no binaries yet) * Linux Subsystem of Windows 10 (not supported) **Free and Open Source Software** * security and functionality can be audited independently * licensed under the BSD (3-clause) license, see `License`_ for the complete license Easy to use ~~~~~~~~~~~ Initialize a new backup repository and create a backup archive:: $ borg init /path/to/repo $ borg create /path/to/repo::Saturday1 ~/Documents Now doing another backup, just to show off the great deduplication:: $ borg create -v --stats /path/to/repo::Saturday2 ~/Documents ----------------------------------------------------------------------------- Archive name: Saturday2 Archive fingerprint: 622b7c53c... Time (start): Sat, 2016-02-27 14:48:13 Time (end): Sat, 2016-02-27 14:48:14 Duration: 0.88 seconds Number of files: 163 ----------------------------------------------------------------------------- Original size Compressed size Deduplicated size This archive: 6.85 MB 6.85 MB 30.79 kB <-- ! All archives: 13.69 MB 13.71 MB 6.88 MB Unique chunks Total chunks Chunk index: 167 330 ----------------------------------------------------------------------------- For a graphical frontend refer to our complementary project `BorgWeb `_. Helping, Donations and Bounties ------------------------------- Your help is always welcome! Spread the word, give feedback, help with documentation, testing or development. You can also give monetary support to the project, see there for details: https://borgbackup.readthedocs.io/en/stable/support.html#bounties-and-fundraisers Links ----- * `Main Web Site `_ * `Releases `_, `PyPI packages `_ and `ChangeLog `_ * `GitHub `_ and `Issue Tracker `_. * `Web-Chat (IRC) `_ and `Mailing List `_ * `License `_ * `Security contact `_ Compatibility notes ------------------- EXPECT THAT WE WILL BREAK COMPATIBILITY REPEATEDLY WHEN MAJOR RELEASE NUMBER CHANGES (like when going from 0.x.y to 1.0.0 or from 1.x.y to 2.0.0). NOT RELEASED DEVELOPMENT VERSIONS HAVE UNKNOWN COMPATIBILITY PROPERTIES. THIS IS SOFTWARE IN DEVELOPMENT, DECIDE YOURSELF WHETHER IT FITS YOUR NEEDS. Security issues should be reported to the `Security contact`_ (or see ``docs/suppport.rst`` in the source distribution). Platform: Linux Platform: MacOS X Platform: FreeBSD Platform: OpenBSD Platform: NetBSD Classifier: Development Status :: 4 - Beta Classifier: Environment :: Console Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: POSIX :: BSD :: FreeBSD Classifier: Operating System :: POSIX :: BSD :: OpenBSD Classifier: Operating System :: POSIX :: BSD :: NetBSD Classifier: Operating System :: MacOS :: MacOS X Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Topic :: Security :: Cryptography Classifier: Topic :: System :: Archiving :: Backup Provides-Extra: fuse borgbackup-1.0.13/README.rst0000644000175000017500000001632513431513133015364 0ustar useruser00000000000000|screencast| .. highlight:: bash What is BorgBackup? ------------------- BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets. See the `installation manual`_ or, if you have already downloaded Borg, ``docs/installation.rst`` to get started with Borg. .. _installation manual: https://borgbackup.readthedocs.org/en/stable/installation.html Main features ~~~~~~~~~~~~~ **Space efficient storage** Deduplication based on content-defined chunking is used to reduce the number of bytes stored: each file is split into a number of variable length chunks and only chunks that have never been seen before are added to the repository. A chunk is considered duplicate if its id_hash value is identical. A cryptographically strong hash or MAC function is used as id_hash, e.g. (hmac-)sha256. To deduplicate, all the chunks in the same repository are considered, no matter whether they come from different machines, from previous backups, from the same backup or even from the same single file. Compared to other deduplication approaches, this method does NOT depend on: * file/directory names staying the same: So you can move your stuff around without killing the deduplication, even between machines sharing a repo. * complete files or time stamps staying the same: If a big file changes a little, only a few new chunks need to be stored - this is great for VMs or raw disks. * The absolute position of a data chunk inside a file: Stuff may get shifted and will still be found by the deduplication algorithm. **Speed** * performance critical code (chunking, compression, encryption) is implemented in C/Cython * local caching of files/chunks index data * quick detection of unmodified files **Data encryption** All data can be protected using 256-bit AES encryption, data integrity and authenticity is verified using HMAC-SHA256. Data is encrypted clientside. **Compression** All data can be compressed by lz4 (super fast, low compression), zlib (medium speed and compression) or lzma (low speed, high compression). **Off-site backups** Borg can store data on any remote host accessible over SSH. If Borg is installed on the remote host, big performance gains can be achieved compared to using a network filesystem (sshfs, nfs, ...). **Backups mountable as filesystems** Backup archives are mountable as userspace filesystems for easy interactive backup examination and restores (e.g. by using a regular file manager). **Easy installation on multiple platforms** We offer single-file binaries that do not require installing anything - you can just run them on these platforms: * Linux * Mac OS X * FreeBSD * OpenBSD and NetBSD (no xattrs/ACLs support or binaries yet) * Cygwin (not supported, no binaries yet) * Linux Subsystem of Windows 10 (not supported) **Free and Open Source Software** * security and functionality can be audited independently * licensed under the BSD (3-clause) license, see `License`_ for the complete license Easy to use ~~~~~~~~~~~ Initialize a new backup repository and create a backup archive:: $ borg init /path/to/repo $ borg create /path/to/repo::Saturday1 ~/Documents Now doing another backup, just to show off the great deduplication:: $ borg create -v --stats /path/to/repo::Saturday2 ~/Documents ----------------------------------------------------------------------------- Archive name: Saturday2 Archive fingerprint: 622b7c53c... Time (start): Sat, 2016-02-27 14:48:13 Time (end): Sat, 2016-02-27 14:48:14 Duration: 0.88 seconds Number of files: 163 ----------------------------------------------------------------------------- Original size Compressed size Deduplicated size This archive: 6.85 MB 6.85 MB 30.79 kB <-- ! All archives: 13.69 MB 13.71 MB 6.88 MB Unique chunks Total chunks Chunk index: 167 330 ----------------------------------------------------------------------------- For a graphical frontend refer to our complementary project `BorgWeb `_. Helping, Donations and Bounties ------------------------------- Your help is always welcome! Spread the word, give feedback, help with documentation, testing or development. You can also give monetary support to the project, see there for details: https://borgbackup.readthedocs.io/en/stable/support.html#bounties-and-fundraisers Links ----- * `Main Web Site `_ * `Releases `_, `PyPI packages `_ and `ChangeLog `_ * `GitHub `_ and `Issue Tracker `_. * `Web-Chat (IRC) `_ and `Mailing List `_ * `License `_ * `Security contact `_ Compatibility notes ------------------- EXPECT THAT WE WILL BREAK COMPATIBILITY REPEATEDLY WHEN MAJOR RELEASE NUMBER CHANGES (like when going from 0.x.y to 1.0.0 or from 1.x.y to 2.0.0). NOT RELEASED DEVELOPMENT VERSIONS HAVE UNKNOWN COMPATIBILITY PROPERTIES. THIS IS SOFTWARE IN DEVELOPMENT, DECIDE YOURSELF WHETHER IT FITS YOUR NEEDS. Security issues should be reported to the `Security contact`_ (or see ``docs/suppport.rst`` in the source distribution). .. start-badges |doc| |build| |coverage| |bestpractices| |bounties| .. |bounties| image:: https://api.bountysource.com/badge/team?team_id=78284&style=bounties_posted :alt: Bounty Source :target: https://www.bountysource.com/teams/borgbackup .. |doc| image:: https://readthedocs.org/projects/borgbackup/badge/?version=stable :alt: Documentation :target: https://borgbackup.readthedocs.org/en/stable/ .. |build| image:: https://api.travis-ci.org/borgbackup/borg.svg :alt: Build Status :target: https://travis-ci.org/borgbackup/borg .. |coverage| image:: https://codecov.io/github/borgbackup/borg/coverage.svg?branch=master :alt: Test Coverage :target: https://codecov.io/github/borgbackup/borg?branch=master .. |screencast| image:: https://asciinema.org/a/28691.png :alt: BorgBackup Installation and Basic Usage :target: https://asciinema.org/a/28691?autoplay=1&speed=2 .. |bestpractices| image:: https://bestpractices.coreinfrastructure.org/projects/271/badge :alt: Best Practices Score :target: https://bestpractices.coreinfrastructure.org/projects/271 .. end-badges borgbackup-1.0.13/borg/0000755000175000017500000000000013431514213014617 5ustar useruser00000000000000borgbackup-1.0.13/borg/__init__.py0000644000175000017500000000023413431513133016727 0ustar useruser00000000000000from distutils.version import LooseVersion from ._version import version as __version__ __version_tuple__ = tuple(LooseVersion(__version__).version[:3]) borgbackup-1.0.13/borg/__main__.py0000644000175000017500000000004613431513133016711 0ustar useruser00000000000000from borg.archiver import main main() borgbackup-1.0.13/borg/_chunker.c0000644000175000017500000002341713431513133016570 0ustar useruser00000000000000#include #include /* Cyclic polynomial / buzhash: https://en.wikipedia.org/wiki/Rolling_hash */ static uint32_t table_base[] = { 0xe7f831ec, 0xf4026465, 0xafb50cae, 0x6d553c7a, 0xd639efe3, 0x19a7b895, 0x9aba5b21, 0x5417d6d4, 0x35fd2b84, 0xd1f6a159, 0x3f8e323f, 0xb419551c, 0xf444cebf, 0x21dc3b80, 0xde8d1e36, 0x84a32436, 0xbeb35a9d, 0xa36f24aa, 0xa4e60186, 0x98d18ffe, 0x3f042f9e, 0xdb228bcd, 0x096474b7, 0x5c20c2f7, 0xf9eec872, 0xe8625275, 0xb9d38f80, 0xd48eb716, 0x22a950b4, 0x3cbaaeaa, 0xc37cddd3, 0x8fea6f6a, 0x1d55d526, 0x7fd6d3b3, 0xdaa072ee, 0x4345ac40, 0xa077c642, 0x8f2bd45b, 0x28509110, 0x55557613, 0xffc17311, 0xd961ffef, 0xe532c287, 0xaab95937, 0x46d38365, 0xb065c703, 0xf2d91d0f, 0x92cd4bb0, 0x4007c712, 0xf35509dd, 0x505b2f69, 0x557ead81, 0x310f4563, 0xbddc5be8, 0x9760f38c, 0x701e0205, 0x00157244, 0x14912826, 0xdc4ca32b, 0x67b196de, 0x5db292e8, 0x8c1b406b, 0x01f34075, 0xfa2520f7, 0x73bc37ab, 0x1e18bc30, 0xfe2c6cb3, 0x20c522d0, 0x5639e3db, 0x942bda35, 0x899af9d1, 0xced44035, 0x98cc025b, 0x255f5771, 0x70fefa24, 0xe928fa4d, 0x2c030405, 0xb9325590, 0x20cb63bd, 0xa166305d, 0x80e52c0a, 0xa8fafe2f, 0x1ad13f7d, 0xcfaf3685, 0x6c83a199, 0x7d26718a, 0xde5dfcd9, 0x79cf7355, 0x8979d7fb, 0xebf8c55e, 0xebe408e4, 0xcd2affba, 0xe483be6e, 0xe239d6de, 0x5dc1e9e0, 0x0473931f, 0x851b097c, 0xac5db249, 0x09c0f9f2, 0xd8d2f134, 0xe6f38e41, 0xb1c71bf1, 0x52b6e4db, 0x07224424, 0x6cf73e85, 0x4f25d89c, 0x782a7d74, 0x10a68dcd, 0x3a868189, 0xd570d2dc, 0x69630745, 0x9542ed86, 0x331cd6b2, 0xa84b5b28, 0x07879c9d, 0x38372f64, 0x7185db11, 0x25ba7c83, 0x01061523, 0xe6792f9f, 0xe5df07d1, 0x4321b47f, 0x7d2469d8, 0x1a3a4f90, 0x48be29a3, 0x669071af, 0x8ec8dd31, 0x0810bfbf, 0x813a06b4, 0x68538345, 0x65865ddc, 0x43a71b8e, 0x78619a56, 0x5a34451d, 0x5bdaa3ed, 0x71edc7e9, 0x17ac9a20, 0x78d10bfa, 0x6c1e7f35, 0xd51839d9, 0x240cbc51, 0x33513cc1, 0xd2b4f795, 0xccaa8186, 0x0babe682, 0xa33cf164, 0x18c643ea, 0xc1ca105f, 0x9959147a, 0x6d3d94de, 0x0b654fbe, 0xed902ca0, 0x7d835cb5, 0x99ba1509, 0x6445c922, 0x495e76c2, 0xf07194bc, 0xa1631d7e, 0x677076a5, 0x89fffe35, 0x1a49bcf3, 0x8e6c948a, 0x0144c917, 0x8d93aea1, 0x16f87ddf, 0xc8f25d49, 0x1fb11297, 0x27e750cd, 0x2f422da1, 0xdee89a77, 0x1534c643, 0x457b7b8b, 0xaf172f7a, 0x6b9b09d6, 0x33573f7f, 0xf14e15c4, 0x526467d5, 0xaf488241, 0x87c3ee0d, 0x33be490c, 0x95aa6e52, 0x43ec242e, 0xd77de99b, 0xd018334f, 0x5b78d407, 0x498eb66b, 0xb1279fa8, 0xb38b0ea6, 0x90718376, 0xe325dee2, 0x8e2f2cba, 0xcaa5bdec, 0x9d652c56, 0xad68f5cb, 0xa77591af, 0x88e37ee8, 0xf8faa221, 0xfcbbbe47, 0x4f407786, 0xaf393889, 0xf444a1d9, 0x15ae1a2f, 0x40aa7097, 0x6f9486ac, 0x29d232a3, 0xe47609e9, 0xe8b631ff, 0xba8565f4, 0x11288749, 0x46c9a838, 0xeb1b7cd8, 0xf516bbb1, 0xfb74fda0, 0x010996e6, 0x4c994653, 0x1d889512, 0x53dcd9a3, 0xdd074697, 0x1e78e17c, 0x637c98bf, 0x930bb219, 0xcf7f75b0, 0xcb9355fb, 0x9e623009, 0xe466d82c, 0x28f968d3, 0xfeb385d9, 0x238e026c, 0xb8ed0560, 0x0c6a027a, 0x3d6fec4b, 0xbb4b2ec2, 0xe715031c, 0xeded011d, 0xcdc4d3b9, 0xc456fc96, 0xdd0eea20, 0xb3df8ec9, 0x12351993, 0xd9cbb01c, 0x603147a2, 0xcf37d17d, 0xf7fcd9dc, 0xd8556fa3, 0x104c8131, 0x13152774, 0xb4715811, 0x6a72c2c9, 0xc5ae37bb, 0xa76ce12a, 0x8150d8f3, 0x2ec29218, 0xa35f0984, 0x48c0647e, 0x0b5ff98c, 0x71893f7b }; #define BARREL_SHIFT(v, shift) ( ((v) << (shift)) | ((v) >> ((32 - (shift)) & 0x1f)) ) size_t pagemask; static uint32_t * buzhash_init_table(uint32_t seed) { int i; uint32_t *table = malloc(1024); for(i = 0; i < 256; i++) { table[i] = table_base[i] ^ seed; } return table; } static uint32_t buzhash(const unsigned char *data, size_t len, const uint32_t *h) { uint32_t i; uint32_t sum = 0, imod; for(i = len - 1; i > 0; i--) { imod = i & 0x1f; sum ^= BARREL_SHIFT(h[*data], imod); data++; } return sum ^ h[*data]; } static uint32_t buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, const uint32_t *h) { uint32_t lenmod = len & 0x1f; return BARREL_SHIFT(sum, 1) ^ BARREL_SHIFT(h[remove], lenmod) ^ h[add]; } typedef struct { uint32_t chunk_mask; uint32_t *table; uint8_t *data; PyObject *fd; int fh; int done, eof; size_t min_size, buf_size, window_size, remaining, position, last; off_t bytes_read, bytes_yielded; } Chunker; static Chunker * chunker_init(size_t window_size, uint32_t chunk_mask, size_t min_size, size_t max_size, uint32_t seed) { Chunker *c = calloc(sizeof(Chunker), 1); c->window_size = window_size; c->chunk_mask = chunk_mask; c->min_size = min_size; c->table = buzhash_init_table(seed); c->buf_size = max_size; c->data = malloc(c->buf_size); c->fh = -1; return c; } static void chunker_set_fd(Chunker *c, PyObject *fd, int fh) { Py_XDECREF(c->fd); c->fd = fd; Py_INCREF(fd); c->fh = fh; c->done = 0; c->remaining = 0; c->bytes_read = 0; c->bytes_yielded = 0; c->position = 0; c->last = 0; c->eof = 0; } static void chunker_free(Chunker *c) { Py_XDECREF(c->fd); free(c->table); free(c->data); free(c); } static int chunker_fill(Chunker *c) { ssize_t n; off_t offset, length; int overshoot; PyObject *data; memmove(c->data, c->data + c->last, c->position + c->remaining - c->last); c->position -= c->last; c->last = 0; n = c->buf_size - c->position - c->remaining; if(c->eof || n == 0) { return 1; } if(c->fh >= 0) { offset = c->bytes_read; // if we have a os-level file descriptor, use os-level API n = read(c->fh, c->data + c->position + c->remaining, n); if(n > 0) { c->remaining += n; c->bytes_read += n; } else if(n == 0) { c->eof = 1; } else { // some error happened PyErr_SetFromErrno(PyExc_OSError); return 0; } length = c->bytes_read - offset; #if ( ( _XOPEN_SOURCE >= 600 || _POSIX_C_SOURCE >= 200112L ) && defined(POSIX_FADV_DONTNEED) ) // Only do it once per run. if (pagemask == 0) pagemask = getpagesize() - 1; // We tell the OS that we do not need the data that we just have read any // more (that it maybe has in the cache). This avoids that we spoil the // complete cache with data that we only read once and (due to cache // size limit) kick out data from the cache that might be still useful // for the OS or other processes. // We rollback the initial offset back to the start of the page, // to avoid it not being truncated as a partial page request. if (length > 0) { // Linux kernels prior to 4.7 have a bug where they truncate // last partial page of POSIX_FADV_DONTNEED request, so we need // to page-align it ourselves. We'll need the rest of this page // on the next read (assuming this was not EOF) overshoot = (offset + length) & pagemask; } else { // For length == 0 we set overshoot 0, so the below // length - overshoot is 0, which means till end of file for // fadvise. This will cancel the final page and is not part // of the above workaround. overshoot = 0; } posix_fadvise(c->fh, offset & ~pagemask, length - overshoot, POSIX_FADV_DONTNEED); #endif } else { // no os-level file descriptor, use Python file object API data = PyObject_CallMethod(c->fd, "read", "i", n); if(!data) { return 0; } n = PyBytes_Size(data); if(PyErr_Occurred()) { // we wanted bytes(), but got something else return 0; } if(n) { memcpy(c->data + c->position + c->remaining, PyBytes_AsString(data), n); c->remaining += n; c->bytes_read += n; } else { c->eof = 1; } Py_DECREF(data); } return 1; } static PyObject * chunker_process(Chunker *c) { uint32_t sum, chunk_mask = c->chunk_mask; size_t n = 0, old_last, min_size = c->min_size, window_size = c->window_size; if(c->done) { if(c->bytes_read == c->bytes_yielded) PyErr_SetNone(PyExc_StopIteration); else PyErr_SetString(PyExc_Exception, "chunkifier byte count mismatch"); return NULL; } while(c->remaining <= window_size && !c->eof) { if(!chunker_fill(c)) { return NULL; } } if(c->eof) { c->done = 1; if(c->remaining) { c->bytes_yielded += c->remaining; return PyMemoryView_FromMemory((char *)(c->data + c->position), c->remaining, PyBUF_READ); } else { if(c->bytes_read == c->bytes_yielded) PyErr_SetNone(PyExc_StopIteration); else PyErr_SetString(PyExc_Exception, "chunkifier byte count mismatch"); return NULL; } } sum = buzhash(c->data + c->position, window_size, c->table); while(c->remaining > c->window_size && ((sum & chunk_mask) || n < min_size)) { sum = buzhash_update(sum, c->data[c->position], c->data[c->position + window_size], window_size, c->table); c->position++; c->remaining--; n++; if(c->remaining <= window_size) { if(!chunker_fill(c)) { return NULL; } } } if(c->remaining <= window_size) { c->position += c->remaining; c->remaining = 0; } old_last = c->last; c->last = c->position; n = c->last - old_last; c->bytes_yielded += n; return PyMemoryView_FromMemory((char *)(c->data + old_last), n, PyBUF_READ); } borgbackup-1.0.13/borg/_hashindex.c0000644000175000017500000003662013431513133017104 0ustar useruser00000000000000#include #include #include #include #include #include #include #include #include #if defined (__SVR4) && defined (__sun) #include #endif #if (defined(BYTE_ORDER) && (BYTE_ORDER == BIG_ENDIAN)) || \ (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_BIG_ENDIAN__)) || \ (defined(_BIG_ENDIAN) && defined(__SVR4)&&defined(__sun)) #define BORG_BIG_ENDIAN 1 #elif (defined(BYTE_ORDER) && (BYTE_ORDER == LITTLE_ENDIAN)) || \ (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__)) || \ (defined(_LITTLE_ENDIAN) && defined(__SVR4)&&defined(__sun)) #define BORG_BIG_ENDIAN 0 #else #error Unknown byte order #endif #if BORG_BIG_ENDIAN #define _le32toh(x) __builtin_bswap32(x) #define _htole32(x) __builtin_bswap32(x) #else #define _le32toh(x) (x) #define _htole32(x) (x) #endif #define MAGIC "BORG_IDX" #define MAGIC_LEN 8 typedef struct { char magic[MAGIC_LEN]; int32_t num_entries; int32_t num_buckets; int8_t key_size; int8_t value_size; } __attribute__((__packed__)) HashHeader; typedef struct { void *buckets; int num_entries; int num_buckets; int num_empty; int key_size; int value_size; off_t bucket_size; int lower_limit; int upper_limit; int min_empty; } HashIndex; /* prime (or w/ big prime factors) hash table sizes * not sure we need primes for borg's usage (as we have a hash function based * on sha256, we can assume an even, seemingly random distribution of values), * but OTOH primes don't harm. * also, growth of the sizes starts with fast-growing 2x steps, but slows down * more and more down to 1.1x. this is to avoid huge jumps in memory allocation, * like e.g. 4G -> 8G. * these values are generated by hash_sizes.py. */ static int hash_sizes[] = { 1031, 2053, 4099, 8209, 16411, 32771, 65537, 131101, 262147, 445649, 757607, 1287917, 2189459, 3065243, 4291319, 6007867, 8410991, 11775359, 16485527, 23079703, 27695653, 33234787, 39881729, 47858071, 57429683, 68915617, 82698751, 99238507, 119086189, 144378011, 157223263, 173476439, 190253911, 209915011, 230493629, 253169431, 278728861, 306647623, 337318939, 370742809, 408229973, 449387209, 493428073, 543105119, 596976533, 657794869, 722676499, 795815791, 874066969, 962279771, 1057701643, 1164002657, 1280003147, 1407800297, 1548442699, 1703765389, 1873768367, 2062383853, /* 32bit int ends about here */ }; #define HASH_MIN_LOAD .25 #define HASH_MAX_LOAD .75 /* don't go higher than 0.75, otherwise performance severely suffers! */ #define HASH_MAX_EFF_LOAD .93 #define MAX(x, y) ((x) > (y) ? (x): (y)) #define NELEMS(x) (sizeof(x) / sizeof((x)[0])) #define EMPTY _htole32(0xffffffff) #define DELETED _htole32(0xfffffffe) #define BUCKET_ADDR(index, idx) (index->buckets + ((idx) * index->bucket_size)) #define BUCKET_MATCHES_KEY(index, idx, key) (memcmp(key, BUCKET_ADDR(index, idx), index->key_size) == 0) #define BUCKET_IS_DELETED(index, idx) (*((uint32_t *)(BUCKET_ADDR(index, idx) + index->key_size)) == DELETED) #define BUCKET_IS_EMPTY(index, idx) (*((uint32_t *)(BUCKET_ADDR(index, idx) + index->key_size)) == EMPTY) #define BUCKET_MARK_DELETED(index, idx) (*((uint32_t *)(BUCKET_ADDR(index, idx) + index->key_size)) = DELETED) #define BUCKET_MARK_EMPTY(index, idx) (*((uint32_t *)(BUCKET_ADDR(index, idx) + index->key_size)) = EMPTY) #define EPRINTF_MSG(msg, ...) fprintf(stderr, "hashindex: " msg "\n", ##__VA_ARGS__) #define EPRINTF_MSG_PATH(path, msg, ...) fprintf(stderr, "hashindex: %s: " msg "\n", path, ##__VA_ARGS__) #define EPRINTF(msg, ...) fprintf(stderr, "hashindex: " msg "(%s)\n", ##__VA_ARGS__, strerror(errno)) #define EPRINTF_PATH(path, msg, ...) fprintf(stderr, "hashindex: %s: " msg " (%s)\n", path, ##__VA_ARGS__, strerror(errno)) static HashIndex *hashindex_read(const char *path); static int hashindex_write(HashIndex *index, const char *path); static HashIndex *hashindex_init(int capacity, int key_size, int value_size); static const void *hashindex_get(HashIndex *index, const void *key); static int hashindex_set(HashIndex *index, const void *key, const void *value); static int hashindex_delete(HashIndex *index, const void *key); static void *hashindex_next_key(HashIndex *index, const void *key); /* Private API */ static void hashindex_free(HashIndex *index); static int hashindex_index(HashIndex *index, const void *key) { return _le32toh(*((uint32_t *)key)) % index->num_buckets; } static int hashindex_lookup(HashIndex *index, const void *key) { int didx = -1; int start = hashindex_index(index, key); int idx = start; for(;;) { if(BUCKET_IS_EMPTY(index, idx)) { return -1; } if(BUCKET_IS_DELETED(index, idx)) { if(didx == -1) { didx = idx; } } else if(BUCKET_MATCHES_KEY(index, idx, key)) { if (didx != -1) { memcpy(BUCKET_ADDR(index, didx), BUCKET_ADDR(index, idx), index->bucket_size); BUCKET_MARK_DELETED(index, idx); idx = didx; } return idx; } idx++; if (idx >= index->num_buckets) { idx -= index->num_buckets; } if(idx == start) { return -1; } } } static int hashindex_resize(HashIndex *index, int capacity) { HashIndex *new; void *key = NULL; int32_t key_size = index->key_size; if(!(new = hashindex_init(capacity, key_size, index->value_size))) { return 0; } while((key = hashindex_next_key(index, key))) { if(!hashindex_set(new, key, key + key_size)) { /* This can only happen if there's a bug in the code calculating capacity */ hashindex_free(new); return 0; } } free(index->buckets); index->buckets = new->buckets; index->num_buckets = new->num_buckets; index->num_empty = index->num_buckets - index->num_entries; index->lower_limit = new->lower_limit; index->upper_limit = new->upper_limit; index->min_empty = new->min_empty; free(new); return 1; } int get_lower_limit(int num_buckets){ int min_buckets = hash_sizes[0]; if (num_buckets <= min_buckets) return 0; return (int)(num_buckets * HASH_MIN_LOAD); } int get_upper_limit(int num_buckets){ int max_buckets = hash_sizes[NELEMS(hash_sizes) - 1]; if (num_buckets >= max_buckets) return num_buckets; return (int)(num_buckets * HASH_MAX_LOAD); } int get_min_empty(int num_buckets){ /* Differently from load, the effective load also considers tombstones (deleted buckets). */ return (int)(num_buckets * (1.0 - HASH_MAX_EFF_LOAD)); } int size_idx(int size){ /* find the hash_sizes index with entry >= size */ int elems = NELEMS(hash_sizes); int entry, i=0; do{ entry = hash_sizes[i++]; }while((entry < size) && (i < elems)); if (i >= elems) return elems - 1; i--; return i; } int fit_size(int current){ int i = size_idx(current); return hash_sizes[i]; } int grow_size(int current){ int i = size_idx(current) + 1; int elems = NELEMS(hash_sizes); if (i >= elems) return hash_sizes[elems - 1]; return hash_sizes[i]; } int shrink_size(int current){ int i = size_idx(current) - 1; if (i < 0) return hash_sizes[0]; return hash_sizes[i]; } int count_empty(HashIndex *index) { /* count empty (never used) buckets. this does NOT include deleted buckets (tombstones). * TODO: if we ever change HashHeader, save the count there so we do not need this function. */ int i, count = 0, capacity = index->num_buckets; for(i = 0; i < capacity; i++) { if(BUCKET_IS_EMPTY(index, i)) count++; } return count; } /* Public API */ static HashIndex * hashindex_read(const char *path) { FILE *fd; off_t length, buckets_length, bytes_read; HashHeader header; HashIndex *index = NULL; if((fd = fopen(path, "rb")) == NULL) { EPRINTF_PATH(path, "fopen for reading failed"); return NULL; } bytes_read = fread(&header, 1, sizeof(HashHeader), fd); if(bytes_read != sizeof(HashHeader)) { if(ferror(fd)) { EPRINTF_PATH(path, "fread header failed (expected %ju, got %ju)", (uintmax_t) sizeof(HashHeader), (uintmax_t) bytes_read); } else { EPRINTF_MSG_PATH(path, "fread header failed (expected %ju, got %ju)", (uintmax_t) sizeof(HashHeader), (uintmax_t) bytes_read); } goto fail; } if(fseek(fd, 0, SEEK_END) < 0) { EPRINTF_PATH(path, "fseek failed"); goto fail; } if((length = ftell(fd)) < 0) { EPRINTF_PATH(path, "ftell failed"); goto fail; } if(fseek(fd, sizeof(HashHeader), SEEK_SET) < 0) { EPRINTF_PATH(path, "fseek failed"); goto fail; } if(memcmp(header.magic, MAGIC, MAGIC_LEN)) { EPRINTF_MSG_PATH(path, "Unknown MAGIC in header"); goto fail; } buckets_length = (off_t)_le32toh(header.num_buckets) * (header.key_size + header.value_size); if((size_t) length != sizeof(HashHeader) + buckets_length) { EPRINTF_MSG_PATH(path, "Incorrect file length (expected %ju, got %ju)", (uintmax_t) sizeof(HashHeader) + buckets_length, (uintmax_t) length); goto fail; } if(!(index = malloc(sizeof(HashIndex)))) { EPRINTF_PATH(path, "malloc header failed"); goto fail; } if(!(index->buckets = malloc(buckets_length))) { EPRINTF_PATH(path, "malloc buckets failed"); free(index); index = NULL; goto fail; } bytes_read = fread(index->buckets, 1, buckets_length, fd); if(bytes_read != buckets_length) { if(ferror(fd)) { EPRINTF_PATH(path, "fread buckets failed (expected %ju, got %ju)", (uintmax_t) buckets_length, (uintmax_t) bytes_read); } else { EPRINTF_MSG_PATH(path, "fread buckets failed (expected %ju, got %ju)", (uintmax_t) buckets_length, (uintmax_t) bytes_read); } free(index->buckets); free(index); index = NULL; goto fail; } index->num_entries = _le32toh(header.num_entries); index->num_buckets = _le32toh(header.num_buckets); index->key_size = header.key_size; index->value_size = header.value_size; index->bucket_size = index->key_size + index->value_size; index->lower_limit = get_lower_limit(index->num_buckets); index->upper_limit = get_upper_limit(index->num_buckets); index->min_empty = get_min_empty(index->num_buckets); index->num_empty = count_empty(index); if(index->num_empty < index->min_empty) { /* too many tombstones here / not enough empty buckets, do a same-size rebuild */ if(!hashindex_resize(index, index->num_buckets)) { free(index->buckets); free(index); index = NULL; goto fail; } } fail: if(fclose(fd) < 0) { EPRINTF_PATH(path, "fclose failed"); } return index; } static HashIndex * hashindex_init(int capacity, int key_size, int value_size) { HashIndex *index; int i; capacity = fit_size(capacity); if(!(index = malloc(sizeof(HashIndex)))) { EPRINTF("malloc header failed"); return NULL; } if(!(index->buckets = calloc(capacity, key_size + value_size))) { EPRINTF("malloc buckets failed"); free(index); return NULL; } index->num_entries = 0; index->key_size = key_size; index->value_size = value_size; index->num_buckets = capacity; index->num_empty = capacity; index->bucket_size = index->key_size + index->value_size; index->lower_limit = get_lower_limit(index->num_buckets); index->upper_limit = get_upper_limit(index->num_buckets); index->min_empty = get_min_empty(index->num_buckets); for(i = 0; i < capacity; i++) { BUCKET_MARK_EMPTY(index, i); } return index; } static void hashindex_free(HashIndex *index) { free(index->buckets); free(index); } static int hashindex_write(HashIndex *index, const char *path) { off_t buckets_length = (off_t)index->num_buckets * index->bucket_size; FILE *fd; HashHeader header = { .magic = MAGIC, .num_entries = _htole32(index->num_entries), .num_buckets = _htole32(index->num_buckets), .key_size = index->key_size, .value_size = index->value_size }; int ret = 1; if((fd = fopen(path, "wb")) == NULL) { EPRINTF_PATH(path, "fopen for writing failed"); return 0; } if(fwrite(&header, 1, sizeof(header), fd) != sizeof(header)) { EPRINTF_PATH(path, "fwrite header failed"); ret = 0; } if(fwrite(index->buckets, 1, buckets_length, fd) != (size_t) buckets_length) { EPRINTF_PATH(path, "fwrite buckets failed"); ret = 0; } if(fclose(fd) < 0) { EPRINTF_PATH(path, "fclose failed"); } return ret; } static const void * hashindex_get(HashIndex *index, const void *key) { int idx = hashindex_lookup(index, key); if(idx < 0) { return NULL; } return BUCKET_ADDR(index, idx) + index->key_size; } static int hashindex_set(HashIndex *index, const void *key, const void *value) { int idx = hashindex_lookup(index, key); uint8_t *ptr; if(idx < 0) { if(index->num_entries > index->upper_limit) { if(!hashindex_resize(index, grow_size(index->num_buckets))) { return 0; } } idx = hashindex_index(index, key); while(!BUCKET_IS_EMPTY(index, idx) && !BUCKET_IS_DELETED(index, idx)) { idx++; if (idx >= index->num_buckets){ idx -= index->num_buckets; } } if(BUCKET_IS_EMPTY(index, idx)){ index->num_empty--; if(index->num_empty < index->min_empty) { /* too many tombstones here / not enough empty buckets, do a same-size rebuild */ if(!hashindex_resize(index, index->num_buckets)) { return 0; } } } ptr = BUCKET_ADDR(index, idx); memcpy(ptr, key, index->key_size); memcpy(ptr + index->key_size, value, index->value_size); index->num_entries += 1; } else { memcpy(BUCKET_ADDR(index, idx) + index->key_size, value, index->value_size); } return 1; } static int hashindex_delete(HashIndex *index, const void *key) { int idx = hashindex_lookup(index, key); if (idx < 0) { return 1; } BUCKET_MARK_DELETED(index, idx); index->num_entries -= 1; if(index->num_entries < index->lower_limit) { if(!hashindex_resize(index, shrink_size(index->num_buckets))) { return 0; } } return 1; } static void * hashindex_next_key(HashIndex *index, const void *key) { int idx = 0; if(key) { idx = 1 + (key - index->buckets) / index->bucket_size; } if (idx == index->num_buckets) { return NULL; } while(BUCKET_IS_EMPTY(index, idx) || BUCKET_IS_DELETED(index, idx)) { idx ++; if (idx == index->num_buckets) { return NULL; } } return BUCKET_ADDR(index, idx); } static int hashindex_get_size(HashIndex *index) { return index->num_entries; } borgbackup-1.0.13/borg/_version.py0000644000175000017500000000016513431514212017016 0ustar useruser00000000000000# coding: utf-8 # file generated by setuptools_scm # don't change, don't track in version control version = '1.0.13' borgbackup-1.0.13/borg/archive.py0000644000175000017500000014457213431513133016627 0ustar useruser00000000000000from contextlib import contextmanager from datetime import datetime, timezone, timedelta from getpass import getuser from itertools import groupby import errno from .logger import create_logger logger = create_logger() from .key import key_factory from .remote import cache_if_remote import os import socket import stat import sys import time from io import BytesIO from . import xattr from .helpers import Error, uid2user, user2uid, gid2group, group2gid, bin_to_hex, \ parse_timestamp, to_localtime, ISO_FORMAT, format_time, format_timedelta, remove_surrogates, \ Manifest, Statistics, decode_dict, make_path_safe, StableDict, int_to_bigint, bigint_to_int, \ ProgressIndicatorPercent, IntegrityError, set_ec, EXIT_WARNING, safe_ns from .platform import acl_get, acl_set from .chunker import Chunker from .hashindex import ChunkIndex from .repository import Repository, LIST_SCAN_LIMIT import msgpack ITEMS_BUFFER = 1024 * 1024 CHUNK_MIN_EXP = 19 # 2**19 == 512kiB CHUNK_MAX_EXP = 23 # 2**23 == 8MiB HASH_WINDOW_SIZE = 0xfff # 4095B HASH_MASK_BITS = 21 # results in ~2MiB chunks statistically # defaults, use --chunker-params to override CHUNKER_PARAMS = (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, HASH_WINDOW_SIZE) # chunker params for the items metadata stream, finer granularity ITEMS_CHUNKER_PARAMS = (15, 19, 17, HASH_WINDOW_SIZE) has_lchmod = hasattr(os, 'lchmod') has_lchflags = hasattr(os, 'lchflags') flags_normal = os.O_RDONLY | getattr(os, 'O_BINARY', 0) flags_noatime = flags_normal | getattr(os, 'O_NOATIME', 0) def is_special(mode): # file types that get special treatment in --read-special mode return stat.S_ISBLK(mode) or stat.S_ISCHR(mode) or stat.S_ISFIFO(mode) class BackupOSError(Exception): """ Wrapper for OSError raised while accessing backup files. Borg does different kinds of IO, and IO failures have different consequences. This wrapper represents failures of input file or extraction IO. These are non-critical and are only reported (exit code = 1, warning). Any unwrapped IO error is critical and aborts execution (for example repository IO failure). """ def __init__(self, os_error): self.os_error = os_error self.errno = os_error.errno self.strerror = os_error.strerror self.filename = os_error.filename def __str__(self): return str(self.os_error) @contextmanager def backup_io(): """Context manager changing OSError to BackupOSError.""" try: yield except OSError as os_error: raise BackupOSError(os_error) from os_error def backup_io_iter(iterator): while True: try: with backup_io(): item = next(iterator) except StopIteration: return yield item class DownloadPipeline: def __init__(self, repository, key): self.repository = repository self.key = key def unpack_many(self, ids, filter=None, preload=False): """ Return iterator of items. *ids* is a chunk ID list of an item stream. *filter* is a callable to decide whether an item will be yielded. *preload* preloads the data chunks of every yielded item. Warning: if *preload* is True then all data chunks of every yielded item have to be retrieved, otherwise preloaded chunks will accumulate in RemoteRepository and create a memory leak. """ unpacker = msgpack.Unpacker(use_list=False) for data in self.fetch_many(ids): unpacker.feed(data) items = [decode_dict(item, (b'path', b'source', b'user', b'group')) for item in unpacker] if filter: items = [item for item in items if filter(item)] if preload: for item in items: if b'chunks' in item: self.repository.preload([c[0] for c in item[b'chunks']]) for item in items: yield item def fetch_many(self, ids, is_preloaded=False): for id_, data in zip(ids, self.repository.get_many(ids, is_preloaded=is_preloaded)): yield self.key.decrypt(id_, data) class ChunkBuffer: BUFFER_SIZE = 1 * 1024 * 1024 def __init__(self, key, chunker_params=ITEMS_CHUNKER_PARAMS): self.buffer = BytesIO() self.packer = msgpack.Packer(unicode_errors='surrogateescape') self.chunks = [] self.key = key self.chunker = Chunker(self.key.chunk_seed, *chunker_params) def add(self, item): self.buffer.write(self.packer.pack(StableDict(item))) if self.is_full(): self.flush() def write_chunk(self, chunk): raise NotImplementedError def flush(self, flush=False): if self.buffer.tell() == 0: return self.buffer.seek(0) chunks = list(bytes(s) for s in self.chunker.chunkify(self.buffer)) self.buffer.seek(0) self.buffer.truncate(0) # Leave the last partial chunk in the buffer unless flush is True end = None if flush or len(chunks) == 1 else -1 for chunk in chunks[:end]: self.chunks.append(self.write_chunk(chunk)) if end == -1: self.buffer.write(chunks[-1]) def is_full(self): return self.buffer.tell() > self.BUFFER_SIZE class CacheChunkBuffer(ChunkBuffer): def __init__(self, cache, key, stats, chunker_params=ITEMS_CHUNKER_PARAMS): super().__init__(key, chunker_params) self.cache = cache self.stats = stats def write_chunk(self, chunk): id_, _, _ = self.cache.add_chunk(self.key.id_hash(chunk), chunk, self.stats) return id_ class Archive: class DoesNotExist(Error): """Archive {} does not exist""" class AlreadyExists(Error): """Archive {} already exists""" class IncompatibleFilesystemEncodingError(Error): """Failed to encode filename "{}" into file system encoding "{}". Consider configuring the LANG environment variable.""" def __init__(self, repository, key, manifest, name, cache=None, create=False, checkpoint_interval=1800, numeric_owner=False, noatime=False, noctime=False, progress=False, chunker_params=CHUNKER_PARAMS, start=None, start_monotonic=None, end=None): self.cwd = os.getcwd() self.key = key self.repository = repository self.cache = cache self.manifest = manifest self.hard_links = {} self.stats = Statistics() self.show_progress = progress self.name = name self.checkpoint_interval = checkpoint_interval self.numeric_owner = numeric_owner self.noatime = noatime self.noctime = noctime assert (start is None) == (start_monotonic is None), 'Logic error: if start is given, start_monotonic must be given as well and vice versa.' if start is None: start = datetime.utcnow() start_monotonic = time.monotonic() self.chunker_params = chunker_params self.start = start self.start_monotonic = start_monotonic if end is None: end = datetime.utcnow() self.end = end self.pipeline = DownloadPipeline(self.repository, self.key) if create: self.items_buffer = CacheChunkBuffer(self.cache, self.key, self.stats) self.chunker = Chunker(self.key.chunk_seed, *chunker_params) if name in manifest.archives: raise self.AlreadyExists(name) self.last_checkpoint = time.monotonic() i = 0 while True: self.checkpoint_name = '%s.checkpoint%s' % (name, i and ('.%d' % i) or '') if self.checkpoint_name not in manifest.archives: break i += 1 else: if name not in self.manifest.archives: raise self.DoesNotExist(name) info = self.manifest.archives[name] self.load(info[b'id']) self.zeros = None def _load_meta(self, id): data = self.key.decrypt(id, self.repository.get(id)) metadata = msgpack.unpackb(data, unicode_errors='surrogateescape') if metadata[b'version'] != 1: raise Exception('Unknown archive metadata version') return metadata def load(self, id): self.id = id self.metadata = self._load_meta(self.id) decode_dict(self.metadata, (b'name', b'hostname', b'username', b'time', b'time_end')) self.metadata[b'cmdline'] = [arg.decode('utf-8', 'surrogateescape') for arg in self.metadata[b'cmdline']] self.name = self.metadata[b'name'] @property def ts(self): """Timestamp of archive creation (start) in UTC""" ts = self.metadata[b'time'] return parse_timestamp(ts) @property def ts_end(self): """Timestamp of archive creation (end) in UTC""" # fall back to time if there is no time_end present in metadata ts = self.metadata.get(b'time_end') or self.metadata[b'time'] return parse_timestamp(ts) @property def fpr(self): return bin_to_hex(self.id) @property def duration(self): return format_timedelta(self.end - self.start) def __str__(self): return '''\ Archive name: {0.name} Archive fingerprint: {0.fpr} Time (start): {start} Time (end): {end} Duration: {0.duration} Number of files: {0.stats.nfiles}'''.format( self, start=format_time(to_localtime(self.start.replace(tzinfo=timezone.utc))), end=format_time(to_localtime(self.end.replace(tzinfo=timezone.utc)))) def __repr__(self): return 'Archive(%r)' % self.name def iter_items(self, filter=None, preload=False): for item in self.pipeline.unpack_many(self.metadata[b'items'], filter=filter, preload=preload): yield item def add_item(self, item): unknown_keys = set(item) - self.manifest.item_keys assert not unknown_keys, ('unknown item metadata keys detected, please update ITEM_KEYS: %s', ','.join(k.decode('ascii') for k in unknown_keys)) if self.show_progress: self.stats.show_progress(item=item, dt=0.2) self.items_buffer.add(item) if time.monotonic() - self.last_checkpoint > self.checkpoint_interval: self.write_checkpoint() self.last_checkpoint = time.monotonic() def write_checkpoint(self): self.save(self.checkpoint_name) del self.manifest.archives[self.checkpoint_name] self.cache.chunk_decref(self.id, self.stats) def save(self, name=None, timestamp=None): name = name or self.name if name in self.manifest.archives: raise self.AlreadyExists(name) self.items_buffer.flush(flush=True) duration = timedelta(seconds=time.monotonic() - self.start_monotonic) if timestamp is None: end = datetime.utcnow() start = end - duration else: end = timestamp + duration start = timestamp self.start = start self.end = end metadata = StableDict({ 'version': 1, 'name': name, 'items': self.items_buffer.chunks, 'cmdline': sys.argv, 'hostname': socket.gethostname(), 'username': getuser(), 'time': start.strftime(ISO_FORMAT), 'time_end': end.strftime(ISO_FORMAT), }) data = self.key.pack_and_authenticate_metadata(metadata, context=b'archive') self.id = self.key.id_hash(data) self.cache.add_chunk(self.id, data, self.stats) self.manifest.archives[name] = {'id': self.id, 'time': metadata['time']} self.manifest.write() self.repository.commit() self.cache.commit() def calc_stats(self, cache): def add(id): count, size, csize = cache.chunks[id] stats.update(size, csize, count == 1) cache.chunks[id] = count - 1, size, csize def add_file_chunks(chunks): for id, _, _ in chunks: add(id) # This function is a bit evil since it abuses the cache to calculate # the stats. The cache transaction must be rolled back afterwards unpacker = msgpack.Unpacker(use_list=False) cache.begin_txn() stats = Statistics() add(self.id) for id, chunk in zip(self.metadata[b'items'], self.repository.get_many(self.metadata[b'items'])): add(id) unpacker.feed(self.key.decrypt(id, chunk)) for item in unpacker: if b'chunks' in item: stats.nfiles += 1 add_file_chunks(item[b'chunks']) cache.rollback() return stats def extract_item(self, item, restore_attrs=True, dry_run=False, stdout=False, sparse=False): has_damaged_chunks = b'chunks_healthy' in item if dry_run or stdout: if b'chunks' in item: for data in self.pipeline.fetch_many([c[0] for c in item[b'chunks']], is_preloaded=True): if stdout: sys.stdout.buffer.write(data) if stdout: sys.stdout.buffer.flush() if has_damaged_chunks: logger.warning('File %s has damaged (all-zero) chunks. Try running borg check --repair.' % remove_surrogates(item[b'path'])) return dest = self.cwd if item[b'path'].startswith(('/', '../')): raise Exception('Path should be relative and local') path = os.path.join(dest, item[b'path']) # Attempt to remove existing files, ignore errors on failure try: st = os.stat(path, follow_symlinks=False) if stat.S_ISDIR(st.st_mode): os.rmdir(path) else: os.unlink(path) except UnicodeEncodeError: raise self.IncompatibleFilesystemEncodingError(path, sys.getfilesystemencoding()) from None except OSError: pass def make_parent(path): parent_dir = os.path.dirname(path) if not os.path.exists(parent_dir): os.makedirs(parent_dir) mode = item[b'mode'] if stat.S_ISREG(mode): with backup_io(): make_parent(path) # Hard link? if b'source' in item: source = os.path.join(dest, item[b'source']) with backup_io(): os.link(source, path) else: if sparse and self.zeros is None: self.zeros = b'\0' * (1 << self.chunker_params[1]) with backup_io(): fd = open(path, 'wb') with fd: ids = [c[0] for c in item[b'chunks']] for data in self.pipeline.fetch_many(ids, is_preloaded=True): with backup_io(): if sparse and self.zeros.startswith(data): # all-zero chunk: create a hole in a sparse file fd.seek(len(data), 1) else: fd.write(data) with backup_io(): pos = fd.tell() fd.truncate(pos) fd.flush() self.restore_attrs(path, item, fd=fd.fileno()) if has_damaged_chunks: logger.warning('File %s has damaged (all-zero) chunks. Try running borg check --repair.' % remove_surrogates(item[b'path'])) return with backup_io(): # No repository access beyond this point. if stat.S_ISDIR(mode): make_parent(path) if not os.path.exists(path): os.mkdir(path) if restore_attrs: self.restore_attrs(path, item) elif stat.S_ISLNK(mode): make_parent(path) source = item[b'source'] try: os.symlink(source, path) except UnicodeEncodeError: raise self.IncompatibleFilesystemEncodingError(source, sys.getfilesystemencoding()) from None self.restore_attrs(path, item, symlink=True) elif stat.S_ISFIFO(mode): make_parent(path) os.mkfifo(path) self.restore_attrs(path, item) elif stat.S_ISCHR(mode) or stat.S_ISBLK(mode): make_parent(path) os.mknod(path, item[b'mode'], item[b'rdev']) self.restore_attrs(path, item) else: raise Exception('Unknown archive item type %r' % item[b'mode']) def restore_attrs(self, path, item, symlink=False, fd=None): """ Restore filesystem attributes on *path* (*fd*) from *item*. Does not access the repository. """ uid = gid = None if not self.numeric_owner: uid = user2uid(item[b'user']) gid = group2gid(item[b'group']) uid = item[b'uid'] if uid is None else uid gid = item[b'gid'] if gid is None else gid # This code is a bit of a mess due to os specific differences try: if fd: os.fchown(fd, uid, gid) else: os.chown(path, uid, gid, follow_symlinks=False) except OSError: pass if fd: os.fchmod(fd, item[b'mode']) elif not symlink: os.chmod(path, item[b'mode']) elif has_lchmod: # Not available on Linux os.lchmod(path, item[b'mode']) mtime = bigint_to_int(item[b'mtime']) if b'atime' in item: atime = bigint_to_int(item[b'atime']) else: # old archives only had mtime in item metadata atime = mtime if fd: os.utime(fd, None, ns=(atime, mtime)) else: os.utime(path, None, ns=(atime, mtime), follow_symlinks=False) acl_set(path, item, self.numeric_owner) # chown removes Linux capabilities, so set the extended attributes at the end, after chown, since they include # the Linux capabilities in the "security.capability" attribute. xattrs = item.get(b'xattrs', {}) for k, v in xattrs.items(): try: xattr.setxattr(fd or path, k, v, follow_symlinks=False) except OSError as e: if e.errno == errno.E2BIG: # xattr is too big logger.warning('%s: Value or key of extended attribute %s is too big for this filesystem' % (path, k.decode())) set_ec(EXIT_WARNING) elif e.errno == errno.ENOTSUP: # xattrs not supported here logger.warning('%s: Extended attributes are not supported on this filesystem' % path) set_ec(EXIT_WARNING) elif e.errno == errno.EACCES: # permission denied to set this specific xattr (this may happen related to security.* keys) logger.warning('%s: Permission denied when setting extended attribute %s' % (path, k.decode())) set_ec(EXIT_WARNING) elif e.errno == errno.ENOSPC: # no space left on device while setting this specific xattr # ext4 reports ENOSPC when trying to set an xattr with >4kiB while ext4 can only support 4kiB xattrs # (in this case, this is NOT a "disk full" error, just a ext4 limitation). logger.warning('%s: No space left on device while setting extended attribute %s (len = %d)' % ( path, k.decode(), len(v))) set_ec(EXIT_WARNING) else: raise # Only available on OS X and FreeBSD if has_lchflags and b'bsdflags' in item: try: os.lchflags(path, item[b'bsdflags']) except OSError: pass def rename(self, name): if name in self.manifest.archives: raise self.AlreadyExists(name) metadata = StableDict(self._load_meta(self.id)) metadata[b'name'] = name data = msgpack.packb(metadata, unicode_errors='surrogateescape') new_id = self.key.id_hash(data) self.cache.add_chunk(new_id, data, self.stats) self.manifest.archives[name] = {'id': new_id, 'time': metadata[b'time']} self.cache.chunk_decref(self.id, self.stats) del self.manifest.archives[self.name] def delete(self, stats, progress=False, forced=False): class ChunksIndexError(Error): """Chunk ID {} missing from chunks index, corrupted chunks index - aborting transaction.""" def chunk_decref(id, stats): nonlocal error try: self.cache.chunk_decref(id, stats) except KeyError: cid = bin_to_hex(id) raise ChunksIndexError(cid) except Repository.ObjectNotFound as e: # object not in repo - strange, but we wanted to delete it anyway. if forced == 0: raise error = True error = False try: unpacker = msgpack.Unpacker(use_list=False) items_ids = self.metadata[b'items'] pi = ProgressIndicatorPercent(total=len(items_ids), msg="Decrementing references %3.0f%%", same_line=True) for (i, (items_id, data)) in enumerate(zip(items_ids, self.repository.get_many(items_ids))): if progress: pi.show(i) unpacker.feed(self.key.decrypt(items_id, data)) chunk_decref(items_id, stats) try: for item in unpacker: if b'chunks' in item: for chunk_id, size, csize in item[b'chunks']: chunk_decref(chunk_id, stats) except (TypeError, ValueError): # if items metadata spans multiple chunks and one chunk got dropped somehow, # it could be that unpacker yields bad types if forced == 0: raise error = True if progress: pi.finish() except (msgpack.UnpackException, Repository.ObjectNotFound): # items metadata corrupted if forced == 0: raise error = True # in forced delete mode, we try hard to delete at least the manifest entry, # if possible also the archive superblock, even if processing the items raises # some harmless exception. chunk_decref(self.id, stats) del self.manifest.archives[self.name] if error: logger.warning('forced deletion succeeded, but the deleted archive was corrupted.') logger.warning('borg check --repair is required to free all space.') def stat_attrs(self, st, path): item = { b'mode': st.st_mode, b'uid': st.st_uid, b'user': uid2user(st.st_uid), b'gid': st.st_gid, b'group': gid2group(st.st_gid), b'mtime': int_to_bigint(safe_ns(st.st_mtime_ns)), } # borg can work with archives only having mtime (older attic archives do not have # atime/ctime). it can be useful to omit atime/ctime, if they change without the # file content changing - e.g. to get better metadata deduplication. if not self.noatime: item[b'atime'] = int_to_bigint(safe_ns(st.st_atime_ns)) if not self.noctime: item[b'ctime'] = int_to_bigint(safe_ns(st.st_ctime_ns)) if self.numeric_owner: item[b'user'] = item[b'group'] = None with backup_io(): xattrs = xattr.get_all(path, follow_symlinks=False) if xattrs: item[b'xattrs'] = StableDict(xattrs) if has_lchflags and st.st_flags: item[b'bsdflags'] = st.st_flags with backup_io(): acl_get(path, item, st, self.numeric_owner) return item def process_dir(self, path, st): item = {b'path': make_path_safe(path)} item.update(self.stat_attrs(st, path)) self.add_item(item) return 'd' # directory def process_fifo(self, path, st): item = {b'path': make_path_safe(path)} item.update(self.stat_attrs(st, path)) self.add_item(item) return 'f' # fifo def process_dev(self, path, st): item = {b'path': make_path_safe(path), b'rdev': st.st_rdev} item.update(self.stat_attrs(st, path)) self.add_item(item) if stat.S_ISCHR(st.st_mode): return 'c' # char device elif stat.S_ISBLK(st.st_mode): return 'b' # block device def process_symlink(self, path, st): # note: we can not support hardlinked symlinks, # due to the dual-use of item[b'source'], see issue #2343: # hardlinked symlinks will be archived [and extracted] as non-hardlinked symlinks. with backup_io(): source = os.readlink(path) item = {b'path': make_path_safe(path), b'source': source} item.update(self.stat_attrs(st, path)) self.add_item(item) return 's' # symlink def process_stdin(self, path, cache): uid, gid = 0, 0 fd = sys.stdin.buffer # binary chunks = [] for chunk in backup_io_iter(self.chunker.chunkify(fd)): chunks.append(cache.add_chunk(self.key.id_hash(chunk), chunk, self.stats)) self.stats.nfiles += 1 t = int_to_bigint(int(time.time()) * 1000000000) item = { b'path': path, b'chunks': chunks, b'mode': 0o100660, # regular file, ug=rw b'uid': uid, b'user': uid2user(uid), b'gid': gid, b'group': gid2group(gid), b'mtime': t, b'atime': t, b'ctime': t, } self.add_item(item) return 'i' # stdin def process_file(self, path, st, cache, ignore_inode=False): status = None safe_path = make_path_safe(path) # Is it a hard link? if st.st_nlink > 1: source = self.hard_links.get((st.st_ino, st.st_dev)) if source is not None: item = self.stat_attrs(st, path) item.update({b'path': safe_path, b'source': source}) self.add_item(item) status = 'h' # regular file, hardlink (to already seen inodes) return status is_special_file = is_special(st.st_mode) if not is_special_file: path_hash = self.key.id_hash(os.path.join(self.cwd, path).encode('utf-8', 'surrogateescape')) ids = cache.file_known_and_unchanged(path_hash, st, ignore_inode) else: # in --read-special mode, we may be called for special files. # there should be no information in the cache about special files processed in # read-special mode, but we better play safe as this was wrong in the past: path_hash = ids = None first_run = not cache.files if first_run: logger.debug('Processing files ...') chunks = None if ids is not None: # Make sure all ids are available for id_ in ids: if not cache.seen_chunk(id_): break else: chunks = [cache.chunk_incref(id_, self.stats) for id_ in ids] status = 'U' # regular file, unchanged else: status = 'A' # regular file, added item = {b'path': safe_path} # Only chunkify the file if needed if chunks is None: with backup_io(): fh = Archive._open_rb(path) with os.fdopen(fh, 'rb') as fd: chunks = [] for chunk in backup_io_iter(self.chunker.chunkify(fd, fh)): chunks.append(cache.add_chunk(self.key.id_hash(chunk), chunk, self.stats)) if self.show_progress: self.stats.show_progress(item=item, dt=0.2) if not is_special_file: # we must not memorize special files, because the contents of e.g. a # block or char device will change without its mtime/size/inode changing. cache.memorize_file(path_hash, st, [c[0] for c in chunks]) status = status or 'M' # regular file, modified (if not 'A' already) item[b'chunks'] = chunks item.update(self.stat_attrs(st, path)) if is_special_file: # we processed a special file like a regular file. reflect that in mode, # so it can be extracted / accessed in FUSE mount like a regular file: item[b'mode'] = stat.S_IFREG | stat.S_IMODE(item[b'mode']) self.stats.nfiles += 1 self.add_item(item) if st.st_nlink > 1 and source is None: # Add the hard link reference *after* the file has been added to the archive. self.hard_links[st.st_ino, st.st_dev] = safe_path return status @staticmethod def list_archives(repository, key, manifest, cache=None): # expensive! see also Manifest.list_archive_infos. for name, info in manifest.archives.items(): yield Archive(repository, key, manifest, name, cache=cache) @staticmethod def _open_rb(path): try: # if we have O_NOATIME, this likely will succeed if we are root or owner of file: return os.open(path, flags_noatime) except PermissionError: if flags_noatime == flags_normal: # we do not have O_NOATIME, no need to try again: raise # Was this EPERM due to the O_NOATIME flag? Try again without it: return os.open(path, flags_normal) # this set must be kept complete, otherwise the RobustUnpacker might malfunction: ITEM_KEYS = frozenset([b'path', b'source', b'rdev', b'chunks', b'chunks_healthy', b'mode', b'user', b'group', b'uid', b'gid', b'mtime', b'atime', b'ctime', b'xattrs', b'bsdflags', b'acl_nfs4', b'acl_access', b'acl_default', b'acl_extended', ]) # this is the set of keys that are always present in items: REQUIRED_ITEM_KEYS = frozenset([b'path', b'mtime', ]) # this set must be kept complete, otherwise rebuild_manifest might malfunction: ARCHIVE_KEYS = frozenset([b'version', b'name', b'items', b'cmdline', b'hostname', b'username', b'time', b'time_end', ]) # this is the set of keys that are always present in archives: REQUIRED_ARCHIVE_KEYS = frozenset([b'version', b'name', b'items', b'cmdline', b'time', ]) def valid_msgpacked_dict(d, keys_serialized): """check if the data looks like a msgpacked dict""" d_len = len(d) if d_len == 0: return False if d[0] & 0xf0 == 0x80: # object is a fixmap (up to 15 elements) offs = 1 elif d[0] == 0xde: # object is a map16 (up to 2^16-1 elements) offs = 3 else: # object is not a map (dict) # note: we must not have dicts with > 2^16-1 elements return False if d_len <= offs: return False # is the first dict key a bytestring? if d[offs] & 0xe0 == 0xa0: # key is a small bytestring (up to 31 chars) pass elif d[offs] in (0xd9, 0xda, 0xdb): # key is a str8, str16 or str32 pass else: # key is not a bytestring return False # is the bytestring any of the expected key names? key_serialized = d[offs:] return any(key_serialized.startswith(pattern) for pattern in keys_serialized) class RobustUnpacker: """A restartable/robust version of the streaming msgpack unpacker """ class UnpackerCrashed(Exception): """raise if unpacker crashed""" def __init__(self, validator, item_keys): super().__init__() self.item_keys = [msgpack.packb(name) for name in item_keys] self.validator = validator self._buffered_data = [] self._resync = False self._unpacker = msgpack.Unpacker(object_hook=StableDict) def resync(self): self._buffered_data = [] self._resync = True def feed(self, data): if self._resync: self._buffered_data.append(data) else: self._unpacker.feed(data) def __iter__(self): return self def __next__(self): def unpack_next(): try: return next(self._unpacker) except (TypeError, ValueError) as err: # transform exceptions that might be raised when feeding # msgpack with invalid data to a more specific exception raise self.UnpackerCrashed(str(err)) if self._resync: data = b''.join(self._buffered_data) while self._resync: if not data: raise StopIteration # Abort early if the data does not look like a serialized item dict if not valid_msgpacked_dict(data, self.item_keys): data = data[1:] continue self._unpacker = msgpack.Unpacker(object_hook=StableDict) self._unpacker.feed(data) try: item = unpack_next() except (self.UnpackerCrashed, StopIteration): # as long as we are resyncing, we also ignore StopIteration pass else: if self.validator(item): self._resync = False return item data = data[1:] else: return unpack_next() class ArchiveChecker: def __init__(self): self.error_found = False self.possibly_superseded = set() def check(self, repository, repair=False, archive=None, last=None, prefix=None, save_space=False): logger.info('Starting archive consistency check...') self.check_all = archive is None and last is None and prefix is None self.repair = repair self.repository = repository self.init_chunks() if not self.chunks: logger.error('Repository contains no apparent data at all, cannot continue check/repair.') return False self.key = self.identify_key(repository) if Manifest.MANIFEST_ID not in self.chunks: logger.error("Repository manifest not found!") self.error_found = True self.manifest = self.rebuild_manifest() else: try: self.manifest, _ = Manifest.load(repository, (Manifest.Operation.CHECK,), key=self.key) except IntegrityError as exc: logger.error('Repository manifest is corrupted: %s', exc) self.error_found = True del self.chunks[Manifest.MANIFEST_ID] self.manifest = self.rebuild_manifest() self.rebuild_refcounts(archive=archive, last=last, prefix=prefix) self.orphan_chunks_check() self.finish(save_space=save_space) if self.error_found: logger.error('Archive consistency check complete, problems found.') else: logger.info('Archive consistency check complete, no problems found.') return self.repair or not self.error_found def init_chunks(self): """Fetch a list of all object keys from repository """ # Explicitly set the initial hash table capacity to avoid performance issues # due to hash table "resonance". # Since reconstruction of archive items can add some new chunks, add 10 % headroom capacity = int(len(self.repository) / ChunkIndex.MAX_LOAD_FACTOR * 1.1) self.chunks = ChunkIndex(capacity) marker = None while True: result = self.repository.list(limit=LIST_SCAN_LIMIT, marker=marker) if not result: break marker = result[-1] for id_ in result: self.chunks[id_] = (0, 0, 0) def identify_key(self, repository): try: some_chunkid, _ = next(self.chunks.iteritems()) except StopIteration: # repo is completely empty, no chunks return None cdata = repository.get(some_chunkid) return key_factory(repository, cdata) def rebuild_manifest(self): """Rebuild the manifest object if it is missing Iterates through all objects in the repository looking for archive metadata blocks. """ def valid_archive(obj): if not isinstance(obj, dict): return False keys = set(obj) return REQUIRED_ARCHIVE_KEYS.issubset(keys) logger.info('Rebuilding missing manifest, this might take some time...') # as we have lost the manifest, we do not know any more what valid item keys we had. # collecting any key we encounter in a damaged repo seems unwise, thus we just use # the hardcoded list from the source code. thus, it is not recommended to rebuild a # lost manifest on a older borg version than the most recent one that was ever used # within this repository (assuming that newer borg versions support more item keys). manifest = Manifest(self.key, self.repository) archive_keys_serialized = [msgpack.packb(name) for name in ARCHIVE_KEYS] for chunk_id, _ in self.chunks.iteritems(): cdata = self.repository.get(chunk_id) try: data = self.key.decrypt(chunk_id, cdata) except IntegrityError as exc: logger.error('Skipping corrupted chunk: %s', exc) self.error_found = True continue if not valid_msgpacked_dict(data, archive_keys_serialized): continue if b'cmdline' not in data or b'\xa7version\x01' not in data: continue try: archive = msgpack.unpackb(data) # Ignore exceptions that might be raised when feeding # msgpack with invalid data except (TypeError, ValueError, StopIteration): continue if valid_archive(archive): name = archive[b'name'].decode() logger.info('Found archive %s', name) if name in manifest.archives: i = 1 while True: new_name = '%s.%d' % (name, i) if new_name not in manifest.archives: break i += 1 logger.warning('Duplicate archive name %s, storing as %s', name, new_name) name = new_name manifest.archives[name] = {b'id': chunk_id, b'time': archive[b'time']} logger.info('Manifest rebuild complete.') return manifest def rebuild_refcounts(self, archive=None, last=None, prefix=None): """Rebuild object reference counts by walking the metadata Missing and/or incorrect data is repaired when detected """ # Exclude the manifest from chunks del self.chunks[Manifest.MANIFEST_ID] def mark_as_possibly_superseded(id_): if self.chunks.get(id_, (0,))[0] == 0: self.possibly_superseded.add(id_) def add_callback(chunk): id_ = self.key.id_hash(chunk) cdata = self.key.encrypt(chunk) add_reference(id_, len(chunk), len(cdata), cdata) return id_ def add_reference(id_, size, csize, cdata=None): try: self.chunks.incref(id_) except KeyError: assert cdata is not None self.chunks[id_] = 1, size, csize if self.repair: self.repository.put(id_, cdata) def verify_file_chunks(item): """Verifies that all file chunks are present. Missing file chunks will be replaced with new chunks of the same length containing all zeros. If a previously missing file chunk re-appears, the replacement chunk is replaced by the correct one. """ def replacement_chunk(size): data = bytes(size) chunk_id = self.key.id_hash(data) cdata = self.key.encrypt(data) csize = len(cdata) return chunk_id, size, csize, cdata offset = 0 chunk_list = [] chunks_replaced = False has_chunks_healthy = b'chunks_healthy' in item chunks_current = item[b'chunks'] chunks_healthy = item[b'chunks_healthy'] if has_chunks_healthy else chunks_current assert len(chunks_current) == len(chunks_healthy) for chunk_current, chunk_healthy in zip(chunks_current, chunks_healthy): chunk_id, size, csize = chunk_healthy if chunk_id not in self.chunks: # a chunk of the healthy list is missing if chunk_current == chunk_healthy: logger.error('{}: New missing file chunk detected (Byte {}-{}). ' 'Replacing with all-zero chunk.'.format( item[b'path'].decode('utf-8', 'surrogateescape'), offset, offset + size)) self.error_found = chunks_replaced = True chunk_id, size, csize, cdata = replacement_chunk(size) add_reference(chunk_id, size, csize, cdata) else: logger.info('{}: Previously missing file chunk is still missing (Byte {}-{}). ' 'It has a all-zero replacement chunk already.'.format( item[b'path'].decode('utf-8', 'surrogateescape'), offset, offset + size)) chunk_id, size, csize = chunk_current if chunk_id in self.chunks: add_reference(chunk_id, size, csize) else: logger.warning('{}: Missing all-zero replacement chunk detected (Byte {}-{}). ' 'Generating new replacement chunk.'.format(item.path, offset, offset + size)) self.error_found = chunks_replaced = True chunk_id, size, csize, cdata = replacement_chunk(size) add_reference(chunk_id, size, csize, cdata) else: if chunk_current == chunk_healthy: # normal case, all fine. add_reference(chunk_id, size, csize) else: logger.info('{}: Healed previously missing file chunk! (Byte {}-{}).'.format( item[b'path'].decode('utf-8', 'surrogateescape'), offset, offset + size)) add_reference(chunk_id, size, csize) mark_as_possibly_superseded(chunk_current[0]) # maybe orphaned the all-zero replacement chunk chunk_list.append([chunk_id, size, csize]) # list-typed element as chunks_healthy is list-of-lists offset += size if chunks_replaced and not has_chunks_healthy: # if this is first repair, remember the correct chunk IDs, so we can maybe heal the file later item[b'chunks_healthy'] = item[b'chunks'] if has_chunks_healthy and chunk_list == chunks_healthy: logger.info('{}: Completely healed previously damaged file!'.format( item[b'path'].decode('utf-8', 'surrogateescape'))) del item[b'chunks_healthy'] item[b'chunks'] = chunk_list def robust_iterator(archive): """Iterates through all archive items Missing item chunks will be skipped and the msgpack stream will be restarted """ item_keys = self.manifest.item_keys unpacker = RobustUnpacker(lambda item: isinstance(item, dict) and b'path' in item, item_keys) _state = 0 def missing_chunk_detector(chunk_id): nonlocal _state if _state % 2 != int(chunk_id not in self.chunks): _state += 1 return _state def report(msg, chunk_id, chunk_no): cid = bin_to_hex(chunk_id) msg += ' [chunk: %06d_%s]' % (chunk_no, cid) # see debug-dump-archive-items self.error_found = True logger.error(msg) def list_keys_safe(keys): return ', '.join((k.decode() if isinstance(k, bytes) else str(k) for k in keys)) def valid_item(obj): if not isinstance(obj, StableDict): return False, 'not a dictionary' # A bug in Attic up to and including release 0.13 added a (meaningless) b'acl' key to every item. # We ignore it here, should it exist. See test_attic013_acl_bug for details. obj.pop(b'acl', None) keys = set(obj) if not REQUIRED_ITEM_KEYS.issubset(keys): return False, 'missing required keys: ' + list_keys_safe(REQUIRED_ITEM_KEYS - keys) if not keys.issubset(item_keys): return False, 'invalid keys: ' + list_keys_safe(keys - item_keys) return True, '' i = 0 for state, items in groupby(archive[b'items'], missing_chunk_detector): items = list(items) if state % 2: for chunk_id in items: report('item metadata chunk missing', chunk_id, i) i += 1 continue if state > 0: unpacker.resync() for chunk_id, cdata in zip(items, repository.get_many(items)): unpacker.feed(self.key.decrypt(chunk_id, cdata)) try: for item in unpacker: valid, reason = valid_item(item) if valid: yield item else: report('Did not get expected metadata dict when unpacking item metadata (%s)' % reason, chunk_id, i) except RobustUnpacker.UnpackerCrashed as err: report('Unpacker crashed while unpacking item metadata, trying to resync...', chunk_id, i) unpacker.resync() except Exception: report('Exception while unpacking item metadata', chunk_id, i) raise i += 1 if archive is None: # we need last N or all archives archive_items = sorted(self.manifest.archives.items(), reverse=True, key=lambda name_info: name_info[1][b'time']) if prefix is not None: archive_items = [item for item in archive_items if item[0].startswith(prefix)] if not archive_items: logger.warning('--prefix %s does not match any archives', prefix) num_archives = len(archive_items) end = None if last is None else min(num_archives, last) if last is not None and end < last: logger.warning('--last %d archives: only found %d archives', last, end) else: # we only want one specific archive archive_items = [item for item in self.manifest.archives.items() if item[0] == archive] num_archives = 1 end = 1 if not archive_items: logger.error('Archive %s does not exist', archive) self.error_found = True return with cache_if_remote(self.repository) as repository: for i, (name, info) in enumerate(archive_items[:end]): logger.info('Analyzing archive {} ({}/{})'.format(name, num_archives - i, num_archives)) archive_id = info[b'id'] if archive_id not in self.chunks: logger.error('Archive metadata block is missing!') self.error_found = True del self.manifest.archives[name] continue mark_as_possibly_superseded(archive_id) cdata = self.repository.get(archive_id) data = self.key.decrypt(archive_id, cdata) archive = StableDict(msgpack.unpackb(data)) if archive[b'version'] != 1: raise Exception('Unknown archive metadata version') decode_dict(archive, (b'name', b'hostname', b'username', b'time', b'time_end')) archive[b'cmdline'] = [arg.decode('utf-8', 'surrogateescape') for arg in archive[b'cmdline']] items_buffer = ChunkBuffer(self.key) items_buffer.write_chunk = add_callback for item in robust_iterator(archive): if b'chunks' in item: verify_file_chunks(item) items_buffer.add(item) items_buffer.flush(flush=True) for previous_item_id in archive[b'items']: mark_as_possibly_superseded(previous_item_id) archive[b'items'] = items_buffer.chunks data = msgpack.packb(archive, unicode_errors='surrogateescape') new_archive_id = self.key.id_hash(data) cdata = self.key.encrypt(data) add_reference(new_archive_id, len(data), len(cdata), cdata) info[b'id'] = new_archive_id def orphan_chunks_check(self): if self.check_all: unused = set() for id_, (count, size, csize) in self.chunks.iteritems(): if count == 0: unused.add(id_) orphaned = unused - self.possibly_superseded if orphaned: logger.error('{} orphaned objects found!'.format(len(orphaned))) self.error_found = True if self.repair: for id_ in unused: self.repository.delete(id_) else: logger.info('Orphaned objects check skipped (needs all archives checked).') def finish(self, save_space=False): if self.repair: self.manifest.write() self.repository.commit(save_space=save_space) borgbackup-1.0.13/borg/archiver.py0000644000175000017500000035170313431513133017005 0ustar useruser00000000000000from binascii import unhexlify from datetime import datetime from hashlib import sha256 from operator import attrgetter import argparse import faulthandler import functools import inspect import itertools import os import re import shlex import signal import stat import sys import textwrap import time import traceback import collections from . import __version__ from .helpers import Error, location_validator, archivename_validator, format_line, format_time, format_file_size, \ parse_pattern, PathPrefixPattern, to_localtime, timestamp, safe_timestamp, bin_to_hex, \ get_cache_dir, interval, prune_within, prune_split, check_python, \ Manifest, NoManifestError, remove_surrogates, update_excludes, format_archive, check_extension_modules, Statistics, \ dir_is_tagged, bigint_to_int, ChunkerParams, CompressionSpec, PrefixSpec, is_slow_msgpack, yes, sysinfo, \ EXIT_SUCCESS, EXIT_WARNING, EXIT_ERROR, log_multi, PatternMatcher, ErrorIgnoringTextIOWrapper, set_ec, \ replace_placeholders from .helpers import signal_handler, raising_signal_handler, SigHup, SigTerm from .logger import create_logger, setup_logging logger = create_logger() from .compress import Compressor from .upgrader import AtticRepositoryUpgrader, BorgRepositoryUpgrader from .repository import Repository, LIST_SCAN_LIMIT from .cache import Cache from .key import key_creator, tam_required_file, tam_required, RepoKey, PassphraseKey from .keymanager import KeyManager from .archive import backup_io, BackupOSError, Archive, ArchiveChecker, CHUNKER_PARAMS, is_special from .remote import RepositoryServer, RemoteRepository, cache_if_remote from .platform import umount has_lchflags = hasattr(os, 'lchflags') # default umask, overriden by --umask, defaults to read/write only for owner UMASK_DEFAULT = 0o077 DASHES = '-' * 78 def argument(args, str_or_bool): """If bool is passed, return it. If str is passed, retrieve named attribute from args.""" if isinstance(str_or_bool, str): return getattr(args, str_or_bool) if isinstance(str_or_bool, (list, tuple)): return any(getattr(args, item) for item in str_or_bool) return str_or_bool def with_repository(fake=False, invert_fake=False, create=False, lock=True, exclusive=False, manifest=True, cache=False, compatibility=None): """ Method decorator for subcommand-handling methods: do_XYZ(self, args, repository, …) If a parameter (where allowed) is a str the attribute named of args is used instead. :param fake: (str or bool) use None instead of repository, don't do anything else :param create: create repository :param lock: lock repository :param exclusive: (str or bool) lock repository exclusively (for writing) :param manifest: load manifest and key, pass them as keyword arguments :param cache: open cache, pass it as keyword argument (implies manifest) :param compatibility: mandatory if not create and (manifest or cache), specifies mandatory feature categories to check """ if not create and (manifest or cache): if compatibility is None: raise AssertionError("with_repository decorator used without compatibility argument") if type(compatibility) is not tuple: raise AssertionError("with_repository decorator compatibility argument must be of type tuple") else: if compatibility is not None: raise AssertionError("with_repository called with compatibility argument but would not check" + repr(compatibility)) if create: compatibility = Manifest.NO_OPERATION_CHECK def decorator(method): @functools.wraps(method) def wrapper(self, args, **kwargs): location = args.location # note: 'location' must be always present in args append_only = getattr(args, 'append_only', False) if argument(args, fake) ^ invert_fake: return method(self, args, repository=None, **kwargs) elif location.proto == 'ssh': repository = RemoteRepository(location, create=create, exclusive=argument(args, exclusive), lock_wait=self.lock_wait, lock=lock, append_only=append_only, args=args) else: repository = Repository(location.path, create=create, exclusive=argument(args, exclusive), lock_wait=self.lock_wait, lock=lock, append_only=append_only) with repository: if manifest or cache: kwargs['manifest'], kwargs['key'] = Manifest.load(repository, compatibility) if cache: with Cache(repository, kwargs['key'], kwargs['manifest'], do_files=getattr(args, 'cache_files', False), lock_wait=self.lock_wait) as cache_: return method(self, args, repository=repository, cache=cache_, **kwargs) else: return method(self, args, repository=repository, **kwargs) return wrapper return decorator def with_archive(method): @functools.wraps(method) def wrapper(self, args, repository, key, manifest, **kwargs): archive = Archive(repository, key, manifest, args.location.archive, numeric_owner=getattr(args, 'numeric_owner', False), cache=kwargs.get('cache')) return method(self, args, repository=repository, manifest=manifest, key=key, archive=archive, **kwargs) return wrapper class Archiver: def __init__(self, lock_wait=None): self.exit_code = EXIT_SUCCESS self.lock_wait = lock_wait def print_error(self, msg, *args): msg = args and msg % args or msg self.exit_code = EXIT_ERROR logger.error(msg) def print_warning(self, msg, *args): msg = args and msg % args or msg self.exit_code = EXIT_WARNING # we do not terminate here, so it is a warning logger.warning(msg) def print_file_status(self, status, path): if self.output_list and (self.output_filter is None or status in self.output_filter): logger.info("%1s %s", status, remove_surrogates(path)) def do_serve(self, args): """Start in server mode. This command is usually not used manually. """ RepositoryServer(restrict_to_paths=args.restrict_to_paths, append_only=args.append_only).serve() return EXIT_SUCCESS @with_repository(create=True, exclusive=True, manifest=False) def do_init(self, args, repository): """Initialize an empty repository""" path = args.location.canonical_path() logger.info('Initializing repository at "%s"' % path) key = key_creator(repository, args) manifest = Manifest(key, repository) manifest.key = key manifest.write() repository.commit() with Cache(repository, key, manifest, warn_if_unencrypted=False): pass if key.tam_required: tam_file = tam_required_file(repository) open(tam_file, 'w').close() logger.warning( '\n' 'By default repositories initialized with this version will produce security\n' 'errors if written to with an older version (up to and including Borg 1.0.8).\n' '\n' 'If you want to use these older versions, you can disable the check by runnning:\n' 'borg upgrade --disable-tam \'%s\'\n' '\n' 'See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability ' 'for details about the security implications.', path) return self.exit_code @with_repository(exclusive=True, manifest=False) def do_check(self, args, repository): """Check repository consistency""" if args.repair: msg = ("'check --repair' is an experimental feature that might result in data loss." + "\n" + "Type 'YES' if you understand this and want to continue: ") if not yes(msg, false_msg="Aborting.", invalid_msg="Invalid answer, aborting.", truish=('YES', ), retry=False, env_var_override='BORG_CHECK_I_KNOW_WHAT_I_AM_DOING'): return EXIT_ERROR if not args.archives_only: if not repository.check(repair=args.repair, save_space=args.save_space): return EXIT_WARNING if not args.repo_only and not ArchiveChecker().check( repository, repair=args.repair, archive=args.location.archive, last=args.last, prefix=args.prefix, save_space=args.save_space): return EXIT_WARNING return EXIT_SUCCESS @with_repository(compatibility=(Manifest.Operation.CHECK,)) def do_change_passphrase(self, args, repository, manifest, key): """Change repository key file passphrase""" if not hasattr(key, 'change_passphrase'): print('This repository is not encrypted, cannot change the passphrase.') return EXIT_ERROR key.change_passphrase() logger.info('Key updated') if hasattr(key, 'find_key'): # print key location to make backing it up easier logger.info('Key location: %s', key.find_key()) return EXIT_SUCCESS @with_repository(lock=False, exclusive=False, manifest=False, cache=False) def do_key_export(self, args, repository): """Export the repository key for backup""" manager = KeyManager(repository) manager.load_keyblob() if args.paper: manager.export_paperkey(args.path) else: if not args.path: self.print_error("output file to export key to expected") return EXIT_ERROR if args.qr: manager.export_qr(args.path) else: manager.export(args.path) return EXIT_SUCCESS @with_repository(lock=False, exclusive=False, manifest=False, cache=False) def do_key_import(self, args, repository): """Import the repository key from backup""" manager = KeyManager(repository) if args.paper: if args.path: self.print_error("with --paper import from file is not supported") return EXIT_ERROR manager.import_paperkey(args) else: if not args.path: self.print_error("input file to import key from expected") return EXIT_ERROR if not os.path.exists(args.path): self.print_error("input file does not exist: " + args.path) return EXIT_ERROR manager.import_keyfile(args) return EXIT_SUCCESS @with_repository(manifest=False) def do_migrate_to_repokey(self, args, repository): """Migrate passphrase -> repokey""" manifest_data = repository.get(Manifest.MANIFEST_ID) key_old = PassphraseKey.detect(repository, manifest_data) key_new = RepoKey(repository) key_new.target = repository key_new.repository_id = repository.id key_new.enc_key = key_old.enc_key key_new.enc_hmac_key = key_old.enc_hmac_key key_new.id_key = key_old.id_key key_new.chunk_seed = key_old.chunk_seed key_new.change_passphrase() # option to change key protection passphrase, save logger.info('Key updated') return EXIT_SUCCESS @with_repository(fake='dry_run', exclusive=True, compatibility=(Manifest.Operation.WRITE,)) def do_create(self, args, repository, manifest=None, key=None): """Create new archive""" matcher = PatternMatcher(fallback=True) if args.excludes: matcher.add(args.excludes, False) def create_inner(archive, cache): # Add cache dir to inode_skip list skip_inodes = set() try: st = os.stat(get_cache_dir()) skip_inodes.add((st.st_ino, st.st_dev)) except OSError: pass # Add local repository dir to inode_skip list if not args.location.host: try: st = os.stat(args.location.path) skip_inodes.add((st.st_ino, st.st_dev)) except OSError: pass for path in args.paths: if path == '-': # stdin path = 'stdin' if not dry_run: try: status = archive.process_stdin(path, cache) except BackupOSError as e: status = 'E' self.print_warning('%s: %s', path, e) else: status = '-' self.print_file_status(status, path) continue path = os.path.normpath(path) if args.one_file_system: try: restrict_dev = os.stat(path, follow_symlinks=False).st_dev except OSError as e: self.print_warning('%s: %s', path, e) continue else: restrict_dev = None self._process(archive, cache, matcher, args.exclude_caches, args.exclude_if_present, args.keep_tag_files, skip_inodes, path, restrict_dev, read_special=args.read_special, dry_run=dry_run) if not dry_run: archive.save(timestamp=args.timestamp) if args.progress: archive.stats.show_progress(final=True) if args.stats: log_multi(DASHES, str(archive), DASHES, str(archive.stats), str(cache), DASHES) self.output_filter = args.output_filter self.output_list = args.output_list self.ignore_inode = args.ignore_inode dry_run = args.dry_run t0 = datetime.utcnow() t0_monotonic = time.monotonic() if not dry_run: key.compressor = Compressor(**args.compression) with Cache(repository, key, manifest, do_files=args.cache_files, lock_wait=self.lock_wait) as cache: archive = Archive(repository, key, manifest, args.location.archive, cache=cache, create=True, checkpoint_interval=args.checkpoint_interval, numeric_owner=args.numeric_owner, noatime=args.noatime, noctime=args.noctime, progress=args.progress, chunker_params=args.chunker_params, start=t0, start_monotonic=t0_monotonic) create_inner(archive, cache) else: create_inner(None, None) return self.exit_code def _process(self, archive, cache, matcher, exclude_caches, exclude_if_present, keep_tag_files, skip_inodes, path, restrict_dev, read_special=False, dry_run=False): """ Process *path* recursively according to the various parameters. This should only raise on critical errors. Per-item errors must be handled within this method. """ if not matcher.match(path): return try: with backup_io(): st = os.stat(path, follow_symlinks=False) if (st.st_ino, st.st_dev) in skip_inodes: return # Entering a new filesystem? if restrict_dev is not None and st.st_dev != restrict_dev: return status = None # Ignore if nodump flag is set if has_lchflags and (st.st_flags & stat.UF_NODUMP): return if stat.S_ISREG(st.st_mode): if not dry_run: status = archive.process_file(path, st, cache, self.ignore_inode) elif stat.S_ISDIR(st.st_mode): tag_paths = dir_is_tagged(path, exclude_caches, exclude_if_present) if tag_paths: if keep_tag_files and not dry_run: archive.process_dir(path, st) for tag_path in tag_paths: self._process(archive, cache, matcher, exclude_caches, exclude_if_present, keep_tag_files, skip_inodes, tag_path, restrict_dev, read_special=read_special, dry_run=dry_run) self.print_file_status('x', path) return if not dry_run: status = archive.process_dir(path, st) with backup_io(): entries = os.listdir(path) for filename in sorted(entries): entry_path = os.path.normpath(os.path.join(path, filename)) self._process(archive, cache, matcher, exclude_caches, exclude_if_present, keep_tag_files, skip_inodes, entry_path, restrict_dev, read_special=read_special, dry_run=dry_run) elif stat.S_ISLNK(st.st_mode): if not dry_run: if not read_special: status = archive.process_symlink(path, st) else: try: st_target = os.stat(path) except OSError: special = False else: special = is_special(st_target.st_mode) if special: status = archive.process_file(path, st_target, cache) else: status = archive.process_symlink(path, st) elif stat.S_ISFIFO(st.st_mode): if not dry_run: if not read_special: status = archive.process_fifo(path, st) else: status = archive.process_file(path, st, cache) elif stat.S_ISCHR(st.st_mode) or stat.S_ISBLK(st.st_mode): if not dry_run: if not read_special: status = archive.process_dev(path, st) else: status = archive.process_file(path, st, cache) elif stat.S_ISSOCK(st.st_mode): # Ignore unix sockets return elif stat.S_ISDOOR(st.st_mode): # Ignore Solaris doors return elif stat.S_ISPORT(st.st_mode): # Ignore Solaris event ports return else: self.print_warning('Unknown file type: %s', path) return except BackupOSError as e: self.print_warning('%s: %s', path, e) status = 'E' # Status output if status is None: if not dry_run: status = '?' # need to add a status code somewhere else: status = '-' # dry run, item was not backed up self.print_file_status(status, path) @staticmethod def build_filter(matcher, strip_components=0): if strip_components: def item_filter(item): return matcher.match(item[b'path']) and os.sep.join(item[b'path'].split(os.sep)[strip_components:]) else: def item_filter(item): return matcher.match(item[b'path']) return item_filter @with_repository(compatibility=(Manifest.Operation.READ,)) @with_archive def do_extract(self, args, repository, manifest, key, archive): """Extract archive contents""" # be restrictive when restoring files, restore permissions later if sys.getfilesystemencoding() == 'ascii': logger.warning('Warning: File system encoding is "ascii", extracting non-ascii filenames will not be supported.') if sys.platform.startswith(('linux', 'freebsd', 'netbsd', 'openbsd', 'darwin', )): logger.warning('Hint: You likely need to fix your locale setup. E.g. install locales and use: LANG=en_US.UTF-8') matcher = PatternMatcher() if args.excludes: matcher.add(args.excludes, False) include_patterns = [] if args.paths: include_patterns.extend(parse_pattern(i, PathPrefixPattern) for i in args.paths) matcher.add(include_patterns, True) matcher.fallback = not include_patterns output_list = args.output_list dry_run = args.dry_run stdout = args.stdout sparse = args.sparse strip_components = args.strip_components dirs = [] filter = self.build_filter(matcher, strip_components) for item in archive.iter_items(filter, preload=True): orig_path = item[b'path'] if strip_components: item[b'path'] = os.sep.join(orig_path.split(os.sep)[strip_components:]) if not args.dry_run: while dirs and not item[b'path'].startswith(dirs[-1][b'path']): dir_item = dirs.pop(-1) try: archive.extract_item(dir_item, stdout=stdout) except BackupOSError as e: self.print_warning('%s: %s', remove_surrogates(dir_item[b'path']), e) if output_list: logger.info(remove_surrogates(orig_path)) try: if dry_run: archive.extract_item(item, dry_run=True) else: if stat.S_ISDIR(item[b'mode']): dirs.append(item) archive.extract_item(item, stdout=stdout, restore_attrs=False) else: archive.extract_item(item, stdout=stdout, sparse=sparse) except BackupOSError as e: self.print_warning('%s: %s', remove_surrogates(orig_path), e) if not args.dry_run: while dirs: dir_item = dirs.pop(-1) try: archive.extract_item(dir_item, stdout=stdout) except BackupOSError as e: self.print_warning('%s: %s', remove_surrogates(dir_item[b'path']), e) for pattern in include_patterns: if pattern.match_count == 0: self.print_warning("Include pattern '%s' never matched.", pattern) return self.exit_code @with_repository(exclusive=True, cache=True, compatibility=(Manifest.Operation.CHECK,)) @with_archive def do_rename(self, args, repository, manifest, key, cache, archive): """Rename an existing archive""" name = replace_placeholders(args.name) archive.rename(name) manifest.write() repository.commit() cache.commit() return self.exit_code @with_repository(exclusive=True, manifest=False) def do_delete(self, args, repository): """Delete an existing repository or archive""" if args.location.archive: archive_name = args.location.archive manifest, key = Manifest.load(repository, (Manifest.Operation.DELETE,)) if args.forced == 2: try: del manifest.archives[archive_name] except KeyError: raise Archive.DoesNotExist(archive_name) logger.info('Archive deleted.') manifest.write() # note: might crash in compact() after committing the repo repository.commit() logger.info('Done. Run "borg check --repair" to clean up the mess.') return self.exit_code with Cache(repository, key, manifest, lock_wait=self.lock_wait) as cache: archive = Archive(repository, key, manifest, archive_name, cache=cache) stats = Statistics() archive.delete(stats, progress=args.progress, forced=args.forced) manifest.write() repository.commit(save_space=args.save_space) cache.commit() logger.info("Archive deleted.") if args.stats: log_multi(DASHES, stats.summary.format(label='Deleted data:', stats=stats), str(cache), DASHES) else: if not args.cache_only: msg = [] try: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) except NoManifestError: msg.append("You requested to completely DELETE the repository *including* all archives it may contain.") msg.append("This repository seems to have no manifest, so we can't tell anything about its contents.") else: msg.append("You requested to completely DELETE the repository *including* all archives it contains:") for archive_info in manifest.list_archive_infos(sort_by='ts'): msg.append(format_archive(archive_info)) msg.append("Type 'YES' if you understand this and want to continue: ") msg = '\n'.join(msg) if not yes(msg, false_msg="Aborting.", invalid_msg='Invalid answer, aborting.', truish=('YES', ), retry=False, env_var_override='BORG_DELETE_I_KNOW_WHAT_I_AM_DOING'): self.exit_code = EXIT_ERROR return self.exit_code repository.destroy() logger.info("Repository deleted.") Cache.destroy(repository) logger.info("Cache deleted.") return self.exit_code def do_mount(self, args): """Mount archive or an entire repository as a FUSE filesystem""" # Perform these checks before opening the repository and asking for a passphrase. try: import borg.fuse except ImportError as e: self.print_error('borg mount not available: loading fuse support failed [ImportError: %s]' % str(e)) return self.exit_code if not os.path.isdir(args.mountpoint) or not os.access(args.mountpoint, os.R_OK | os.W_OK | os.X_OK): self.print_error('%s: Mountpoint must be a writable directory' % args.mountpoint) return self.exit_code return self._do_mount(args) @with_repository(compatibility=(Manifest.Operation.READ,)) def _do_mount(self, args, repository, manifest, key): from .fuse import FuseOperations with cache_if_remote(repository) as cached_repo: if args.location.archive: archive = Archive(repository, key, manifest, args.location.archive) else: archive = None operations = FuseOperations(key, repository, manifest, archive, cached_repo) logger.info("Mounting filesystem") try: operations.mount(args.mountpoint, args.options, args.foreground) except RuntimeError: # Relevant error message already printed to stderr by fuse self.exit_code = EXIT_ERROR return self.exit_code def do_umount(self, args): """un-mount the FUSE filesystem""" return umount(args.mountpoint) @with_repository(compatibility=(Manifest.Operation.READ,)) def do_list(self, args, repository, manifest, key): """List archive or repository contents""" if args.location.archive: archive = Archive(repository, key, manifest, args.location.archive) """use_user_format flag is used to speed up default listing. When user issues format options, listing is a bit slower, but more keys are available and precalculated. """ use_user_format = args.listformat is not None if use_user_format: list_format = args.listformat elif args.short: list_format = "{path}{LF}" else: list_format = "{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{LF}" for item in archive.iter_items(): mode = stat.filemode(item[b'mode']) type = mode[0] size = 0 if type == '-': try: size = sum(size for _, size, _ in item[b'chunks']) except KeyError: pass mtime = safe_timestamp(item[b'mtime']) if use_user_format: atime = safe_timestamp(item.get(b'atime') or item[b'mtime']) ctime = safe_timestamp(item.get(b'ctime') or item[b'mtime']) if b'source' in item: source = item[b'source'] if type == 'l': extra = ' -> %s' % item[b'source'] else: mode = 'h' + mode[1:] extra = ' link to %s' % item[b'source'] else: extra = '' source = '' item_data = { 'mode': mode, 'user': item[b'user'] or item[b'uid'], 'group': item[b'group'] or item[b'gid'], 'size': size, 'isomtime': format_time(mtime), 'path': remove_surrogates(item[b'path']), 'extra': extra, 'LF': '\n', } if use_user_format: item_data_advanced = { 'bmode': item[b'mode'], 'type': type, 'source': source, 'linktarget': source, 'uid': item[b'uid'], 'gid': item[b'gid'], 'mtime': mtime, 'isoctime': format_time(ctime), 'ctime': ctime, 'isoatime': format_time(atime), 'atime': atime, 'archivename': archive.name, 'SPACE': ' ', 'TAB': '\t', 'CR': '\r', 'NEWLINE': os.linesep, } item_data.update(item_data_advanced) item_data['formatkeys'] = list(item_data.keys()) print(format_line(list_format, item_data), end='') else: for archive_info in manifest.list_archive_infos(sort_by='ts'): if args.prefix and not archive_info.name.startswith(args.prefix): continue if args.short: print(archive_info.name) else: print(format_archive(archive_info)) return self.exit_code @with_repository(cache=True, compatibility=(Manifest.Operation.READ,)) @with_archive def do_info(self, args, repository, manifest, key, archive, cache): """Show archive details such as disk space used""" stats = archive.calc_stats(cache) print('Name:', archive.name) print('Fingerprint: %s' % bin_to_hex(archive.id)) print('Hostname:', archive.metadata[b'hostname']) print('Username:', archive.metadata[b'username']) print('Time (start): %s' % format_time(to_localtime(archive.ts))) print('Time (end): %s' % format_time(to_localtime(archive.ts_end))) print('Command line:', remove_surrogates(' '.join(archive.metadata[b'cmdline']))) print('Number of files: %d' % stats.nfiles) print() print(str(stats)) print(str(cache)) return self.exit_code @with_repository(exclusive=True, compatibility=(Manifest.Operation.DELETE,)) def do_prune(self, args, repository, manifest, key): """Prune repository archives according to specified rules""" if not any((args.hourly, args.daily, args.weekly, args.monthly, args.yearly, args.within)): self.print_error('At least one of the "keep-within", ' '"keep-hourly", "keep-daily", ' '"keep-weekly", "keep-monthly" or "keep-yearly" settings must be specified.') return self.exit_code archives = manifest.list_archive_infos(sort_by='ts', reverse=True) # just a ArchiveInfo list if args.prefix: archives = [archive for archive in archives if archive.name.startswith(args.prefix)] # ignore all checkpoint archives to avoid keeping one (which is an incomplete backup) # that is newer than a successfully completed backup - and killing the successful backup. is_checkpoint = re.compile(r'\.checkpoint(\.\d+)?$').search archives = [archive for archive in archives if not is_checkpoint(archive.name)] keep = [] if args.within: keep += prune_within(archives, args.within) if args.hourly: keep += prune_split(archives, '%Y-%m-%d %H', args.hourly, keep) if args.daily: keep += prune_split(archives, '%Y-%m-%d', args.daily, keep) if args.weekly: keep += prune_split(archives, '%G-%V', args.weekly, keep) if args.monthly: keep += prune_split(archives, '%Y-%m', args.monthly, keep) if args.yearly: keep += prune_split(archives, '%Y', args.yearly, keep) keep.sort(key=attrgetter('ts'), reverse=True) to_delete = [a for a in archives if a not in keep] stats = Statistics() with Cache(repository, key, manifest, do_files=args.cache_files, lock_wait=self.lock_wait) as cache: for archive in keep: if args.output_list: logger.info('Keeping archive: %s' % format_archive(archive)) for archive in to_delete: if args.dry_run: if args.output_list: logger.info('Would prune: %s' % format_archive(archive)) else: if args.output_list: logger.info('Pruning archive: %s' % format_archive(archive)) Archive(repository, key, manifest, archive.name, cache).delete(stats, forced=args.forced) if to_delete and not args.dry_run: manifest.write() repository.commit(save_space=args.save_space) cache.commit() if args.stats: log_multi(DASHES, stats.summary.format(label='Deleted data:', stats=stats), str(cache), DASHES) return self.exit_code @with_repository(fake=('tam', 'disable_tam'), invert_fake=True, manifest=False, exclusive=True) def do_upgrade(self, args, repository, manifest=None, key=None): """upgrade a repository from a previous version""" if args.tam: manifest, key = Manifest.load(repository, (Manifest.Operation.CHECK,), force_tam_not_required=args.force) if not hasattr(key, 'change_passphrase'): print('This repository is not encrypted, cannot enable TAM.') return EXIT_ERROR if not manifest.tam_verified or not manifest.config.get(b'tam_required', False): # The standard archive listing doesn't include the archive ID like in borg 1.1.x print('Manifest contents:') for archive_info in manifest.list_archive_infos(sort_by='ts'): print(format_archive(archive_info), '[%s]' % bin_to_hex(archive_info.id)) manifest.config[b'tam_required'] = True manifest.write() repository.commit() if not key.tam_required: key.tam_required = True key.change_passphrase(key._passphrase) print('Key updated') if hasattr(key, 'find_key'): print('Key location:', key.find_key()) if not tam_required(repository): tam_file = tam_required_file(repository) open(tam_file, 'w').close() print('Updated security database') elif args.disable_tam: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK, force_tam_not_required=True) if tam_required(repository): os.unlink(tam_required_file(repository)) if key.tam_required: key.tam_required = False key.change_passphrase(key._passphrase) print('Key updated') if hasattr(key, 'find_key'): print('Key location:', key.find_key()) manifest.config[b'tam_required'] = False manifest.write() repository.commit() else: # mainly for upgrades from Attic repositories, # but also supports borg 0.xx -> 1.0 upgrade. repo = AtticRepositoryUpgrader(args.location.path, create=False) try: repo.upgrade(args.dry_run, inplace=args.inplace, progress=args.progress) except NotImplementedError as e: print("warning: %s" % e) repo = BorgRepositoryUpgrader(args.location.path, create=False) try: repo.upgrade(args.dry_run, inplace=args.inplace, progress=args.progress) except NotImplementedError as e: print("warning: %s" % e) return self.exit_code def do_debug_info(self, args): """display system information for debugging / bug reports""" print(sysinfo()) return EXIT_SUCCESS @with_repository(compatibility=Manifest.NO_OPERATION_CHECK) def do_debug_dump_archive_items(self, args, repository, manifest, key): """dump (decrypted, decompressed) archive items metadata (not: data)""" archive = Archive(repository, key, manifest, args.location.archive) for i, item_id in enumerate(archive.metadata[b'items']): data = key.decrypt(item_id, repository.get(item_id)) filename = '%06d_%s.items' % (i, bin_to_hex(item_id)) print('Dumping', filename) with open(filename, 'wb') as fd: fd.write(data) print('Done.') return EXIT_SUCCESS @with_repository(compatibility=Manifest.NO_OPERATION_CHECK) def do_debug_dump_repo_objs(self, args, repository, manifest, key): """dump (decrypted, decompressed) repo objects""" marker = None i = 0 while True: result = repository.list(limit=LIST_SCAN_LIMIT, marker=marker) if not result: break marker = result[-1] for id in result: cdata = repository.get(id) give_id = id if id != Manifest.MANIFEST_ID else None data = key.decrypt(give_id, cdata) filename = '%06d_%s.obj' % (i, bin_to_hex(id)) print('Dumping', filename) with open(filename, 'wb') as fd: fd.write(data) i += 1 print('Done.') return EXIT_SUCCESS @with_repository(manifest=False) def do_debug_get_obj(self, args, repository): """get object contents from the repository and write it into file""" hex_id = args.id try: id = unhexlify(hex_id) except ValueError: print("object id %s is invalid." % hex_id) else: try: data = repository.get(id) except Repository.ObjectNotFound: print("object %s not found." % hex_id) else: with open(args.path, "wb") as f: f.write(data) print("object %s fetched." % hex_id) return EXIT_SUCCESS @with_repository(manifest=False, exclusive=True) def do_debug_put_obj(self, args, repository): """put file(s) contents into the repository""" for path in args.paths: with open(path, "rb") as f: data = f.read() h = sha256(data) # XXX hardcoded repository.put(h.digest(), data) print("object %s put." % h.hexdigest()) repository.commit() return EXIT_SUCCESS @with_repository(manifest=False, exclusive=True) def do_debug_delete_obj(self, args, repository): """delete the objects with the given IDs from the repo""" modified = False for hex_id in args.ids: try: id = unhexlify(hex_id) except ValueError: print("object id %s is invalid." % hex_id) else: try: repository.delete(id) modified = True print("object %s deleted." % hex_id) except Repository.ObjectNotFound: print("object %s not found." % hex_id) if modified: repository.commit() print('Done.') return EXIT_SUCCESS @with_repository(manifest=False, exclusive=True, cache=True, compatibility=Manifest.NO_OPERATION_CHECK) def do_debug_refcount_obj(self, args, repository, manifest, key, cache): """display refcounts for the objects with the given IDs""" for hex_id in args.ids: try: id = unhexlify(hex_id) except ValueError: print("object id %s is invalid." % hex_id) else: try: refcount = cache.chunks[id][0] print("object %s has %d referrers [info from chunks cache]." % (hex_id, refcount)) except KeyError: print("object %s not found [info from chunks cache]." % hex_id) return EXIT_SUCCESS @with_repository(lock=False, manifest=False) def do_break_lock(self, args, repository): """Break the repository lock (e.g. in case it was left by a dead borg.""" repository.break_lock() Cache.break_lock(repository) return self.exit_code helptext = collections.OrderedDict() helptext['patterns'] = textwrap.dedent(''' Exclusion patterns support four separate styles, fnmatch, shell, regular expressions and path prefixes. By default, fnmatch is used. If followed by a colon (':') the first two characters of a pattern are used as a style selector. Explicit style selection is necessary when a non-default style is desired or when the desired pattern starts with two alphanumeric characters followed by a colon (i.e. `aa:something/*`). `Fnmatch `_, selector `fm:` This is the default style. These patterns use a variant of shell pattern syntax, with '*' matching any number of characters, '?' matching any single character, '[...]' matching any single character specified, including ranges, and '[!...]' matching any character not specified. For the purpose of these patterns, the path separator ('\\' for Windows and '/' on other systems) is not treated specially. Wrap meta-characters in brackets for a literal match (i.e. `[?]` to match the literal character `?`). For a path to match a pattern, it must completely match from start to end, or must match from the start to just before a path separator. Except for the root path, paths will never end in the path separator when matching is attempted. Thus, if a given pattern ends in a path separator, a '*' is appended before matching is attempted. Shell-style patterns, selector `sh:` Like fnmatch patterns these are similar to shell patterns. The difference is that the pattern may include `**/` for matching zero or more directory levels, `*` for matching zero or more arbitrary characters with the exception of any path separator. Regular expressions, selector `re:` Regular expressions similar to those found in Perl are supported. Unlike shell patterns regular expressions are not required to match the complete path and any substring match is sufficient. It is strongly recommended to anchor patterns to the start ('^'), to the end ('$') or both. Path separators ('\\' for Windows and '/' on other systems) in paths are always normalized to a forward slash ('/') before applying a pattern. The regular expression syntax is described in the `Python documentation for the re module `_. Prefix path, selector `pp:` This pattern style is useful to match whole sub-directories. The pattern `pp:/data/bar` matches `/data/bar` and everything therein. .. note:: `re:`, `sh:` and `fm:` patterns are all implemented on top of the Python SRE engine. It is very easy to formulate patterns for each of these types which requires an inordinate amount of time to match paths. If untrusted users are able to supply patterns, ensure they cannot supply `re:` patterns. Further, ensure that `sh:` and `fm:` patterns only contain a handful of wildcards at most. Exclusions can be passed via the command line option `--exclude`. When used from within a shell the patterns should be quoted to protect them from expansion. The `--exclude-from` option permits loading exclusion patterns from a text file with one pattern per line. Lines empty or starting with the number sign ('#') after removing whitespace on both ends are ignored. The optional style selector prefix is also supported for patterns loaded from a file. Due to whitespace removal paths with whitespace at the beginning or end can only be excluded using regular expressions. Examples:: # Exclude '/home/user/file.o' but not '/home/user/file.odt': $ borg create -e '*.o' backup / # Exclude '/home/user/junk' and '/home/user/subdir/junk' but # not '/home/user/importantjunk' or '/etc/junk': $ borg create -e '/home/*/junk' backup / # Exclude the contents of '/home/user/cache' but not the directory itself: $ borg create -e /home/user/cache/ backup / # The file '/home/user/cache/important' is *not* backed up: $ borg create -e /home/user/cache/ backup / /home/user/cache/important # The contents of directories in '/home' are not backed up when their name # ends in '.tmp' $ borg create --exclude 're:^/home/[^/]+\.tmp/' backup / # Load exclusions from file $ cat >exclude.txt <", where char is "H", "d", "w", "m", "y". For example, "--keep-within 2d" means to keep all archives that were created within the past 48 hours. "1m" is taken to mean "31d". The archives kept with this option do not count towards the totals specified by any other options. If a prefix is set with -P, then only archives that start with the prefix are considered for deletion and only those archives count towards the totals specified by the rules. Otherwise, *all* archives in the repository are candidates for deletion! There is no automatic distinction between archives representing different contents. These need to be distinguished by specifying matching prefixes. """) subparser = subparsers.add_parser('prune', parents=[common_parser], description=self.do_prune.__doc__, epilog=prune_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='prune archives') subparser.set_defaults(func=self.do_prune) subparser.add_argument('-n', '--dry-run', dest='dry_run', default=False, action='store_true', help='do not change repository') subparser.add_argument('--force', dest='forced', action='store_true', default=False, help='force pruning of corrupted archives') subparser.add_argument('-s', '--stats', dest='stats', action='store_true', default=False, help='print statistics for the deleted archive. Requires -v/--verbose.') subparser.add_argument('--list', dest='output_list', action='store_true', default=False, help='output verbose list of archives it keeps/prunes. Requires -v/--verbose.') subparser.add_argument('--keep-within', dest='within', type=interval, metavar='INTERVAL', help='keep all archives within this time interval') subparser.add_argument('-H', '--keep-hourly', dest='hourly', type=int, default=0, help='number of hourly archives to keep') subparser.add_argument('-d', '--keep-daily', dest='daily', type=int, default=0, help='number of daily archives to keep') subparser.add_argument('-w', '--keep-weekly', dest='weekly', type=int, default=0, help='number of weekly archives to keep') subparser.add_argument('-m', '--keep-monthly', dest='monthly', type=int, default=0, help='number of monthly archives to keep') subparser.add_argument('-y', '--keep-yearly', dest='yearly', type=int, default=0, help='number of yearly archives to keep') subparser.add_argument('-P', '--prefix', dest='prefix', type=PrefixSpec, help='only consider archive names starting with this prefix') subparser.add_argument('--save-space', dest='save_space', action='store_true', default=False, help='work slower, but using less space') subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to prune') upgrade_epilog = textwrap.dedent(""" Upgrade an existing, local Borg repository. When you do not need borg upgrade +++++++++++++++++++++++++++++++++ Not every change requires that you run ``borg upgrade``. You do **not** need to run it when: - moving your repository to a different place - upgrading to another point release (like 1.0.x to 1.0.y), except when noted otherwise in the changelog - upgrading from 1.0.x to 1.1.x, except when noted otherwise in the changelog Borg 1.x.y upgrades +++++++++++++++++++ Use ``borg upgrade --tam REPO`` to require manifest authentication introduced with Borg 1.0.9 to address security issues. This means that modifying the repository after doing this with a version prior to 1.0.9 will raise a validation error, so only perform this upgrade after updating all clients using the repository to 1.0.9 or newer. This upgrade should be done on each client for safety reasons. If a repository is accidentally modified with a pre-1.0.9 client after this upgrade, use ``borg upgrade --tam --force REPO`` to remedy it. If you routinely do this you might not want to enable this upgrade (which will leave you exposed to the security issue). You can reverse the upgrade by issuing ``borg upgrade --disable-tam REPO``. See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability for details. Attic and Borg 0.xx to Borg 1.x +++++++++++++++++++++++++++++++ This currently supports converting an Attic repository to Borg and also helps with converting Borg 0.xx to 1.0. Currently, only LOCAL repositories can be upgraded (issue #465). Please note that ``borg create`` (since 1.0.0) uses bigger chunks by default than old borg or attic did, so the new chunks won't deduplicate with the old chunks in the upgraded repository. See ``--chunker-params`` option of ``borg create`` and ``borg recreate``. ``borg upgrade`` will change the magic strings in the repository's segments to match the new Borg magic strings. The keyfiles found in $ATTIC_KEYS_DIR or ~/.attic/keys/ will also be converted and copied to $BORG_KEYS_DIR or ~/.config/borg/keys. The cache files are converted, from $ATTIC_CACHE_DIR or ~/.cache/attic to $BORG_CACHE_DIR or ~/.cache/borg, but the cache layout between Borg and Attic changed, so it is possible the first backup after the conversion takes longer than expected due to the cache resync. Upgrade should be able to resume if interrupted, although it will still iterate over all segments. If you want to start from scratch, use `borg delete` over the copied repository to make sure the cache files are also removed: borg delete borg Unless ``--inplace`` is specified, the upgrade process first creates a backup copy of the repository, in REPOSITORY.upgrade-DATETIME, using hardlinks. This takes longer than in place upgrades, but is much safer and gives progress information (as opposed to ``cp -al``). Once you are satisfied with the conversion, you can safely destroy the backup copy. WARNING: Running the upgrade in place will make the current copy unusable with older version, with no way of going back to previous versions. This can PERMANENTLY DAMAGE YOUR REPOSITORY! Attic CAN NOT READ BORG REPOSITORIES, as the magic strings have changed. You have been warned.""") subparser = subparsers.add_parser('upgrade', parents=[common_parser], description=self.do_upgrade.__doc__, epilog=upgrade_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='upgrade repository format') subparser.set_defaults(func=self.do_upgrade) subparser.add_argument('-p', '--progress', dest='progress', action='store_true', default=False, help="""show progress display while upgrading the repository""") subparser.add_argument('-n', '--dry-run', dest='dry_run', default=False, action='store_true', help='do not change repository') subparser.add_argument('-i', '--inplace', dest='inplace', default=False, action='store_true', help="""rewrite repository in place, with no chance of going back to older versions of the repository.""") subparser.add_argument('--force', dest='force', action='store_true', help="""Force upgrade""") subparser.add_argument('--tam', dest='tam', action='store_true', help="""Enable manifest authentication (in key and cache) (Borg 1.0.9 and later)""") subparser.add_argument('--disable-tam', dest='disable_tam', action='store_true', help="""Disable manifest authentication (in key and cache)""") subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='path to the repository to be upgraded') subparser = subparsers.add_parser('help', parents=[common_parser], description='Extra help') subparser.add_argument('--epilog-only', dest='epilog_only', action='store_true', default=False) subparser.add_argument('--usage-only', dest='usage_only', action='store_true', default=False) subparser.set_defaults(func=functools.partial(self.do_help, parser, subparsers.choices)) subparser.add_argument('topic', metavar='TOPIC', type=str, nargs='?', help='additional help on TOPIC') debug_epilog = textwrap.dedent(""" These commands are not intended for normal use and potentially very dangerous if used incorrectly. They exist to improve debugging capabilities without direct system access, e.g. in case you ever run into some severe malfunction. Use them only if you know what you are doing or if a trusted developer tells you what to do.""") subparser = subparsers.add_parser('debug', parents=[common_parser], description='debugging command (not intended for normal use)', epilog=debug_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='debugging command (not intended for normal use)') debug_parsers = subparser.add_subparsers(title='required arguments', metavar='') subparser.set_defaults(fallback_func=functools.partial(self.do_subcommand_help, subparser)) debug_info_epilog = textwrap.dedent(""" This command displays some system information that might be useful for bug reports and debugging problems. If a traceback happens, this information is already appended at the end of the traceback. """) subparser = subparsers.add_parser('debug-info', parents=[common_parser], description=self.do_debug_info.__doc__, epilog=debug_info_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='show system infos for debugging / bug reports (debug)') subparser.set_defaults(func=self.do_debug_info) subparser = debug_parsers.add_parser('info', parents=[common_parser], description=self.do_debug_info.__doc__, epilog=debug_info_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='show system infos for debugging / bug reports (debug)') subparser.set_defaults(func=self.do_debug_info) debug_dump_archive_items_epilog = textwrap.dedent(""" This command dumps raw (but decrypted and decompressed) archive items (only metadata) to files. """) subparser = subparsers.add_parser('debug-dump-archive-items', parents=[common_parser], description=self.do_debug_dump_archive_items.__doc__, epilog=debug_dump_archive_items_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='dump archive items (metadata) (debug)') subparser.set_defaults(func=self.do_debug_dump_archive_items) subparser.add_argument('location', metavar='ARCHIVE', type=location_validator(archive=True), help='archive to dump') subparser = debug_parsers.add_parser('dump-archive-items', parents=[common_parser], description=self.do_debug_dump_archive_items.__doc__, epilog=debug_dump_archive_items_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='dump archive items (metadata) (debug)') subparser.set_defaults(func=self.do_debug_dump_archive_items) subparser.add_argument('location', metavar='ARCHIVE', type=location_validator(archive=True), help='archive to dump') debug_dump_repo_objs_epilog = textwrap.dedent(""" This command dumps raw (but decrypted and decompressed) repo objects to files. """) subparser = subparsers.add_parser('debug-dump-repo-objs', parents=[common_parser], description=self.do_debug_dump_repo_objs.__doc__, epilog=debug_dump_repo_objs_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='dump repo objects (debug)') subparser.set_defaults(func=self.do_debug_dump_repo_objs) subparser.add_argument('location', metavar='REPOSITORY', type=location_validator(archive=False), help='repo to dump') subparser = debug_parsers.add_parser('dump-repo-objs', parents=[common_parser], description=self.do_debug_dump_repo_objs.__doc__, epilog=debug_dump_repo_objs_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='dump repo objects (debug)') subparser.set_defaults(func=self.do_debug_dump_repo_objs) subparser.add_argument('location', metavar='REPOSITORY', type=location_validator(archive=False), help='repo to dump') debug_get_obj_epilog = textwrap.dedent(""" This command gets an object from the repository. """) subparser = subparsers.add_parser('debug-get-obj', parents=[common_parser], description=self.do_debug_get_obj.__doc__, epilog=debug_get_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='get object from repository (debug)') subparser.set_defaults(func=self.do_debug_get_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('id', metavar='ID', type=str, help='hex object ID to get from the repo') subparser.add_argument('path', metavar='PATH', type=str, help='file to write object data into') subparser = debug_parsers.add_parser('get-obj', parents=[common_parser], description=self.do_debug_get_obj.__doc__, epilog=debug_get_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='get object from repository (debug)') subparser.set_defaults(func=self.do_debug_get_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('id', metavar='ID', type=str, help='hex object ID to get from the repo') subparser.add_argument('path', metavar='PATH', type=str, help='file to write object data into') debug_put_obj_epilog = textwrap.dedent(""" This command puts objects into the repository. """) subparser = subparsers.add_parser('debug-put-obj', parents=[common_parser], description=self.do_debug_put_obj.__doc__, epilog=debug_put_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='put object to repository (debug)') subparser.set_defaults(func=self.do_debug_put_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('paths', metavar='PATH', nargs='+', type=str, help='file(s) to read and create object(s) from') subparser = debug_parsers.add_parser('put-obj', parents=[common_parser], description=self.do_debug_put_obj.__doc__, epilog=debug_put_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='put object to repository (debug)') subparser.set_defaults(func=self.do_debug_put_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('paths', metavar='PATH', nargs='+', type=str, help='file(s) to read and create object(s) from') debug_delete_obj_epilog = textwrap.dedent(""" This command deletes objects from the repository. """) subparser = subparsers.add_parser('debug-delete-obj', parents=[common_parser], description=self.do_debug_delete_obj.__doc__, epilog=debug_delete_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='delete object from repository (debug)') subparser.set_defaults(func=self.do_debug_delete_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('ids', metavar='IDs', nargs='+', type=str, help='hex object ID(s) to delete from the repo') subparser = debug_parsers.add_parser('delete-obj', parents=[common_parser], description=self.do_debug_delete_obj.__doc__, epilog=debug_delete_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='delete object from repository (debug)') subparser.set_defaults(func=self.do_debug_delete_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('ids', metavar='IDs', nargs='+', type=str, help='hex object ID(s) to delete from the repo') debug_refcount_obj_epilog = textwrap.dedent(""" This command displays the reference count for objects from the repository. """) subparser = subparsers.add_parser('debug-refcount-obj', parents=[common_parser], description=self.do_debug_refcount_obj.__doc__, epilog=debug_refcount_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='show refcount for object from repository (debug)') subparser.set_defaults(func=self.do_debug_refcount_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('ids', metavar='IDs', nargs='+', type=str, help='hex object ID(s) to show refcounts for') subparser = debug_parsers.add_parser('refcount-obj', parents=[common_parser], description=self.do_debug_refcount_obj.__doc__, epilog=debug_refcount_obj_epilog, formatter_class=argparse.RawDescriptionHelpFormatter, help='show refcount for object from repository (debug)') subparser.set_defaults(func=self.do_debug_refcount_obj) subparser.add_argument('location', metavar='REPOSITORY', nargs='?', default='', type=location_validator(archive=False), help='repository to use') subparser.add_argument('ids', metavar='IDs', nargs='+', type=str, help='hex object ID(s) to show refcounts for') return parser def get_args(self, argv, cmd): """usually, just returns argv, except if we deal with a ssh forced command for borg serve.""" result = self.parse_args(argv[1:]) if cmd is not None and result.func == self.do_serve: forced_result = result argv = shlex.split(cmd) # Drop environment variables (do *not* interpret them) before trying to parse # the borg command line. argv = list(itertools.dropwhile(lambda arg: '=' in arg, argv)) result = self.parse_args(argv[1:]) if result.func != forced_result.func: # someone is trying to execute a different borg subcommand, don't do that! return forced_result # we only take specific options from the forced "borg serve" command: result.restrict_to_paths = forced_result.restrict_to_paths result.append_only = forced_result.append_only return result def parse_args(self, args=None): # We can't use argparse for "serve" since we don't want it to show up in "Available commands" if args: args = self.preprocess_args(args) parser = self.build_parser(args) args = parser.parse_args(args or ['-h']) update_excludes(args) return args def prerun_checks(self, logger): check_python() check_extension_modules() def run(self, args): os.umask(args.umask) # early, before opening files self.lock_wait = args.lock_wait # This works around http://bugs.python.org/issue9351 func = getattr(args, 'func', None) or getattr(args, 'fallback_func') setup_logging(level=args.log_level, is_serve=func == self.do_serve) # do not use loggers before this! self.prerun_checks(logger) if is_slow_msgpack(): logger.warning("Using a pure-python msgpack! This will result in lower performance.") return set_ec(func(args)) def sig_info_handler(sig_no, stack): # pragma: no cover """search the stack for infos about the currently processed file and print them""" with signal_handler(sig_no, signal.SIG_IGN): for frame in inspect.getouterframes(stack): func, loc = frame[3], frame[0].f_locals if func in ('process_file', '_process', ): # create op path = loc['path'] try: pos = loc['fd'].tell() total = loc['st'].st_size except Exception: pos, total = 0, 0 logger.info("{0} {1}/{2}".format(path, format_file_size(pos), format_file_size(total))) break if func in ('extract_item', ): # extract op path = loc['item'][b'path'] try: pos = loc['fd'].tell() except Exception: pos = 0 logger.info("{0} {1}/???".format(path, format_file_size(pos))) break def sig_trace_handler(sig_no, stack): # pragma: no cover print('\nReceived SIGUSR2 at %s, dumping trace...' % datetime.now().replace(microsecond=0), file=sys.stderr) faulthandler.dump_traceback() def main(): # pragma: no cover # Make sure stdout and stderr have errors='replace') to avoid unicode # issues when print()-ing unicode file names sys.stdout = ErrorIgnoringTextIOWrapper(sys.stdout.buffer, sys.stdout.encoding, 'replace', line_buffering=True) sys.stderr = ErrorIgnoringTextIOWrapper(sys.stderr.buffer, sys.stderr.encoding, 'replace', line_buffering=True) # If we receive SIGINT (ctrl-c), SIGTERM (kill) or SIGHUP (kill -HUP), # catch them and raise a proper exception that can be handled for an # orderly exit. # SIGHUP is important especially for systemd systems, where logind # sends it when a session exits, in addition to any traditional use. # Output some info if we receive SIGUSR1 or SIGINFO (ctrl-t). # Register fault handler for SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL. faulthandler.enable() with signal_handler('SIGINT', raising_signal_handler(KeyboardInterrupt)), \ signal_handler('SIGHUP', raising_signal_handler(SigHup)), \ signal_handler('SIGTERM', raising_signal_handler(SigTerm)), \ signal_handler('SIGUSR1', sig_info_handler), \ signal_handler('SIGUSR2', sig_trace_handler), \ signal_handler('SIGINFO', sig_info_handler): archiver = Archiver() msg = None try: args = archiver.get_args(sys.argv, os.environ.get('SSH_ORIGINAL_COMMAND')) except Error as e: msg = e.get_message() if e.traceback: msg += "\n%s\n%s" % (traceback.format_exc(), sysinfo()) # we might not have logging setup yet, so get out quickly print(msg, file=sys.stderr) sys.exit(e.exit_code) try: exit_code = archiver.run(args) except Error as e: msg = e.get_message() if e.traceback: msg += "\n%s\n%s" % (traceback.format_exc(), sysinfo()) exit_code = e.exit_code except RemoteRepository.RPCError as e: msg = '%s\n%s' % (str(e), sysinfo()) exit_code = EXIT_ERROR except Exception: msg = 'Local Exception.\n%s\n%s' % (traceback.format_exc(), sysinfo()) exit_code = EXIT_ERROR except KeyboardInterrupt: msg = 'Keyboard interrupt.\n%s\n%s' % (traceback.format_exc(), sysinfo()) exit_code = EXIT_ERROR except SigTerm: msg = 'Received SIGTERM.' exit_code = EXIT_ERROR except SigHup: msg = 'Received SIGHUP.' exit_code = EXIT_ERROR if msg: logger.error(msg) if args.show_rc: exit_msg = 'terminating with %s status, rc %d' if exit_code == EXIT_SUCCESS: logger.info(exit_msg % ('success', exit_code)) elif exit_code == EXIT_WARNING: logger.warning(exit_msg % ('warning', exit_code)) elif exit_code == EXIT_ERROR: logger.error(exit_msg % ('error', exit_code)) else: # if you see 666 in output, it usually means exit_code was None logger.error(exit_msg % ('abnormal', exit_code or 666)) sys.exit(exit_code) if __name__ == '__main__': main() borgbackup-1.0.13/borg/cache.py0000644000175000017500000005776413431513133016257 0ustar useruser00000000000000import configparser from .remote import cache_if_remote from collections import namedtuple import os import stat from binascii import unhexlify import shutil from .key import PlaintextKey from .logger import create_logger logger = create_logger() from .helpers import Error, Manifest, get_cache_dir, decode_dict, int_to_bigint, \ bigint_to_int, format_file_size, yes, bin_to_hex, Location, safe_ns, parse_stringified_list from .locking import Lock from .hashindex import ChunkIndex import msgpack class Cache: """Client Side cache """ class RepositoryIDNotUnique(Error): """Cache is newer than repository - do you have multiple, independently updated repos with same ID?""" class RepositoryReplay(Error): """Cache is newer than repository - this is either an attack or unsafe (multiple repos with same ID)""" class CacheInitAbortedError(Error): """Cache initialization aborted""" class RepositoryAccessAborted(Error): """Repository access aborted""" class EncryptionMethodMismatch(Error): """Repository encryption method changed since last access, refusing to continue""" @staticmethod def break_lock(repository, path=None): path = path or os.path.join(get_cache_dir(), bin_to_hex(repository.id)) Lock(os.path.join(path, 'lock'), exclusive=True).break_lock() @staticmethod def destroy(repository, path=None): """destroy the cache for ``repository`` or at ``path``""" path = path or os.path.join(get_cache_dir(), bin_to_hex(repository.id)) config = os.path.join(path, 'config') if os.path.exists(config): os.remove(config) # kill config first shutil.rmtree(path) def __init__(self, repository, key, manifest, path=None, sync=True, do_files=False, warn_if_unencrypted=True, lock_wait=None): """ :param lock_wait: timeout for lock acquisition (int [s] or None [wait forever]) """ self.lock = None self.timestamp = None self.lock = None self.txn_active = False self.repository = repository self.key = key self.manifest = manifest self.path = path or os.path.join(get_cache_dir(), bin_to_hex(repository.id)) self.do_files = do_files # Warn user before sending data to a never seen before unencrypted repository if not os.path.exists(self.path): if warn_if_unencrypted and isinstance(key, PlaintextKey): msg = ("Warning: Attempting to access a previously unknown unencrypted repository!" + "\n" + "Do you want to continue? [yN] ") if not yes(msg, false_msg="Aborting.", invalid_msg="Invalid answer, aborting.", retry=False, env_var_override='BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK'): raise self.CacheInitAbortedError() self.create() self.open(lock_wait=lock_wait) try: # Warn user before sending data to a relocated repository if self.previous_location and self.previous_location != repository._location.canonical_path(): msg = ("Warning: The repository at location {} was previously located at {}".format(repository._location.canonical_path(), self.previous_location) + "\n" + "Do you want to continue? [yN] ") if not yes(msg, false_msg="Aborting.", invalid_msg="Invalid answer, aborting.", retry=False, env_var_override='BORG_RELOCATED_REPO_ACCESS_IS_OK'): raise self.RepositoryAccessAborted() # adapt on-disk config immediately if the new location was accepted self.begin_txn() self.commit() if not self.check_cache_compatibility(): self.wipe_cache() self.update_compatibility() if sync and self.manifest.id != self.manifest_id: # If repository is older than the cache something fishy is going on if self.timestamp and self.timestamp > manifest.timestamp: if isinstance(key, PlaintextKey): raise self.RepositoryIDNotUnique() else: raise self.RepositoryReplay() # Make sure an encrypted repository has not been swapped for an unencrypted repository if self.key_type is not None and self.key_type != str(key.TYPE): raise self.EncryptionMethodMismatch() self.sync() self.commit() except: self.close() raise def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): self.close() def __str__(self): fmt = """\ All archives: {0.total_size:>20s} {0.total_csize:>20s} {0.unique_csize:>20s} Unique chunks Total chunks Chunk index: {0.total_unique_chunks:20d} {0.total_chunks:20d}""" return fmt.format(self.format_tuple()) def format_tuple(self): # XXX: this should really be moved down to `hashindex.pyx` Summary = namedtuple('Summary', ['total_size', 'total_csize', 'unique_size', 'unique_csize', 'total_unique_chunks', 'total_chunks']) stats = Summary(*self.chunks.summarize())._asdict() for field in ['total_size', 'total_csize', 'unique_csize']: stats[field] = format_file_size(stats[field]) return Summary(**stats) def create(self): """Create a new empty cache at `self.path` """ os.makedirs(self.path) with open(os.path.join(self.path, 'README'), 'w') as fd: fd.write('This is a Borg cache') config = configparser.ConfigParser(interpolation=None) config.add_section('cache') config.set('cache', 'version', '1') config.set('cache', 'repository', bin_to_hex(self.repository.id)) config.set('cache', 'manifest', '') with open(os.path.join(self.path, 'config'), 'w') as fd: config.write(fd) ChunkIndex().write(os.path.join(self.path, 'chunks').encode('utf-8')) os.makedirs(os.path.join(self.path, 'chunks.archive.d')) with open(os.path.join(self.path, 'files'), 'wb') as fd: pass # empty file def _check_upgrade(self, config_path): try: cache_version = self.config.getint('cache', 'version') wanted_version = 1 if cache_version != wanted_version: self.close() raise Exception('%s has unexpected cache version %d (wanted: %d).' % ( config_path, cache_version, wanted_version)) except configparser.NoSectionError: self.close() raise Exception('%s does not look like a Borg cache.' % config_path) from None # borg < 1.0.8rc1 had different canonicalization for the repo location (see #1655 and #1741). cache_loc = self.config.get('cache', 'previous_location', fallback=None) if cache_loc: repo_loc = self.repository._location.canonical_path() rl = Location(repo_loc) cl = Location(cache_loc) if cl.proto == rl.proto and cl.user == rl.user and cl.host == rl.host and cl.port == rl.port \ and \ cl.path and rl.path and \ cl.path.startswith('/~/') and rl.path.startswith('/./') and cl.path[3:] == rl.path[3:]: # everything is same except the expected change in relative path canonicalization, # update previous_location to avoid warning / user query about changed location: self.config.set('cache', 'previous_location', repo_loc) def _do_open(self): self.config = configparser.ConfigParser(interpolation=None) config_path = os.path.join(self.path, 'config') self.config.read(config_path) self._check_upgrade(config_path) self.id = self.config.get('cache', 'repository') self.manifest_id = unhexlify(self.config.get('cache', 'manifest')) self.timestamp = self.config.get('cache', 'timestamp', fallback=None) self.key_type = self.config.get('cache', 'key_type', fallback=None) self.previous_location = self.config.get('cache', 'previous_location', fallback=None) self.ignored_features = set(parse_stringified_list(self.config.get('cache', 'ignored_features', fallback=''))) self.mandatory_features = set(parse_stringified_list(self.config.get('cache', 'mandatory_features', fallback=''))) self.chunks = ChunkIndex.read(os.path.join(self.path, 'chunks').encode('utf-8')) self.files = None def open(self, lock_wait=None): if not os.path.isdir(self.path): raise Exception('%s Does not look like a Borg cache' % self.path) self.lock = Lock(os.path.join(self.path, 'lock'), exclusive=True, timeout=lock_wait).acquire() self.rollback() def close(self): if self.lock is not None: self.lock.release() self.lock = None def _read_files(self): self.files = {} self._newest_mtime = None logger.debug('Reading files cache ...') with open(os.path.join(self.path, 'files'), 'rb') as fd: u = msgpack.Unpacker(use_list=True) while True: data = fd.read(64 * 1024) if not data: break u.feed(data) try: for path_hash, item in u: item[0] += 1 # in the end, this takes about 240 Bytes per file self.files[path_hash] = msgpack.packb(item) except (TypeError, ValueError) as exc: logger.warning('The files cache seems corrupt, ignoring it. ' 'Expect lower performance. [%s]' % str(exc)) self.files = {} return def begin_txn(self): # Initialize transaction snapshot txn_dir = os.path.join(self.path, 'txn.tmp') os.mkdir(txn_dir) shutil.copy(os.path.join(self.path, 'config'), txn_dir) shutil.copy(os.path.join(self.path, 'chunks'), txn_dir) shutil.copy(os.path.join(self.path, 'files'), txn_dir) os.rename(os.path.join(self.path, 'txn.tmp'), os.path.join(self.path, 'txn.active')) self.txn_active = True def commit(self): """Commit transaction """ if not self.txn_active: return if self.files is not None: if self._newest_mtime is None: # was never set because no files were modified/added self._newest_mtime = 2 ** 63 - 1 # nanoseconds, good until y2262 ttl = int(os.environ.get('BORG_FILES_CACHE_TTL', 20)) with open(os.path.join(self.path, 'files'), 'wb') as fd: for path_hash, item in self.files.items(): # Only keep files seen in this backup that are older than newest mtime seen in this backup - # this is to avoid issues with filesystem snapshots and mtime granularity. # Also keep files from older backups that have not reached BORG_FILES_CACHE_TTL yet. item = msgpack.unpackb(item) age = item[0] if age == 0 and bigint_to_int(item[3]) < self._newest_mtime or \ age > 0 and age < ttl: msgpack.pack((path_hash, item), fd) self.config.set('cache', 'manifest', bin_to_hex(self.manifest.id)) self.config.set('cache', 'timestamp', self.manifest.timestamp) self.config.set('cache', 'key_type', str(self.key.TYPE)) self.config.set('cache', 'previous_location', self.repository._location.canonical_path()) self.config.set('cache', 'ignored_features', ','.join(self.ignored_features)) self.config.set('cache', 'mandatory_features', ','.join(self.mandatory_features)) with open(os.path.join(self.path, 'config'), 'w') as fd: self.config.write(fd) self.chunks.write(os.path.join(self.path, 'chunks').encode('utf-8')) os.rename(os.path.join(self.path, 'txn.active'), os.path.join(self.path, 'txn.tmp')) shutil.rmtree(os.path.join(self.path, 'txn.tmp')) self.txn_active = False def rollback(self): """Roll back partial and aborted transactions """ # Remove partial transaction if os.path.exists(os.path.join(self.path, 'txn.tmp')): shutil.rmtree(os.path.join(self.path, 'txn.tmp')) # Roll back active transaction txn_dir = os.path.join(self.path, 'txn.active') if os.path.exists(txn_dir): shutil.copy(os.path.join(txn_dir, 'config'), self.path) shutil.copy(os.path.join(txn_dir, 'chunks'), self.path) shutil.copy(os.path.join(txn_dir, 'files'), self.path) os.rename(txn_dir, os.path.join(self.path, 'txn.tmp')) if os.path.exists(os.path.join(self.path, 'txn.tmp')): shutil.rmtree(os.path.join(self.path, 'txn.tmp')) self.txn_active = False self._do_open() def sync(self): """Re-synchronize chunks cache with repository. Maintains a directory with known backup archive indexes, so it only needs to fetch infos from repo and build a chunk index once per backup archive. If out of sync, missing archive indexes get added, outdated indexes get removed and a new master chunks index is built by merging all archive indexes. """ archive_path = os.path.join(self.path, 'chunks.archive.d') def mkpath(id, suffix=''): id_hex = bin_to_hex(id) path = os.path.join(archive_path, id_hex + suffix) return path.encode('utf-8') def cached_archives(): if self.do_cache: fns = os.listdir(archive_path) # filenames with 64 hex digits == 256bit return set(unhexlify(fn) for fn in fns if len(fn) == 64) else: return set() def repo_archives(): return set(info[b'id'] for info in self.manifest.archives.values()) def cleanup_outdated(ids): for id in ids: os.unlink(mkpath(id)) def fetch_and_build_idx(archive_id, repository, key): chunk_idx = ChunkIndex() cdata = repository.get(archive_id) data = key.decrypt(archive_id, cdata) chunk_idx.add(archive_id, 1, len(data), len(cdata)) archive = msgpack.unpackb(data) if archive[b'version'] != 1: raise Exception('Unknown archive metadata version') decode_dict(archive, (b'name',)) unpacker = msgpack.Unpacker() for item_id, chunk in zip(archive[b'items'], repository.get_many(archive[b'items'])): data = key.decrypt(item_id, chunk) chunk_idx.add(item_id, 1, len(data), len(chunk)) unpacker.feed(data) for item in unpacker: if not isinstance(item, dict): logger.error('Error: Did not get expected metadata dict - archive corrupted!') continue if b'chunks' in item: for chunk_id, size, csize in item[b'chunks']: chunk_idx.add(chunk_id, 1, size, csize) if self.do_cache: fn = mkpath(archive_id) fn_tmp = mkpath(archive_id, suffix='.tmp') try: chunk_idx.write(fn_tmp) except Exception: os.unlink(fn_tmp) else: os.rename(fn_tmp, fn) return chunk_idx def lookup_name(archive_id): for name, info in self.manifest.archives.items(): if info[b'id'] == archive_id: return name def create_master_idx(chunk_idx): logger.info('Synchronizing chunks cache...') cached_ids = cached_archives() archive_ids = repo_archives() logger.info('Archives: %d, w/ cached Idx: %d, w/ outdated Idx: %d, w/o cached Idx: %d.' % ( len(archive_ids), len(cached_ids), len(cached_ids - archive_ids), len(archive_ids - cached_ids), )) # deallocates old hashindex, creates empty hashindex: chunk_idx.clear() cleanup_outdated(cached_ids - archive_ids) if archive_ids: chunk_idx = None for archive_id in archive_ids: archive_name = lookup_name(archive_id) if archive_id in cached_ids: archive_chunk_idx_path = mkpath(archive_id) logger.info("Reading cached archive chunk index for %s ..." % archive_name) archive_chunk_idx = ChunkIndex.read(archive_chunk_idx_path) else: logger.info('Fetching and building archive index for %s ...' % archive_name) archive_chunk_idx = fetch_and_build_idx(archive_id, repository, self.key) logger.info("Merging into master chunks index ...") if chunk_idx is None: # we just use the first archive's idx as starting point, # to avoid growing the hash table from 0 size and also # to save 1 merge call. chunk_idx = archive_chunk_idx else: chunk_idx.merge(archive_chunk_idx) logger.info('Done.') return chunk_idx def legacy_cleanup(): """bring old cache dirs into the desired state (cleanup and adapt)""" try: os.unlink(os.path.join(self.path, 'chunks.archive')) except: pass try: os.unlink(os.path.join(self.path, 'chunks.archive.tmp')) except: pass try: os.mkdir(archive_path) except: pass # The cache can be used by a command that e.g. only checks against Manifest.Operation.WRITE, # which does not have to include all flags from Manifest.Operation.READ. # Since the sync will attempt to read archives, check compatibility with Manifest.Operation.READ. self.manifest.check_repository_compatibility((Manifest.Operation.READ, )) self.begin_txn() with cache_if_remote(self.repository) as repository: legacy_cleanup() # TEMPORARY HACK: to avoid archive index caching, create a FILE named ~/.cache/borg/REPOID/chunks.archive.d - # this is only recommended if you have a fast, low latency connection to your repo (e.g. if repo is local disk) self.do_cache = os.path.isdir(archive_path) self.chunks = create_master_idx(self.chunks) def check_cache_compatibility(self): my_features = Manifest.SUPPORTED_REPO_FEATURES if self.ignored_features & my_features: # The cache might not contain references of chunks that need a feature that is mandatory for some operation # and which this version supports. To avoid corruption while executing that operation force rebuild. return False if not self.mandatory_features <= my_features: # The cache was build with consideration to at least one feature that this version does not understand. # This client might misinterpret the cache. Thus force a rebuild. return False return True def wipe_cache(self): logger.warning("Discarding incompatible cache and forcing a cache rebuild") archive_path = os.path.join(self.path, 'chunks.archive.d') if os.path.isdir(archive_path): shutil.rmtree(os.path.join(self.path, 'chunks.archive.d')) os.makedirs(os.path.join(self.path, 'chunks.archive.d')) self.chunks = ChunkIndex() with open(os.path.join(self.path, 'files'), 'wb'): pass # empty file self.manifest_id = '' self.config.set('cache', 'manifest', '') self.ignored_features = set() self.mandatory_features = set() def update_compatibility(self): operation_to_features_map = self.manifest.get_all_mandatory_features() my_features = Manifest.SUPPORTED_REPO_FEATURES repo_features = set() for operation, features in operation_to_features_map.items(): repo_features.update(features) self.ignored_features.update(repo_features - my_features) self.mandatory_features.update(repo_features & my_features) def add_chunk(self, id, data, stats): if not self.txn_active: self.begin_txn() size = len(data) if self.seen_chunk(id, size): return self.chunk_incref(id, stats) data = self.key.encrypt(data) csize = len(data) self.repository.put(id, data, wait=False) self.chunks[id] = (1, size, csize) stats.update(size, csize, True) return id, size, csize def seen_chunk(self, id, size=None): refcount, stored_size, _ = self.chunks.get(id, (0, None, None)) if size is not None and stored_size is not None and size != stored_size: # we already have a chunk with that id, but different size. # this is either a hash collision (unlikely) or corruption or a bug. raise Exception("chunk has same id [%r], but different size (stored: %d new: %d)!" % ( id, stored_size, size)) return refcount def chunk_incref(self, id, stats): if not self.txn_active: self.begin_txn() count, size, csize = self.chunks.incref(id) stats.update(size, csize, False) return id, size, csize def chunk_decref(self, id, stats): if not self.txn_active: self.begin_txn() count, size, csize = self.chunks.decref(id) if count == 0: del self.chunks[id] self.repository.delete(id, wait=False) stats.update(-size, -csize, True) else: stats.update(-size, -csize, False) def file_known_and_unchanged(self, path_hash, st, ignore_inode=False): if not (self.do_files and stat.S_ISREG(st.st_mode)): return None if self.files is None: self._read_files() entry = self.files.get(path_hash) if not entry: return None entry = msgpack.unpackb(entry) if (entry[2] == st.st_size and bigint_to_int(entry[3]) == st.st_mtime_ns and (ignore_inode or entry[1] == st.st_ino)): # reset entry age entry[0] = 0 # we ignored the inode number in the comparison above or it is still same. # if it is still the same, replacing it doesn't change it. # if we ignored it, a reason for doing that is that files were moved to a new # disk / new fs (so a one-time change of inode number is expected) and we wanted # to avoid everything getting chunked again. to be able to re-enable the inode # number comparison in a future backup run (and avoid chunking everything # again at that time), we need to update the inode number in the cache with what # we see in the filesystem. entry[1] = st.st_ino self.files[path_hash] = msgpack.packb(entry) return entry[4] else: return None def memorize_file(self, path_hash, st, ids): if not (self.do_files and stat.S_ISREG(st.st_mode)): return # Entry: Age, inode, size, mtime, chunk ids mtime_ns = safe_ns(st.st_mtime_ns) self.files[path_hash] = msgpack.packb((0, st.st_ino, st.st_size, int_to_bigint(mtime_ns), ids)) self._newest_mtime = max(self._newest_mtime or 0, mtime_ns) borgbackup-1.0.13/borg/chunker.c0000644000175000017500000063510213431514213016431 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__chunker #define __PYX_HAVE_API__borg__chunker /* Early includes */ #include #include #include "_chunker.c" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "stringsource", "borg/chunker.pyx", }; /*--- Type declarations ---*/ struct __pyx_obj_4borg_7chunker_Chunker; /* "borg/chunker.pyx":20 * * * cdef class Chunker: # <<<<<<<<<<<<<< * cdef _Chunker *chunker * */ struct __pyx_obj_4borg_7chunker_Chunker { PyObject_HEAD Chunker *chunker; }; /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* RaiseException.proto */ static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause); /* PyObject_GenericGetAttrNoDict.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttrNoDict PyObject_GenericGetAttr #endif /* PyObject_GenericGetAttr.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttr PyObject_GenericGetAttr #endif /* SetupReduce.proto */ static int __Pyx_setup_reduce(PyObject* type_obj); /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_unsigned_long(unsigned long value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_uint32_t(uint32_t value); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE unsigned long __Pyx_PyInt_As_unsigned_long(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE uint32_t __Pyx_PyInt_As_uint32_t(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE unsigned char __Pyx_PyInt_As_unsigned_char(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE size_t __Pyx_PyInt_As_size_t(PyObject *); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); /* Module declarations from 'libc.string' */ /* Module declarations from 'libc.stdlib' */ /* Module declarations from 'borg.chunker' */ static PyTypeObject *__pyx_ptype_4borg_7chunker_Chunker = 0; #define __Pyx_MODULE_NAME "borg.chunker" extern int __pyx_module_is_main_borg__chunker; int __pyx_module_is_main_borg__chunker = 0; /* Implementation of 'borg.chunker' */ static PyObject *__pyx_builtin_TypeError; static const char __pyx_k_fd[] = "fd"; static const char __pyx_k_fh[] = "fh"; static const char __pyx_k_add[] = "add"; static const char __pyx_k_len[] = "len"; static const char __pyx_k_sum[] = "sum"; static const char __pyx_k_data[] = "data"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_name[] = "__name__"; static const char __pyx_k_seed[] = "seed"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_table[] = "table"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_reduce[] = "__reduce__"; static const char __pyx_k_remove[] = "remove"; static const char __pyx_k_Chunker[] = "Chunker"; static const char __pyx_k_buzhash[] = "buzhash"; static const char __pyx_k_getstate[] = "__getstate__"; static const char __pyx_k_setstate[] = "__setstate__"; static const char __pyx_k_TypeError[] = "TypeError"; static const char __pyx_k_reduce_ex[] = "__reduce_ex__"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_borg_chunker[] = "borg.chunker"; static const char __pyx_k_chunk_max_exp[] = "chunk_max_exp"; static const char __pyx_k_chunk_min_exp[] = "chunk_min_exp"; static const char __pyx_k_reduce_cython[] = "__reduce_cython__"; static const char __pyx_k_buzhash_update[] = "buzhash_update"; static const char __pyx_k_hash_mask_bits[] = "hash_mask_bits"; static const char __pyx_k_setstate_cython[] = "__setstate_cython__"; static const char __pyx_k_borg_chunker_pyx[] = "borg/chunker.pyx"; static const char __pyx_k_hash_window_size[] = "hash_window_size"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_no_default___reduce___due_to_non[] = "no default __reduce__ due to non-trivial __cinit__"; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_n_s_Chunker; static PyObject *__pyx_n_s_TypeError; static PyObject *__pyx_n_s_add; static PyObject *__pyx_n_s_borg_chunker; static PyObject *__pyx_kp_s_borg_chunker_pyx; static PyObject *__pyx_n_s_buzhash; static PyObject *__pyx_n_s_buzhash_update; static PyObject *__pyx_n_s_chunk_max_exp; static PyObject *__pyx_n_s_chunk_min_exp; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_data; static PyObject *__pyx_n_s_fd; static PyObject *__pyx_n_s_fh; static PyObject *__pyx_n_s_getstate; static PyObject *__pyx_n_s_hash_mask_bits; static PyObject *__pyx_n_s_hash_window_size; static PyObject *__pyx_n_s_len; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_name; static PyObject *__pyx_kp_s_no_default___reduce___due_to_non; static PyObject *__pyx_n_s_reduce; static PyObject *__pyx_n_s_reduce_cython; static PyObject *__pyx_n_s_reduce_ex; static PyObject *__pyx_n_s_remove; static PyObject *__pyx_n_s_seed; static PyObject *__pyx_n_s_setstate; static PyObject *__pyx_n_s_setstate_cython; static PyObject *__pyx_n_s_sum; static PyObject *__pyx_n_s_table; static PyObject *__pyx_n_s_test; static int __pyx_pf_4borg_7chunker_7Chunker___cinit__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self, int __pyx_v_seed, int __pyx_v_chunk_min_exp, int __pyx_v_chunk_max_exp, int __pyx_v_hash_mask_bits, int __pyx_v_hash_window_size); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_7Chunker_2chunkify(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self, PyObject *__pyx_v_fd, PyObject *__pyx_v_fh); /* proto */ static void __pyx_pf_4borg_7chunker_7Chunker_4__dealloc__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_7Chunker_6__iter__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_7Chunker_8__next__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_7Chunker_10__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_7Chunker_12__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_buzhash(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_data, unsigned long __pyx_v_seed); /* proto */ static PyObject *__pyx_pf_4borg_7chunker_2buzhash_update(CYTHON_UNUSED PyObject *__pyx_self, uint32_t __pyx_v_sum, unsigned char __pyx_v_remove, unsigned char __pyx_v_add, size_t __pyx_v_len, unsigned long __pyx_v_seed); /* proto */ static PyObject *__pyx_tp_new_4borg_7chunker_Chunker(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_int_4294967295; static PyObject *__pyx_int_neg_1; static PyObject *__pyx_tuple_; static PyObject *__pyx_tuple__2; static PyObject *__pyx_tuple__3; static PyObject *__pyx_tuple__5; static PyObject *__pyx_codeobj__4; static PyObject *__pyx_codeobj__6; /* Late includes */ /* "borg/chunker.pyx":23 * cdef _Chunker *chunker * * def __cinit__(self, int seed, int chunk_min_exp, int chunk_max_exp, int hash_mask_bits, int hash_window_size): # <<<<<<<<<<<<<< * min_size = 1 << chunk_min_exp * max_size = 1 << chunk_max_exp */ /* Python wrapper */ static int __pyx_pw_4borg_7chunker_7Chunker_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static int __pyx_pw_4borg_7chunker_7Chunker_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { int __pyx_v_seed; int __pyx_v_chunk_min_exp; int __pyx_v_chunk_max_exp; int __pyx_v_hash_mask_bits; int __pyx_v_hash_window_size; int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_seed,&__pyx_n_s_chunk_min_exp,&__pyx_n_s_chunk_max_exp,&__pyx_n_s_hash_mask_bits,&__pyx_n_s_hash_window_size,0}; PyObject* values[5] = {0,0,0,0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4); CYTHON_FALLTHROUGH; case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_seed)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_chunk_min_exp)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 5, 5, 1); __PYX_ERR(1, 23, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_chunk_max_exp)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 5, 5, 2); __PYX_ERR(1, 23, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (likely((values[3] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_hash_mask_bits)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 5, 5, 3); __PYX_ERR(1, 23, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 4: if (likely((values[4] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_hash_window_size)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 5, 5, 4); __PYX_ERR(1, 23, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(1, 23, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 5) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[3] = PyTuple_GET_ITEM(__pyx_args, 3); values[4] = PyTuple_GET_ITEM(__pyx_args, 4); } __pyx_v_seed = __Pyx_PyInt_As_int(values[0]); if (unlikely((__pyx_v_seed == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 23, __pyx_L3_error) __pyx_v_chunk_min_exp = __Pyx_PyInt_As_int(values[1]); if (unlikely((__pyx_v_chunk_min_exp == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 23, __pyx_L3_error) __pyx_v_chunk_max_exp = __Pyx_PyInt_As_int(values[2]); if (unlikely((__pyx_v_chunk_max_exp == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 23, __pyx_L3_error) __pyx_v_hash_mask_bits = __Pyx_PyInt_As_int(values[3]); if (unlikely((__pyx_v_hash_mask_bits == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 23, __pyx_L3_error) __pyx_v_hash_window_size = __Pyx_PyInt_As_int(values[4]); if (unlikely((__pyx_v_hash_window_size == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 23, __pyx_L3_error) } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 5, 5, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 23, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.chunker.Chunker.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return -1; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_7chunker_7Chunker___cinit__(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self), __pyx_v_seed, __pyx_v_chunk_min_exp, __pyx_v_chunk_max_exp, __pyx_v_hash_mask_bits, __pyx_v_hash_window_size); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_7chunker_7Chunker___cinit__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self, int __pyx_v_seed, int __pyx_v_chunk_min_exp, int __pyx_v_chunk_max_exp, int __pyx_v_hash_mask_bits, int __pyx_v_hash_window_size) { long __pyx_v_min_size; long __pyx_v_max_size; long __pyx_v_hash_mask; int __pyx_r; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; uint32_t __pyx_t_3; __Pyx_RefNannySetupContext("__cinit__", 0); /* "borg/chunker.pyx":24 * * def __cinit__(self, int seed, int chunk_min_exp, int chunk_max_exp, int hash_mask_bits, int hash_window_size): * min_size = 1 << chunk_min_exp # <<<<<<<<<<<<<< * max_size = 1 << chunk_max_exp * hash_mask = (1 << hash_mask_bits) - 1 */ __pyx_v_min_size = (1 << __pyx_v_chunk_min_exp); /* "borg/chunker.pyx":25 * def __cinit__(self, int seed, int chunk_min_exp, int chunk_max_exp, int hash_mask_bits, int hash_window_size): * min_size = 1 << chunk_min_exp * max_size = 1 << chunk_max_exp # <<<<<<<<<<<<<< * hash_mask = (1 << hash_mask_bits) - 1 * self.chunker = chunker_init(hash_window_size, hash_mask, min_size, max_size, seed & 0xffffffff) */ __pyx_v_max_size = (1 << __pyx_v_chunk_max_exp); /* "borg/chunker.pyx":26 * min_size = 1 << chunk_min_exp * max_size = 1 << chunk_max_exp * hash_mask = (1 << hash_mask_bits) - 1 # <<<<<<<<<<<<<< * self.chunker = chunker_init(hash_window_size, hash_mask, min_size, max_size, seed & 0xffffffff) * */ __pyx_v_hash_mask = ((1 << __pyx_v_hash_mask_bits) - 1); /* "borg/chunker.pyx":27 * max_size = 1 << chunk_max_exp * hash_mask = (1 << hash_mask_bits) - 1 * self.chunker = chunker_init(hash_window_size, hash_mask, min_size, max_size, seed & 0xffffffff) # <<<<<<<<<<<<<< * * def chunkify(self, fd, fh=-1): */ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_seed); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 27, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyNumber_And(__pyx_t_1, __pyx_int_4294967295); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 27, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(1, 27, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_self->chunker = chunker_init(__pyx_v_hash_window_size, __pyx_v_hash_mask, __pyx_v_min_size, __pyx_v_max_size, __pyx_t_3); /* "borg/chunker.pyx":23 * cdef _Chunker *chunker * * def __cinit__(self, int seed, int chunk_min_exp, int chunk_max_exp, int hash_mask_bits, int hash_window_size): # <<<<<<<<<<<<<< * min_size = 1 << chunk_min_exp * max_size = 1 << chunk_max_exp */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.chunker.Chunker.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/chunker.pyx":29 * self.chunker = chunker_init(hash_window_size, hash_mask, min_size, max_size, seed & 0xffffffff) * * def chunkify(self, fd, fh=-1): # <<<<<<<<<<<<<< * """ * Cut a file into chunks. */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_3chunkify(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_7chunker_7Chunker_2chunkify[] = "\n Cut a file into chunks.\n\n :param fd: Python file object\n :param fh: OS-level file handle (if available),\n defaults to -1 which means not to use OS-level fd.\n "; static PyObject *__pyx_pw_4borg_7chunker_7Chunker_3chunkify(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_fd = 0; PyObject *__pyx_v_fh = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("chunkify (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_fd,&__pyx_n_s_fh,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)__pyx_int_neg_1); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fd)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_fh); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "chunkify") < 0)) __PYX_ERR(1, 29, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_fd = values[0]; __pyx_v_fh = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("chunkify", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 29, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.chunker.Chunker.chunkify", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_7chunker_7Chunker_2chunkify(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self), __pyx_v_fd, __pyx_v_fh); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_7Chunker_2chunkify(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self, PyObject *__pyx_v_fd, PyObject *__pyx_v_fh) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; __Pyx_RefNannySetupContext("chunkify", 0); /* "borg/chunker.pyx":37 * defaults to -1 which means not to use OS-level fd. * """ * chunker_set_fd(self.chunker, fd, fh) # <<<<<<<<<<<<<< * return self * */ __pyx_t_1 = __Pyx_PyInt_As_int(__pyx_v_fh); if (unlikely((__pyx_t_1 == (int)-1) && PyErr_Occurred())) __PYX_ERR(1, 37, __pyx_L1_error) chunker_set_fd(__pyx_v_self->chunker, __pyx_v_fd, __pyx_t_1); /* "borg/chunker.pyx":38 * """ * chunker_set_fd(self.chunker, fd, fh) * return self # <<<<<<<<<<<<<< * * def __dealloc__(self): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(((PyObject *)__pyx_v_self)); __pyx_r = ((PyObject *)__pyx_v_self); goto __pyx_L0; /* "borg/chunker.pyx":29 * self.chunker = chunker_init(hash_window_size, hash_mask, min_size, max_size, seed & 0xffffffff) * * def chunkify(self, fd, fh=-1): # <<<<<<<<<<<<<< * """ * Cut a file into chunks. */ /* function exit code */ __pyx_L1_error:; __Pyx_AddTraceback("borg.chunker.Chunker.chunkify", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/chunker.pyx":40 * return self * * def __dealloc__(self): # <<<<<<<<<<<<<< * if self.chunker: * chunker_free(self.chunker) */ /* Python wrapper */ static void __pyx_pw_4borg_7chunker_7Chunker_5__dealloc__(PyObject *__pyx_v_self); /*proto*/ static void __pyx_pw_4borg_7chunker_7Chunker_5__dealloc__(PyObject *__pyx_v_self) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0); __pyx_pf_4borg_7chunker_7Chunker_4__dealloc__(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); } static void __pyx_pf_4borg_7chunker_7Chunker_4__dealloc__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self) { __Pyx_RefNannyDeclarations int __pyx_t_1; __Pyx_RefNannySetupContext("__dealloc__", 0); /* "borg/chunker.pyx":41 * * def __dealloc__(self): * if self.chunker: # <<<<<<<<<<<<<< * chunker_free(self.chunker) * */ __pyx_t_1 = (__pyx_v_self->chunker != 0); if (__pyx_t_1) { /* "borg/chunker.pyx":42 * def __dealloc__(self): * if self.chunker: * chunker_free(self.chunker) # <<<<<<<<<<<<<< * * def __iter__(self): */ chunker_free(__pyx_v_self->chunker); /* "borg/chunker.pyx":41 * * def __dealloc__(self): * if self.chunker: # <<<<<<<<<<<<<< * chunker_free(self.chunker) * */ } /* "borg/chunker.pyx":40 * return self * * def __dealloc__(self): # <<<<<<<<<<<<<< * if self.chunker: * chunker_free(self.chunker) */ /* function exit code */ __Pyx_RefNannyFinishContext(); } /* "borg/chunker.pyx":44 * chunker_free(self.chunker) * * def __iter__(self): # <<<<<<<<<<<<<< * return self * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_7__iter__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_7__iter__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__iter__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_7chunker_7Chunker_6__iter__(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_7Chunker_6__iter__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__iter__", 0); /* "borg/chunker.pyx":45 * * def __iter__(self): * return self # <<<<<<<<<<<<<< * * def __next__(self): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(((PyObject *)__pyx_v_self)); __pyx_r = ((PyObject *)__pyx_v_self); goto __pyx_L0; /* "borg/chunker.pyx":44 * chunker_free(self.chunker) * * def __iter__(self): # <<<<<<<<<<<<<< * return self * */ /* function exit code */ __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/chunker.pyx":47 * return self * * def __next__(self): # <<<<<<<<<<<<<< * return chunker_process(self.chunker) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_9__next__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_9__next__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__next__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_7chunker_7Chunker_8__next__(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_7Chunker_8__next__(struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__next__", 0); /* "borg/chunker.pyx":48 * * def __next__(self): * return chunker_process(self.chunker) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = chunker_process(__pyx_v_self->chunker); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/chunker.pyx":47 * return self * * def __next__(self): # <<<<<<<<<<<<<< * return chunker_process(self.chunker) * */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.chunker.Chunker.__next__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_11__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_11__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_7chunker_7Chunker_10__reduce_cython__(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_7Chunker_10__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple_, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(0, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.chunker.Chunker.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_13__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_7chunker_7Chunker_13__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_7chunker_7Chunker_12__setstate_cython__(((struct __pyx_obj_4borg_7chunker_Chunker *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_7Chunker_12__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_7chunker_Chunker *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(0, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.chunker.Chunker.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/chunker.pyx":51 * * * def buzhash(data, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * cdef uint32_t sum */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_1buzhash(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_7chunker_1buzhash = {"buzhash", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_7chunker_1buzhash, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_7chunker_1buzhash(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_data = 0; unsigned long __pyx_v_seed; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("buzhash (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_data,&__pyx_n_s_seed,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_seed)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("buzhash", 1, 2, 2, 1); __PYX_ERR(1, 51, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "buzhash") < 0)) __PYX_ERR(1, 51, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_data = values[0]; __pyx_v_seed = __Pyx_PyInt_As_unsigned_long(values[1]); if (unlikely((__pyx_v_seed == (unsigned long)-1) && PyErr_Occurred())) __PYX_ERR(1, 51, __pyx_L3_error) } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("buzhash", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 51, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.chunker.buzhash", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_7chunker_buzhash(__pyx_self, __pyx_v_data, __pyx_v_seed); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_buzhash(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_data, unsigned long __pyx_v_seed) { uint32_t *__pyx_v_table; uint32_t __pyx_v_sum; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; uint32_t __pyx_t_3; unsigned char const *__pyx_t_4; Py_ssize_t __pyx_t_5; __Pyx_RefNannySetupContext("buzhash", 0); /* "borg/chunker.pyx":54 * cdef uint32_t *table * cdef uint32_t sum * table = buzhash_init_table(seed & 0xffffffff) # <<<<<<<<<<<<<< * sum = c_buzhash( data, len(data), table) * free(table) */ __pyx_t_1 = __Pyx_PyInt_From_unsigned_long(__pyx_v_seed); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyNumber_And(__pyx_t_1, __pyx_int_4294967295); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(1, 54, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_table = buzhash_init_table(__pyx_t_3); /* "borg/chunker.pyx":55 * cdef uint32_t sum * table = buzhash_init_table(seed & 0xffffffff) * sum = c_buzhash( data, len(data), table) # <<<<<<<<<<<<<< * free(table) * return sum */ __pyx_t_4 = __Pyx_PyObject_AsUString(__pyx_v_data); if (unlikely((!__pyx_t_4) && PyErr_Occurred())) __PYX_ERR(1, 55, __pyx_L1_error) __pyx_t_5 = PyObject_Length(__pyx_v_data); if (unlikely(__pyx_t_5 == ((Py_ssize_t)-1))) __PYX_ERR(1, 55, __pyx_L1_error) __pyx_v_sum = buzhash(((unsigned char const *)__pyx_t_4), __pyx_t_5, __pyx_v_table); /* "borg/chunker.pyx":56 * table = buzhash_init_table(seed & 0xffffffff) * sum = c_buzhash( data, len(data), table) * free(table) # <<<<<<<<<<<<<< * return sum * */ free(__pyx_v_table); /* "borg/chunker.pyx":57 * sum = c_buzhash( data, len(data), table) * free(table) * return sum # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyInt_From_uint32_t(__pyx_v_sum); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_r = __pyx_t_2; __pyx_t_2 = 0; goto __pyx_L0; /* "borg/chunker.pyx":51 * * * def buzhash(data, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * cdef uint32_t sum */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.chunker.buzhash", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/chunker.pyx":60 * * * def buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * table = buzhash_init_table(seed & 0xffffffff) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_7chunker_3buzhash_update(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_7chunker_3buzhash_update = {"buzhash_update", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_7chunker_3buzhash_update, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_7chunker_3buzhash_update(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { uint32_t __pyx_v_sum; unsigned char __pyx_v_remove; unsigned char __pyx_v_add; size_t __pyx_v_len; unsigned long __pyx_v_seed; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("buzhash_update (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_sum,&__pyx_n_s_remove,&__pyx_n_s_add,&__pyx_n_s_len,&__pyx_n_s_seed,0}; PyObject* values[5] = {0,0,0,0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 5: values[4] = PyTuple_GET_ITEM(__pyx_args, 4); CYTHON_FALLTHROUGH; case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_sum)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_remove)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("buzhash_update", 1, 5, 5, 1); __PYX_ERR(1, 60, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_add)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("buzhash_update", 1, 5, 5, 2); __PYX_ERR(1, 60, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (likely((values[3] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_len)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("buzhash_update", 1, 5, 5, 3); __PYX_ERR(1, 60, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 4: if (likely((values[4] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_seed)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("buzhash_update", 1, 5, 5, 4); __PYX_ERR(1, 60, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "buzhash_update") < 0)) __PYX_ERR(1, 60, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 5) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[3] = PyTuple_GET_ITEM(__pyx_args, 3); values[4] = PyTuple_GET_ITEM(__pyx_args, 4); } __pyx_v_sum = __Pyx_PyInt_As_uint32_t(values[0]); if (unlikely((__pyx_v_sum == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(1, 60, __pyx_L3_error) __pyx_v_remove = __Pyx_PyInt_As_unsigned_char(values[1]); if (unlikely((__pyx_v_remove == (unsigned char)-1) && PyErr_Occurred())) __PYX_ERR(1, 60, __pyx_L3_error) __pyx_v_add = __Pyx_PyInt_As_unsigned_char(values[2]); if (unlikely((__pyx_v_add == (unsigned char)-1) && PyErr_Occurred())) __PYX_ERR(1, 60, __pyx_L3_error) __pyx_v_len = __Pyx_PyInt_As_size_t(values[3]); if (unlikely((__pyx_v_len == (size_t)-1) && PyErr_Occurred())) __PYX_ERR(1, 60, __pyx_L3_error) __pyx_v_seed = __Pyx_PyInt_As_unsigned_long(values[4]); if (unlikely((__pyx_v_seed == (unsigned long)-1) && PyErr_Occurred())) __PYX_ERR(1, 60, __pyx_L3_error) } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("buzhash_update", 1, 5, 5, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 60, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.chunker.buzhash_update", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_7chunker_2buzhash_update(__pyx_self, __pyx_v_sum, __pyx_v_remove, __pyx_v_add, __pyx_v_len, __pyx_v_seed); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_7chunker_2buzhash_update(CYTHON_UNUSED PyObject *__pyx_self, uint32_t __pyx_v_sum, unsigned char __pyx_v_remove, unsigned char __pyx_v_add, size_t __pyx_v_len, unsigned long __pyx_v_seed) { uint32_t *__pyx_v_table; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; uint32_t __pyx_t_3; __Pyx_RefNannySetupContext("buzhash_update", 0); /* "borg/chunker.pyx":62 * def buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, unsigned long seed): * cdef uint32_t *table * table = buzhash_init_table(seed & 0xffffffff) # <<<<<<<<<<<<<< * sum = c_buzhash_update(sum, remove, add, len, table) * free(table) */ __pyx_t_1 = __Pyx_PyInt_From_unsigned_long(__pyx_v_seed); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 62, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyNumber_And(__pyx_t_1, __pyx_int_4294967295); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 62, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(1, 62, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_table = buzhash_init_table(__pyx_t_3); /* "borg/chunker.pyx":63 * cdef uint32_t *table * table = buzhash_init_table(seed & 0xffffffff) * sum = c_buzhash_update(sum, remove, add, len, table) # <<<<<<<<<<<<<< * free(table) * return sum */ __pyx_v_sum = buzhash_update(__pyx_v_sum, __pyx_v_remove, __pyx_v_add, __pyx_v_len, __pyx_v_table); /* "borg/chunker.pyx":64 * table = buzhash_init_table(seed & 0xffffffff) * sum = c_buzhash_update(sum, remove, add, len, table) * free(table) # <<<<<<<<<<<<<< * return sum */ free(__pyx_v_table); /* "borg/chunker.pyx":65 * sum = c_buzhash_update(sum, remove, add, len, table) * free(table) * return sum # <<<<<<<<<<<<<< */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyInt_From_uint32_t(__pyx_v_sum); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 65, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_r = __pyx_t_2; __pyx_t_2 = 0; goto __pyx_L0; /* "borg/chunker.pyx":60 * * * def buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * table = buzhash_init_table(seed & 0xffffffff) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.chunker.buzhash_update", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_tp_new_4borg_7chunker_Chunker(PyTypeObject *t, PyObject *a, PyObject *k) { PyObject *o; if (likely((t->tp_flags & Py_TPFLAGS_IS_ABSTRACT) == 0)) { o = (*t->tp_alloc)(t, 0); } else { o = (PyObject *) PyBaseObject_Type.tp_new(t, __pyx_empty_tuple, 0); } if (unlikely(!o)) return 0; if (unlikely(__pyx_pw_4borg_7chunker_7Chunker_1__cinit__(o, a, k) < 0)) goto bad; return o; bad: Py_DECREF(o); o = 0; return NULL; } static void __pyx_tp_dealloc_4borg_7chunker_Chunker(PyObject *o) { #if CYTHON_USE_TP_FINALIZE if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && (!PyType_IS_GC(Py_TYPE(o)) || !_PyGC_FINALIZED(o))) { if (PyObject_CallFinalizerFromDealloc(o)) return; } #endif { PyObject *etype, *eval, *etb; PyErr_Fetch(&etype, &eval, &etb); ++Py_REFCNT(o); __pyx_pw_4borg_7chunker_7Chunker_5__dealloc__(o); --Py_REFCNT(o); PyErr_Restore(etype, eval, etb); } (*Py_TYPE(o)->tp_free)(o); } static PyObject *__pyx_specialmethod___pyx_pw_4borg_7chunker_7Chunker_9__next__(PyObject *self, CYTHON_UNUSED PyObject *arg) {return __pyx_pw_4borg_7chunker_7Chunker_9__next__(self);} static PyMethodDef __pyx_methods_4borg_7chunker_Chunker[] = { {"chunkify", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_7chunker_7Chunker_3chunkify, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_7chunker_7Chunker_2chunkify}, {"__next__", (PyCFunction)__pyx_specialmethod___pyx_pw_4borg_7chunker_7Chunker_9__next__, METH_NOARGS|METH_COEXIST, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_7chunker_7Chunker_11__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_7chunker_7Chunker_13__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PyTypeObject __pyx_type_4borg_7chunker_Chunker = { PyVarObject_HEAD_INIT(0, 0) "borg.chunker.Chunker", /*tp_name*/ sizeof(struct __pyx_obj_4borg_7chunker_Chunker), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_7chunker_Chunker, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE, /*tp_flags*/ 0, /*tp_doc*/ 0, /*tp_traverse*/ 0, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ __pyx_pw_4borg_7chunker_7Chunker_7__iter__, /*tp_iter*/ __pyx_pw_4borg_7chunker_7Chunker_9__next__, /*tp_iternext*/ __pyx_methods_4borg_7chunker_Chunker, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_7chunker_Chunker, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_chunker(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_chunker}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "chunker", 0, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_n_s_Chunker, __pyx_k_Chunker, sizeof(__pyx_k_Chunker), 0, 0, 1, 1}, {&__pyx_n_s_TypeError, __pyx_k_TypeError, sizeof(__pyx_k_TypeError), 0, 0, 1, 1}, {&__pyx_n_s_add, __pyx_k_add, sizeof(__pyx_k_add), 0, 0, 1, 1}, {&__pyx_n_s_borg_chunker, __pyx_k_borg_chunker, sizeof(__pyx_k_borg_chunker), 0, 0, 1, 1}, {&__pyx_kp_s_borg_chunker_pyx, __pyx_k_borg_chunker_pyx, sizeof(__pyx_k_borg_chunker_pyx), 0, 0, 1, 0}, {&__pyx_n_s_buzhash, __pyx_k_buzhash, sizeof(__pyx_k_buzhash), 0, 0, 1, 1}, {&__pyx_n_s_buzhash_update, __pyx_k_buzhash_update, sizeof(__pyx_k_buzhash_update), 0, 0, 1, 1}, {&__pyx_n_s_chunk_max_exp, __pyx_k_chunk_max_exp, sizeof(__pyx_k_chunk_max_exp), 0, 0, 1, 1}, {&__pyx_n_s_chunk_min_exp, __pyx_k_chunk_min_exp, sizeof(__pyx_k_chunk_min_exp), 0, 0, 1, 1}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_data, __pyx_k_data, sizeof(__pyx_k_data), 0, 0, 1, 1}, {&__pyx_n_s_fd, __pyx_k_fd, sizeof(__pyx_k_fd), 0, 0, 1, 1}, {&__pyx_n_s_fh, __pyx_k_fh, sizeof(__pyx_k_fh), 0, 0, 1, 1}, {&__pyx_n_s_getstate, __pyx_k_getstate, sizeof(__pyx_k_getstate), 0, 0, 1, 1}, {&__pyx_n_s_hash_mask_bits, __pyx_k_hash_mask_bits, sizeof(__pyx_k_hash_mask_bits), 0, 0, 1, 1}, {&__pyx_n_s_hash_window_size, __pyx_k_hash_window_size, sizeof(__pyx_k_hash_window_size), 0, 0, 1, 1}, {&__pyx_n_s_len, __pyx_k_len, sizeof(__pyx_k_len), 0, 0, 1, 1}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_kp_s_no_default___reduce___due_to_non, __pyx_k_no_default___reduce___due_to_non, sizeof(__pyx_k_no_default___reduce___due_to_non), 0, 0, 1, 0}, {&__pyx_n_s_reduce, __pyx_k_reduce, sizeof(__pyx_k_reduce), 0, 0, 1, 1}, {&__pyx_n_s_reduce_cython, __pyx_k_reduce_cython, sizeof(__pyx_k_reduce_cython), 0, 0, 1, 1}, {&__pyx_n_s_reduce_ex, __pyx_k_reduce_ex, sizeof(__pyx_k_reduce_ex), 0, 0, 1, 1}, {&__pyx_n_s_remove, __pyx_k_remove, sizeof(__pyx_k_remove), 0, 0, 1, 1}, {&__pyx_n_s_seed, __pyx_k_seed, sizeof(__pyx_k_seed), 0, 0, 1, 1}, {&__pyx_n_s_setstate, __pyx_k_setstate, sizeof(__pyx_k_setstate), 0, 0, 1, 1}, {&__pyx_n_s_setstate_cython, __pyx_k_setstate_cython, sizeof(__pyx_k_setstate_cython), 0, 0, 1, 1}, {&__pyx_n_s_sum, __pyx_k_sum, sizeof(__pyx_k_sum), 0, 0, 1, 1}, {&__pyx_n_s_table, __pyx_k_table, sizeof(__pyx_k_table), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { __pyx_builtin_TypeError = __Pyx_GetBuiltinName(__pyx_n_s_TypeError); if (!__pyx_builtin_TypeError) __PYX_ERR(0, 2, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple_); __Pyx_GIVEREF(__pyx_tuple_); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__2 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__2)) __PYX_ERR(0, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__2); __Pyx_GIVEREF(__pyx_tuple__2); /* "borg/chunker.pyx":51 * * * def buzhash(data, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * cdef uint32_t sum */ __pyx_tuple__3 = PyTuple_Pack(4, __pyx_n_s_data, __pyx_n_s_seed, __pyx_n_s_table, __pyx_n_s_sum); if (unlikely(!__pyx_tuple__3)) __PYX_ERR(1, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__3); __Pyx_GIVEREF(__pyx_tuple__3); __pyx_codeobj__4 = (PyObject*)__Pyx_PyCode_New(2, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__3, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_chunker_pyx, __pyx_n_s_buzhash, 51, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__4)) __PYX_ERR(1, 51, __pyx_L1_error) /* "borg/chunker.pyx":60 * * * def buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * table = buzhash_init_table(seed & 0xffffffff) */ __pyx_tuple__5 = PyTuple_Pack(6, __pyx_n_s_sum, __pyx_n_s_remove, __pyx_n_s_add, __pyx_n_s_len, __pyx_n_s_seed, __pyx_n_s_table); if (unlikely(!__pyx_tuple__5)) __PYX_ERR(1, 60, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__5); __Pyx_GIVEREF(__pyx_tuple__5); __pyx_codeobj__6 = (PyObject*)__Pyx_PyCode_New(5, 0, 6, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__5, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_chunker_pyx, __pyx_n_s_buzhash_update, 60, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__6)) __PYX_ERR(1, 60, __pyx_L1_error) __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(1, 1, __pyx_L1_error); __pyx_int_4294967295 = PyInt_FromString((char *)"4294967295", 0, 0); if (unlikely(!__pyx_int_4294967295)) __PYX_ERR(1, 1, __pyx_L1_error) __pyx_int_neg_1 = PyInt_FromLong(-1); if (unlikely(!__pyx_int_neg_1)) __PYX_ERR(1, 1, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ if (PyType_Ready(&__pyx_type_4borg_7chunker_Chunker) < 0) __PYX_ERR(1, 20, __pyx_L1_error) __pyx_type_4borg_7chunker_Chunker.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_7chunker_Chunker.tp_dictoffset && __pyx_type_4borg_7chunker_Chunker.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_7chunker_Chunker.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (PyObject_SetAttr(__pyx_m, __pyx_n_s_Chunker, (PyObject *)&__pyx_type_4borg_7chunker_Chunker) < 0) __PYX_ERR(1, 20, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_7chunker_Chunker) < 0) __PYX_ERR(1, 20, __pyx_L1_error) __pyx_ptype_4borg_7chunker_Chunker = &__pyx_type_4borg_7chunker_Chunker; __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC initchunker(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC initchunker(void) #else __Pyx_PyMODINIT_FUNC PyInit_chunker(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_chunker(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_chunker(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'chunker' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_chunker(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(1, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(1, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(1, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("chunker", __pyx_methods, 0, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(1, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(1, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(1, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(1, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(1, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__chunker) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name, __pyx_n_s_main) < 0) __PYX_ERR(1, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(1, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.chunker")) { if (unlikely(PyDict_SetItemString(modules, "borg.chunker", __pyx_m) < 0)) __PYX_ERR(1, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(1, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(1, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); if (unlikely(__Pyx_modinit_type_init_code() != 0)) goto __pyx_L1_error; (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(1, 1, __pyx_L1_error) #endif /* "borg/chunker.pyx":3 * # -*- coding: utf-8 -*- * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * from libc.stdlib cimport free */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(1, 3, __pyx_L1_error) /* "borg/chunker.pyx":51 * * * def buzhash(data, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * cdef uint32_t sum */ __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_4borg_7chunker_1buzhash, NULL, __pyx_n_s_borg_chunker); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_buzhash, __pyx_t_1) < 0) __PYX_ERR(1, 51, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/chunker.pyx":60 * * * def buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, unsigned long seed): # <<<<<<<<<<<<<< * cdef uint32_t *table * table = buzhash_init_table(seed & 0xffffffff) */ __pyx_t_1 = PyCFunction_NewEx(&__pyx_mdef_4borg_7chunker_3buzhash_update, NULL, __pyx_n_s_borg_chunker); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 60, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_buzhash_update, __pyx_t_1) < 0) __PYX_ERR(1, 60, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/chunker.pyx":1 * # -*- coding: utf-8 -*- # <<<<<<<<<<<<<< * * API_VERSION = '1.0_01' */ __pyx_t_1 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_1) < 0) __PYX_ERR(1, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.chunker", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.chunker"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* RaiseException */ #if PY_MAJOR_VERSION < 3 static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, CYTHON_UNUSED PyObject *cause) { __Pyx_PyThreadState_declare Py_XINCREF(type); if (!value || value == Py_None) value = NULL; else Py_INCREF(value); if (!tb || tb == Py_None) tb = NULL; else { Py_INCREF(tb); if (!PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto raise_error; } } if (PyType_Check(type)) { #if CYTHON_COMPILING_IN_PYPY if (!value) { Py_INCREF(Py_None); value = Py_None; } #endif PyErr_NormalizeException(&type, &value, &tb); } else { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto raise_error; } value = type; type = (PyObject*) Py_TYPE(type); Py_INCREF(type); if (!PyType_IsSubtype((PyTypeObject *)type, (PyTypeObject *)PyExc_BaseException)) { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto raise_error; } } __Pyx_PyThreadState_assign __Pyx_ErrRestore(type, value, tb); return; raise_error: Py_XDECREF(value); Py_XDECREF(type); Py_XDECREF(tb); return; } #else static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause) { PyObject* owned_instance = NULL; if (tb == Py_None) { tb = 0; } else if (tb && !PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto bad; } if (value == Py_None) value = 0; if (PyExceptionInstance_Check(type)) { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto bad; } value = type; type = (PyObject*) Py_TYPE(value); } else if (PyExceptionClass_Check(type)) { PyObject *instance_class = NULL; if (value && PyExceptionInstance_Check(value)) { instance_class = (PyObject*) Py_TYPE(value); if (instance_class != type) { int is_subclass = PyObject_IsSubclass(instance_class, type); if (!is_subclass) { instance_class = NULL; } else if (unlikely(is_subclass == -1)) { goto bad; } else { type = instance_class; } } } if (!instance_class) { PyObject *args; if (!value) args = PyTuple_New(0); else if (PyTuple_Check(value)) { Py_INCREF(value); args = value; } else args = PyTuple_Pack(1, value); if (!args) goto bad; owned_instance = PyObject_Call(type, args, NULL); Py_DECREF(args); if (!owned_instance) goto bad; value = owned_instance; if (!PyExceptionInstance_Check(value)) { PyErr_Format(PyExc_TypeError, "calling %R should have returned an instance of " "BaseException, not %R", type, Py_TYPE(value)); goto bad; } } } else { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto bad; } if (cause) { PyObject *fixed_cause; if (cause == Py_None) { fixed_cause = NULL; } else if (PyExceptionClass_Check(cause)) { fixed_cause = PyObject_CallObject(cause, NULL); if (fixed_cause == NULL) goto bad; } else if (PyExceptionInstance_Check(cause)) { fixed_cause = cause; Py_INCREF(fixed_cause); } else { PyErr_SetString(PyExc_TypeError, "exception causes must derive from " "BaseException"); goto bad; } PyException_SetCause(value, fixed_cause); } PyErr_SetObject(type, value); if (tb) { #if CYTHON_COMPILING_IN_PYPY PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_Fetch(&tmp_type, &tmp_value, &tmp_tb); Py_INCREF(tb); PyErr_Restore(tmp_type, tmp_value, tb); Py_XDECREF(tmp_tb); #else PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* tmp_tb = tstate->curexc_traceback; if (tb != tmp_tb) { Py_INCREF(tb); tstate->curexc_traceback = tb; Py_XDECREF(tmp_tb); } #endif } bad: Py_XDECREF(owned_instance); return; } #endif /* PyObject_GenericGetAttrNoDict */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject *__Pyx_RaiseGenericGetAttributeError(PyTypeObject *tp, PyObject *attr_name) { PyErr_Format(PyExc_AttributeError, #if PY_MAJOR_VERSION >= 3 "'%.50s' object has no attribute '%U'", tp->tp_name, attr_name); #else "'%.50s' object has no attribute '%.400s'", tp->tp_name, PyString_AS_STRING(attr_name)); #endif return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name) { PyObject *descr; PyTypeObject *tp = Py_TYPE(obj); if (unlikely(!PyString_Check(attr_name))) { return PyObject_GenericGetAttr(obj, attr_name); } assert(!tp->tp_dictoffset); descr = _PyType_Lookup(tp, attr_name); if (unlikely(!descr)) { return __Pyx_RaiseGenericGetAttributeError(tp, attr_name); } Py_INCREF(descr); #if PY_MAJOR_VERSION < 3 if (likely(PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_HAVE_CLASS))) #endif { descrgetfunc f = Py_TYPE(descr)->tp_descr_get; if (unlikely(f)) { PyObject *res = f(descr, obj, (PyObject *)tp); Py_DECREF(descr); return res; } } return descr; } #endif /* PyObject_GenericGetAttr */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name) { if (unlikely(Py_TYPE(obj)->tp_dictoffset)) { return PyObject_GenericGetAttr(obj, attr_name); } return __Pyx_PyObject_GenericGetAttrNoDict(obj, attr_name); } #endif /* SetupReduce */ static int __Pyx_setup_reduce_is_named(PyObject* meth, PyObject* name) { int ret; PyObject *name_attr; name_attr = __Pyx_PyObject_GetAttrStr(meth, __pyx_n_s_name); if (likely(name_attr)) { ret = PyObject_RichCompareBool(name_attr, name, Py_EQ); } else { ret = -1; } if (unlikely(ret < 0)) { PyErr_Clear(); ret = 0; } Py_XDECREF(name_attr); return ret; } static int __Pyx_setup_reduce(PyObject* type_obj) { int ret = 0; PyObject *object_reduce = NULL; PyObject *object_reduce_ex = NULL; PyObject *reduce = NULL; PyObject *reduce_ex = NULL; PyObject *reduce_cython = NULL; PyObject *setstate = NULL; PyObject *setstate_cython = NULL; #if CYTHON_USE_PYTYPE_LOOKUP if (_PyType_Lookup((PyTypeObject*)type_obj, __pyx_n_s_getstate)) goto GOOD; #else if (PyObject_HasAttr(type_obj, __pyx_n_s_getstate)) goto GOOD; #endif #if CYTHON_USE_PYTYPE_LOOKUP object_reduce_ex = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #else object_reduce_ex = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #endif reduce_ex = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_ex); if (unlikely(!reduce_ex)) goto BAD; if (reduce_ex == object_reduce_ex) { #if CYTHON_USE_PYTYPE_LOOKUP object_reduce = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #else object_reduce = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #endif reduce = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce); if (unlikely(!reduce)) goto BAD; if (reduce == object_reduce || __Pyx_setup_reduce_is_named(reduce, __pyx_n_s_reduce_cython)) { reduce_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_cython); if (unlikely(!reduce_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce, reduce_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce_cython); if (unlikely(ret < 0)) goto BAD; setstate = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate); if (!setstate) PyErr_Clear(); if (!setstate || __Pyx_setup_reduce_is_named(setstate, __pyx_n_s_setstate_cython)) { setstate_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate_cython); if (unlikely(!setstate_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate, setstate_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate_cython); if (unlikely(ret < 0)) goto BAD; } PyType_Modified((PyTypeObject*)type_obj); } } goto GOOD; BAD: if (!PyErr_Occurred()) PyErr_Format(PyExc_RuntimeError, "Unable to initialize pickling for %s", ((PyTypeObject*)type_obj)->tp_name); ret = -1; GOOD: #if !CYTHON_USE_PYTYPE_LOOKUP Py_XDECREF(object_reduce); Py_XDECREF(object_reduce_ex); #endif Py_XDECREF(reduce); Py_XDECREF(reduce_ex); Py_XDECREF(reduce_cython); Py_XDECREF(setstate); Py_XDECREF(setstate_cython); return ret; } /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(int) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(int) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(int) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(int), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_unsigned_long(unsigned long value) { const unsigned long neg_one = (unsigned long) ((unsigned long) 0 - (unsigned long) 1), const_zero = (unsigned long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(unsigned long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(unsigned long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(unsigned long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(unsigned long), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_uint32_t(uint32_t value) { const uint32_t neg_one = (uint32_t) ((uint32_t) 0 - (uint32_t) 1), const_zero = (uint32_t) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(uint32_t) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(uint32_t) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(uint32_t) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(uint32_t), little, !is_unsigned); } } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* CIntFromPy */ static CYTHON_INLINE unsigned long __Pyx_PyInt_As_unsigned_long(PyObject *x) { const unsigned long neg_one = (unsigned long) ((unsigned long) 0 - (unsigned long) 1), const_zero = (unsigned long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(unsigned long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(unsigned long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (unsigned long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (unsigned long) 0; case 1: __PYX_VERIFY_RETURN_INT(unsigned long, digit, digits[0]) case 2: if (8 * sizeof(unsigned long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) >= 2 * PyLong_SHIFT) { return (unsigned long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); } } break; case 3: if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) >= 3 * PyLong_SHIFT) { return (unsigned long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); } } break; case 4: if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) >= 4 * PyLong_SHIFT) { return (unsigned long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (unsigned long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(unsigned long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (unsigned long) 0; case -1: __PYX_VERIFY_RETURN_INT(unsigned long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(unsigned long, digit, +digits[0]) case -2: if (8 * sizeof(unsigned long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) - 1 > 2 * PyLong_SHIFT) { return (unsigned long) (((unsigned long)-1)*(((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))); } } break; case 2: if (8 * sizeof(unsigned long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) - 1 > 2 * PyLong_SHIFT) { return (unsigned long) ((((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))); } } break; case -3: if (8 * sizeof(unsigned long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) - 1 > 3 * PyLong_SHIFT) { return (unsigned long) (((unsigned long)-1)*(((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))); } } break; case 3: if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) - 1 > 3 * PyLong_SHIFT) { return (unsigned long) ((((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))); } } break; case -4: if (8 * sizeof(unsigned long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) - 1 > 4 * PyLong_SHIFT) { return (unsigned long) (((unsigned long)-1)*(((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))); } } break; case 4: if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned long) - 1 > 4 * PyLong_SHIFT) { return (unsigned long) ((((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))); } } break; } #endif if (sizeof(unsigned long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else unsigned long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (unsigned long) -1; } } else { unsigned long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (unsigned long) -1; val = __Pyx_PyInt_As_unsigned_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to unsigned long"); return (unsigned long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to unsigned long"); return (unsigned long) -1; } /* CIntFromPy */ static CYTHON_INLINE uint32_t __Pyx_PyInt_As_uint32_t(PyObject *x) { const uint32_t neg_one = (uint32_t) ((uint32_t) 0 - (uint32_t) 1), const_zero = (uint32_t) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(uint32_t) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(uint32_t, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (uint32_t) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (uint32_t) 0; case 1: __PYX_VERIFY_RETURN_INT(uint32_t, digit, digits[0]) case 2: if (8 * sizeof(uint32_t) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) >= 2 * PyLong_SHIFT) { return (uint32_t) (((((uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0])); } } break; case 3: if (8 * sizeof(uint32_t) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) >= 3 * PyLong_SHIFT) { return (uint32_t) (((((((uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0])); } } break; case 4: if (8 * sizeof(uint32_t) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) >= 4 * PyLong_SHIFT) { return (uint32_t) (((((((((uint32_t)digits[3]) << PyLong_SHIFT) | (uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (uint32_t) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(uint32_t) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (uint32_t) 0; case -1: __PYX_VERIFY_RETURN_INT(uint32_t, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(uint32_t, digit, +digits[0]) case -2: if (8 * sizeof(uint32_t) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 2 * PyLong_SHIFT) { return (uint32_t) (((uint32_t)-1)*(((((uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case 2: if (8 * sizeof(uint32_t) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 2 * PyLong_SHIFT) { return (uint32_t) ((((((uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case -3: if (8 * sizeof(uint32_t) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 3 * PyLong_SHIFT) { return (uint32_t) (((uint32_t)-1)*(((((((uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case 3: if (8 * sizeof(uint32_t) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 3 * PyLong_SHIFT) { return (uint32_t) ((((((((uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case -4: if (8 * sizeof(uint32_t) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 4 * PyLong_SHIFT) { return (uint32_t) (((uint32_t)-1)*(((((((((uint32_t)digits[3]) << PyLong_SHIFT) | (uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case 4: if (8 * sizeof(uint32_t) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 4 * PyLong_SHIFT) { return (uint32_t) ((((((((((uint32_t)digits[3]) << PyLong_SHIFT) | (uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; } #endif if (sizeof(uint32_t) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else uint32_t val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (uint32_t) -1; } } else { uint32_t val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (uint32_t) -1; val = __Pyx_PyInt_As_uint32_t(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to uint32_t"); return (uint32_t) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to uint32_t"); return (uint32_t) -1; } /* CIntFromPy */ static CYTHON_INLINE unsigned char __Pyx_PyInt_As_unsigned_char(PyObject *x) { const unsigned char neg_one = (unsigned char) ((unsigned char) 0 - (unsigned char) 1), const_zero = (unsigned char) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(unsigned char) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(unsigned char, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (unsigned char) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (unsigned char) 0; case 1: __PYX_VERIFY_RETURN_INT(unsigned char, digit, digits[0]) case 2: if (8 * sizeof(unsigned char) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) >= 2 * PyLong_SHIFT) { return (unsigned char) (((((unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0])); } } break; case 3: if (8 * sizeof(unsigned char) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) >= 3 * PyLong_SHIFT) { return (unsigned char) (((((((unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0])); } } break; case 4: if (8 * sizeof(unsigned char) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) >= 4 * PyLong_SHIFT) { return (unsigned char) (((((((((unsigned char)digits[3]) << PyLong_SHIFT) | (unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (unsigned char) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(unsigned char) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned char) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (unsigned char) 0; case -1: __PYX_VERIFY_RETURN_INT(unsigned char, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(unsigned char, digit, +digits[0]) case -2: if (8 * sizeof(unsigned char) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 2 * PyLong_SHIFT) { return (unsigned char) (((unsigned char)-1)*(((((unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case 2: if (8 * sizeof(unsigned char) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 2 * PyLong_SHIFT) { return (unsigned char) ((((((unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case -3: if (8 * sizeof(unsigned char) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 3 * PyLong_SHIFT) { return (unsigned char) (((unsigned char)-1)*(((((((unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case 3: if (8 * sizeof(unsigned char) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 3 * PyLong_SHIFT) { return (unsigned char) ((((((((unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case -4: if (8 * sizeof(unsigned char) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 4 * PyLong_SHIFT) { return (unsigned char) (((unsigned char)-1)*(((((((((unsigned char)digits[3]) << PyLong_SHIFT) | (unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case 4: if (8 * sizeof(unsigned char) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 4 * PyLong_SHIFT) { return (unsigned char) ((((((((((unsigned char)digits[3]) << PyLong_SHIFT) | (unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; } #endif if (sizeof(unsigned char) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned char) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else unsigned char val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (unsigned char) -1; } } else { unsigned char val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (unsigned char) -1; val = __Pyx_PyInt_As_unsigned_char(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to unsigned char"); return (unsigned char) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to unsigned char"); return (unsigned char) -1; } /* CIntFromPy */ static CYTHON_INLINE size_t __Pyx_PyInt_As_size_t(PyObject *x) { const size_t neg_one = (size_t) ((size_t) 0 - (size_t) 1), const_zero = (size_t) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(size_t) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(size_t, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (size_t) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (size_t) 0; case 1: __PYX_VERIFY_RETURN_INT(size_t, digit, digits[0]) case 2: if (8 * sizeof(size_t) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) >= 2 * PyLong_SHIFT) { return (size_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } } break; case 3: if (8 * sizeof(size_t) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) >= 3 * PyLong_SHIFT) { return (size_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } } break; case 4: if (8 * sizeof(size_t) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) >= 4 * PyLong_SHIFT) { return (size_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (size_t) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(size_t) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(size_t, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(size_t) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(size_t, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (size_t) 0; case -1: __PYX_VERIFY_RETURN_INT(size_t, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(size_t, digit, +digits[0]) case -2: if (8 * sizeof(size_t) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) - 1 > 2 * PyLong_SHIFT) { return (size_t) (((size_t)-1)*(((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0]))); } } break; case 2: if (8 * sizeof(size_t) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) - 1 > 2 * PyLong_SHIFT) { return (size_t) ((((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0]))); } } break; case -3: if (8 * sizeof(size_t) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) - 1 > 3 * PyLong_SHIFT) { return (size_t) (((size_t)-1)*(((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0]))); } } break; case 3: if (8 * sizeof(size_t) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) - 1 > 3 * PyLong_SHIFT) { return (size_t) ((((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0]))); } } break; case -4: if (8 * sizeof(size_t) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) - 1 > 4 * PyLong_SHIFT) { return (size_t) (((size_t)-1)*(((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0]))); } } break; case 4: if (8 * sizeof(size_t) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(size_t, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(size_t) - 1 > 4 * PyLong_SHIFT) { return (size_t) ((((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0]))); } } break; } #endif if (sizeof(size_t) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(size_t, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(size_t) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(size_t, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else size_t val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (size_t) -1; } } else { size_t val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (size_t) -1; val = __Pyx_PyInt_As_size_t(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to size_t"); return (size_t) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to size_t"); return (size_t) -1; } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; ip) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/chunker.pyx0000644000175000017500000000411513431513133017021 0ustar useruser00000000000000# -*- coding: utf-8 -*- API_VERSION = '1.0_01' from libc.stdlib cimport free cdef extern from "_chunker.c": ctypedef int uint32_t ctypedef struct _Chunker "Chunker": pass _Chunker *chunker_init(int window_size, int chunk_mask, int min_size, int max_size, uint32_t seed) void chunker_set_fd(_Chunker *chunker, object f, int fd) void chunker_free(_Chunker *chunker) object chunker_process(_Chunker *chunker) uint32_t *buzhash_init_table(uint32_t seed) uint32_t c_buzhash "buzhash"(unsigned char *data, size_t len, uint32_t *h) uint32_t c_buzhash_update "buzhash_update"(uint32_t sum, unsigned char remove, unsigned char add, size_t len, uint32_t *h) cdef class Chunker: cdef _Chunker *chunker def __cinit__(self, int seed, int chunk_min_exp, int chunk_max_exp, int hash_mask_bits, int hash_window_size): min_size = 1 << chunk_min_exp max_size = 1 << chunk_max_exp hash_mask = (1 << hash_mask_bits) - 1 self.chunker = chunker_init(hash_window_size, hash_mask, min_size, max_size, seed & 0xffffffff) def chunkify(self, fd, fh=-1): """ Cut a file into chunks. :param fd: Python file object :param fh: OS-level file handle (if available), defaults to -1 which means not to use OS-level fd. """ chunker_set_fd(self.chunker, fd, fh) return self def __dealloc__(self): if self.chunker: chunker_free(self.chunker) def __iter__(self): return self def __next__(self): return chunker_process(self.chunker) def buzhash(data, unsigned long seed): cdef uint32_t *table cdef uint32_t sum table = buzhash_init_table(seed & 0xffffffff) sum = c_buzhash( data, len(data), table) free(table) return sum def buzhash_update(uint32_t sum, unsigned char remove, unsigned char add, size_t len, unsigned long seed): cdef uint32_t *table table = buzhash_init_table(seed & 0xffffffff) sum = c_buzhash_update(sum, remove, add, len, table) free(table) return sum borgbackup-1.0.13/borg/compress.c0000644000175000017500000157305513431514213016636 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__compress #define __PYX_HAVE_API__borg__compress /* Early includes */ #include "lz4.h" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "borg/compress.pyx", "stringsource", }; /* NoFastGil.proto */ #define __Pyx_PyGILState_Ensure PyGILState_Ensure #define __Pyx_PyGILState_Release PyGILState_Release #define __Pyx_FastGIL_Remember() #define __Pyx_FastGIL_Forget() #define __Pyx_FastGilFuncInit() /* ForceInitThreads.proto */ #ifndef __PYX_FORCE_INIT_THREADS #define __PYX_FORCE_INIT_THREADS 0 #endif /*--- Type declarations ---*/ struct __pyx_obj_4borg_8compress_CompressorBase; /* "borg/compress.pyx":20 * * * cdef class CompressorBase: # <<<<<<<<<<<<<< * """ * base class for all (de)compression classes, */ struct __pyx_obj_4borg_8compress_CompressorBase { PyObject_HEAD }; /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* PyCFunctionFastCall.proto */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs); #else #define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL) #endif /* PyFunctionFastCall.proto */ #if CYTHON_FAST_PYCALL #define __Pyx_PyFunction_FastCall(func, args, nargs)\ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL) #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #else #define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs) #endif #define __Pyx_BUILD_ASSERT_EXPR(cond)\ (sizeof(char [1 - 2*!(cond)]) - 1) #ifndef Py_MEMBER_SIZE #define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) #endif static size_t __pyx_pyframe_localsplus_offset = 0; #include "frameobject.h" #define __Pxy_PyFrame_Initialize_Offsets()\ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\ (void)(__pyx_pyframe_localsplus_offset = PyFrame_Type.tp_basicsize - Py_MEMBER_SIZE(PyFrameObject, f_localsplus))) #define __Pyx_PyFrame_GetLocalsplus(frame)\ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset)) #endif /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* PyObjectCall2Args.proto */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2); /* PyObjectCallMethO.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg); #endif /* PyObjectCallOneArg.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg); /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* KeywordStringCheck.proto */ static int __Pyx_CheckKeywordStrings(PyObject *kwdict, const char* function_name, int kw_allowed); /* SliceObject.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice( PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** py_start, PyObject** py_stop, PyObject** py_slice, int has_cstart, int has_cstop, int wraparound); /* PyErrExceptionMatches.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_ExceptionMatches(err) __Pyx_PyErr_ExceptionMatchesInState(__pyx_tstate, err) static CYTHON_INLINE int __Pyx_PyErr_ExceptionMatchesInState(PyThreadState* tstate, PyObject* err); #else #define __Pyx_PyErr_ExceptionMatches(err) PyErr_ExceptionMatches(err) #endif /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* GetAttr.proto */ static CYTHON_INLINE PyObject *__Pyx_GetAttr(PyObject *, PyObject *); /* GetAttr3.proto */ static CYTHON_INLINE PyObject *__Pyx_GetAttr3(PyObject *, PyObject *, PyObject *); /* GetModuleGlobalName.proto */ #if CYTHON_USE_DICT_VERSIONS #define __Pyx_GetModuleGlobalName(var, name) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } #define __Pyx_GetModuleGlobalNameUncached(var, name) {\ PY_UINT64_T __pyx_dict_version;\ PyObject *__pyx_dict_cached_value;\ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value); #else #define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name) #define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name) static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name); #endif /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* RaiseException.proto */ static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause); /* PyIntFromDouble.proto */ #if PY_MAJOR_VERSION < 3 static CYTHON_INLINE PyObject* __Pyx_PyInt_FromDouble(double value); #else #define __Pyx_PyInt_FromDouble(value) PyLong_FromDouble(value) #endif /* PyObjectSetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS #define __Pyx_PyObject_DelAttrStr(o,n) __Pyx_PyObject_SetAttrStr(o, n, NULL) static CYTHON_INLINE int __Pyx_PyObject_SetAttrStr(PyObject* obj, PyObject* attr_name, PyObject* value); #else #define __Pyx_PyObject_DelAttrStr(o,n) PyObject_DelAttr(o,n) #define __Pyx_PyObject_SetAttrStr(o,n,v) PyObject_SetAttr(o,n,v) #endif /* GetTopmostException.proto */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate); #endif /* SaveResetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSave(type, value, tb) __Pyx__ExceptionSave(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #define __Pyx_ExceptionReset(type, value, tb) __Pyx__ExceptionReset(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); #else #define __Pyx_ExceptionSave(type, value, tb) PyErr_GetExcInfo(type, value, tb) #define __Pyx_ExceptionReset(type, value, tb) PyErr_SetExcInfo(type, value, tb) #endif /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* GetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_GetException(type, value, tb) __Pyx__GetException(__pyx_tstate, type, value, tb) static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb); #endif /* RaiseTooManyValuesToUnpack.proto */ static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected); /* RaiseNeedMoreValuesToUnpack.proto */ static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index); /* IterFinish.proto */ static CYTHON_INLINE int __Pyx_IterFinish(void); /* UnpackItemEndCheck.proto */ static int __Pyx_IternextUnpackEndCheck(PyObject *retval, Py_ssize_t expected); /* PyIntCompare.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_EqObjC(PyObject *op1, PyObject *op2, long intval, long inplace); /* PyIntBinop.proto */ #if !CYTHON_COMPILING_IN_PYPY static PyObject* __Pyx_PyInt_AndObjC(PyObject *op1, PyObject *op2, long intval, int inplace); #else #define __Pyx_PyInt_AndObjC(op1, op2, intval, inplace)\ (inplace ? PyNumber_InPlaceAnd(op1, op2) : PyNumber_And(op1, op2)) #endif /* PyIntBinop.proto */ #if !CYTHON_COMPILING_IN_PYPY static PyObject* __Pyx_PyInt_RemainderObjC(PyObject *op1, PyObject *op2, long intval, int inplace); #else #define __Pyx_PyInt_RemainderObjC(op1, op2, intval, inplace)\ (inplace ? PyNumber_InPlaceRemainder(op1, op2) : PyNumber_Remainder(op1, op2)) #endif /* GetItemInt.proto */ #define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\ __Pyx_GetItemInt_Generic(o, to_py_func(i)))) #define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); #define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j); static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, int wraparound, int boundscheck); /* ObjectGetItem.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key); #else #define __Pyx_PyObject_GetItem(obj, key) PyObject_GetItem(obj, key) #endif /* Import.proto */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level); /* ImportFrom.proto */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name); /* HasAttr.proto */ static CYTHON_INLINE int __Pyx_HasAttr(PyObject *, PyObject *); /* PyObject_GenericGetAttrNoDict.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttrNoDict PyObject_GenericGetAttr #endif /* PyObject_GenericGetAttr.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttr PyObject_GenericGetAttr #endif /* SetupReduce.proto */ static int __Pyx_setup_reduce(PyObject* type_obj); /* ClassMethod.proto */ #include "descrobject.h" static PyObject* __Pyx_Method_ClassMethod(PyObject *method); /* GetNameInClass.proto */ #define __Pyx_GetNameInClass(var, nmspace, name) (var) = __Pyx__GetNameInClass(nmspace, name) static PyObject *__Pyx__GetNameInClass(PyObject *nmspace, PyObject *name); /* CalculateMetaclass.proto */ static PyObject *__Pyx_CalculateMetaclass(PyTypeObject *metaclass, PyObject *bases); /* SetNameInClass.proto */ #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 #define __Pyx_SetNameInClass(ns, name, value)\ (likely(PyDict_CheckExact(ns)) ? _PyDict_SetItem_KnownHash(ns, name, value, ((PyASCIIObject *) name)->hash) : PyObject_SetItem(ns, name, value)) #elif CYTHON_COMPILING_IN_CPYTHON #define __Pyx_SetNameInClass(ns, name, value)\ (likely(PyDict_CheckExact(ns)) ? PyDict_SetItem(ns, name, value) : PyObject_SetItem(ns, name, value)) #else #define __Pyx_SetNameInClass(ns, name, value) PyObject_SetItem(ns, name, value) #endif /* FetchCommonType.proto */ static PyTypeObject* __Pyx_FetchCommonType(PyTypeObject* type); /* CythonFunction.proto */ #define __Pyx_CyFunction_USED 1 #define __Pyx_CYFUNCTION_STATICMETHOD 0x01 #define __Pyx_CYFUNCTION_CLASSMETHOD 0x02 #define __Pyx_CYFUNCTION_CCLASS 0x04 #define __Pyx_CyFunction_GetClosure(f)\ (((__pyx_CyFunctionObject *) (f))->func_closure) #define __Pyx_CyFunction_GetClassObj(f)\ (((__pyx_CyFunctionObject *) (f))->func_classobj) #define __Pyx_CyFunction_Defaults(type, f)\ ((type *)(((__pyx_CyFunctionObject *) (f))->defaults)) #define __Pyx_CyFunction_SetDefaultsGetter(f, g)\ ((__pyx_CyFunctionObject *) (f))->defaults_getter = (g) typedef struct { PyCFunctionObject func; #if PY_VERSION_HEX < 0x030500A0 PyObject *func_weakreflist; #endif PyObject *func_dict; PyObject *func_name; PyObject *func_qualname; PyObject *func_doc; PyObject *func_globals; PyObject *func_code; PyObject *func_closure; PyObject *func_classobj; void *defaults; int defaults_pyobjects; int flags; PyObject *defaults_tuple; PyObject *defaults_kwdict; PyObject *(*defaults_getter)(PyObject *); PyObject *func_annotations; } __pyx_CyFunctionObject; static PyTypeObject *__pyx_CyFunctionType = 0; #define __Pyx_CyFunction_Check(obj) (__Pyx_TypeCheck(obj, __pyx_CyFunctionType)) #define __Pyx_CyFunction_NewEx(ml, flags, qualname, self, module, globals, code)\ __Pyx_CyFunction_New(__pyx_CyFunctionType, ml, flags, qualname, self, module, globals, code) static PyObject *__Pyx_CyFunction_New(PyTypeObject *, PyMethodDef *ml, int flags, PyObject* qualname, PyObject *self, PyObject *module, PyObject *globals, PyObject* code); static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *m, size_t size, int pyobjects); static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *m, PyObject *tuple); static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *m, PyObject *dict); static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *m, PyObject *dict); static int __pyx_CyFunction_init(void); /* Py3ClassCreate.proto */ static PyObject *__Pyx_Py3MetaclassPrepare(PyObject *metaclass, PyObject *bases, PyObject *name, PyObject *qualname, PyObject *mkw, PyObject *modname, PyObject *doc); static PyObject *__Pyx_Py3ClassCreate(PyObject *metaclass, PyObject *name, PyObject *bases, PyObject *dict, PyObject *mkw, int calculate_metaclass, int allow_py2_metaclass); /* CyFunctionClassCell.proto */ static int __Pyx_CyFunction_InitClassCell(PyObject *cyfunctions, PyObject *classobj); /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); /* Module declarations from 'borg.compress' */ static PyTypeObject *__pyx_ptype_4borg_8compress_CompressorBase = 0; static PyObject *__pyx_f_4borg_8compress___pyx_unpickle_CompressorBase__set_state(struct __pyx_obj_4borg_8compress_CompressorBase *, PyObject *); /*proto*/ #define __Pyx_MODULE_NAME "borg.compress" extern int __pyx_module_is_main_borg__compress; int __pyx_module_is_main_borg__compress = 0; /* Implementation of 'borg.compress' */ static PyObject *__pyx_builtin_ImportError; static PyObject *__pyx_builtin_super; static PyObject *__pyx_builtin_ValueError; static const char __pyx_k_e[] = "e"; static const char __pyx_k_ID[] = "ID"; static const char __pyx_k__5[] = "\377\377"; static const char __pyx_k__6[] = "\000\000"; static const char __pyx_k_LZ4[] = "LZ4"; static const char __pyx_k__11[] = "\001\000"; static const char __pyx_k__18[] = "\002\000"; static const char __pyx_k__26[] = "\010\000"; static const char __pyx_k_buf[] = "buf"; static const char __pyx_k_cls[] = "cls"; static const char __pyx_k_cmf[] = "cmf"; static const char __pyx_k_doc[] = "__doc__"; static const char __pyx_k_flg[] = "flg"; static const char __pyx_k_get[] = "get"; static const char __pyx_k_hdr[] = "hdr"; static const char __pyx_k_lz4[] = "lz4"; static const char __pyx_k_new[] = "__new__"; static const char __pyx_k_LZMA[] = "LZMA"; static const char __pyx_k_ZLIB[] = "ZLIB"; static const char __pyx_k_data[] = "data"; static const char __pyx_k_dest[] = "dest"; static const char __pyx_k_dict[] = "__dict__"; static const char __pyx_k_init[] = "__init__"; static const char __pyx_k_lzma[] = "lzma"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_name[] = "name"; static const char __pyx_k_none[] = "none"; static const char __pyx_k_null[] = "null"; static const char __pyx_k_self[] = "self"; static const char __pyx_k_size[] = "size"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_zlib[] = "zlib"; static const char __pyx_k_CNONE[] = "CNONE"; static const char __pyx_k_check[] = "check"; static const char __pyx_k_error[] = "error"; static const char __pyx_k_idata[] = "idata"; static const char __pyx_k_isize[] = "isize"; static const char __pyx_k_level[] = "level"; static const char __pyx_k_osize[] = "osize"; static const char __pyx_k_rsize[] = "rsize"; static const char __pyx_k_super[] = "super"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_Buffer[] = "Buffer"; static const char __pyx_k_buffer[] = "buffer"; static const char __pyx_k_detect[] = "detect"; static const char __pyx_k_import[] = "__import__"; static const char __pyx_k_kwargs[] = "kwargs"; static const char __pyx_k_module[] = "__module__"; static const char __pyx_k_name_2[] = "__name__"; static const char __pyx_k_params[] = "params"; static const char __pyx_k_pickle[] = "pickle"; static const char __pyx_k_preset[] = "preset"; static const char __pyx_k_reduce[] = "__reduce__"; static const char __pyx_k_source[] = "source"; static const char __pyx_k_update[] = "update"; static const char __pyx_k_helpers[] = "helpers"; static const char __pyx_k_prepare[] = "__prepare__"; static const char __pyx_k_check_ok[] = "check_ok"; static const char __pyx_k_compress[] = "compress"; static const char __pyx_k_getstate[] = "__getstate__"; static const char __pyx_k_pyx_type[] = "__pyx_type"; static const char __pyx_k_qualname[] = "__qualname__"; static const char __pyx_k_setstate[] = "__setstate__"; static const char __pyx_k_LZMAError[] = "LZMAError"; static const char __pyx_k_baseclass[] = "baseclass"; static const char __pyx_k_metaclass[] = "__metaclass__"; static const char __pyx_k_pyx_state[] = "__pyx_state"; static const char __pyx_k_reduce_ex[] = "__reduce_ex__"; static const char __pyx_k_CHECK_NONE[] = "CHECK_NONE"; static const char __pyx_k_Compressor[] = "Compressor"; static const char __pyx_k_LZ4___init[] = "LZ4.__init__"; static const char __pyx_k_ValueError[] = "ValueError"; static const char __pyx_k_compressor[] = "compressor"; static const char __pyx_k_decompress[] = "decompress"; static const char __pyx_k_is_deflate[] = "is_deflate"; static const char __pyx_k_pyx_result[] = "__pyx_result"; static const char __pyx_k_startswith[] = "startswith"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_ImportError[] = "ImportError"; static const char __pyx_k_LZMA___init[] = "LZMA.__init__"; static const char __pyx_k_PickleError[] = "PickleError"; static const char __pyx_k_ZLIB___init[] = "ZLIB.__init__"; static const char __pyx_k_ZLIB_detect[] = "ZLIB.detect"; static const char __pyx_k_LZ4_compress[] = "LZ4.compress"; static const char __pyx_k_pyx_checksum[] = "__pyx_checksum"; static const char __pyx_k_stringsource[] = "stringsource"; static const char __pyx_k_LZMA_compress[] = "LZMA.compress"; static const char __pyx_k_ZLIB_compress[] = "ZLIB.compress"; static const char __pyx_k_borg_compress[] = "borg.compress"; static const char __pyx_k_reduce_cython[] = "__reduce_cython__"; static const char __pyx_k_CNONE_compress[] = "CNONE.compress"; static const char __pyx_k_CompressorBase[] = "CompressorBase"; static const char __pyx_k_LZ4_decompress[] = "LZ4.decompress"; static const char __pyx_k_get_compressor[] = "get_compressor"; static const char __pyx_k_COMPRESSOR_LIST[] = "COMPRESSOR_LIST"; static const char __pyx_k_LZMA_decompress[] = "LZMA.decompress"; static const char __pyx_k_ZLIB_decompress[] = "ZLIB.decompress"; static const char __pyx_k_pyx_PickleError[] = "__pyx_PickleError"; static const char __pyx_k_setstate_cython[] = "__setstate_cython__"; static const char __pyx_k_CNONE_decompress[] = "CNONE.decompress"; static const char __pyx_k_COMPRESSOR_TABLE[] = "COMPRESSOR_TABLE"; static const char __pyx_k_Compressor___init[] = "Compressor.__init__"; static const char __pyx_k_borg_compress_pyx[] = "borg/compress.pyx"; static const char __pyx_k_DecompressionError[] = "DecompressionError"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_Compressor_compress[] = "Compressor.compress"; static const char __pyx_k_lz4_compress_failed[] = "lz4 compress failed"; static const char __pyx_k_Compressor_decompress[] = "Compressor.decompress"; static const char __pyx_k_No_lzma_support_found[] = "No lzma support found."; static const char __pyx_k_lz4_decompress_failed[] = "lz4 decompress failed"; static const char __pyx_k_pyx_unpickle_CompressorBase[] = "__pyx_unpickle_CompressorBase"; static const char __pyx_k_lzma_compression_decompression[] = "\n lzma compression / decompression\n "; static const char __pyx_k_zlib_compression_decompression[] = "\n zlib compression / decompression (python stdlib)\n "; static const char __pyx_k_compresses_using_a_compressor_w[] = "\n compresses using a compressor with given name and parameters\n decompresses everything we can handle (autodetect)\n "; static const char __pyx_k_none_no_compression_just_pass_t[] = "\n none - no compression, just pass through data\n "; static const char __pyx_k_raw_LZ4_compression_decompressi[] = "\n raw LZ4 compression / decompression (liblz4).\n\n Features:\n - lz4 is super fast\n - wrapper releases CPython's GIL to support multithreaded code\n - uses safe lz4 methods that never go beyond the end of the output buffer\n "; static const char __pyx_k_Incompatible_checksums_s_vs_0xd4[] = "Incompatible checksums (%s vs 0xd41d8cd = ())"; static const char __pyx_k_No_decompressor_for_this_data_fo[] = "No decompressor for this data found: %r."; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_n_s_Buffer; static PyObject *__pyx_n_s_CHECK_NONE; static PyObject *__pyx_n_s_CNONE; static PyObject *__pyx_n_s_CNONE_compress; static PyObject *__pyx_n_s_CNONE_decompress; static PyObject *__pyx_n_s_COMPRESSOR_LIST; static PyObject *__pyx_n_s_COMPRESSOR_TABLE; static PyObject *__pyx_n_s_Compressor; static PyObject *__pyx_n_s_CompressorBase; static PyObject *__pyx_n_s_Compressor___init; static PyObject *__pyx_n_s_Compressor_compress; static PyObject *__pyx_n_s_Compressor_decompress; static PyObject *__pyx_n_s_DecompressionError; static PyObject *__pyx_n_s_ID; static PyObject *__pyx_n_s_ImportError; static PyObject *__pyx_kp_s_Incompatible_checksums_s_vs_0xd4; static PyObject *__pyx_n_s_LZ4; static PyObject *__pyx_n_s_LZ4___init; static PyObject *__pyx_n_s_LZ4_compress; static PyObject *__pyx_n_s_LZ4_decompress; static PyObject *__pyx_n_s_LZMA; static PyObject *__pyx_n_s_LZMAError; static PyObject *__pyx_n_s_LZMA___init; static PyObject *__pyx_n_s_LZMA_compress; static PyObject *__pyx_n_s_LZMA_decompress; static PyObject *__pyx_kp_s_No_decompressor_for_this_data_fo; static PyObject *__pyx_kp_s_No_lzma_support_found; static PyObject *__pyx_n_s_PickleError; static PyObject *__pyx_n_s_ValueError; static PyObject *__pyx_n_s_ZLIB; static PyObject *__pyx_n_s_ZLIB___init; static PyObject *__pyx_n_s_ZLIB_compress; static PyObject *__pyx_n_s_ZLIB_decompress; static PyObject *__pyx_n_s_ZLIB_detect; static PyObject *__pyx_kp_b__11; static PyObject *__pyx_kp_b__18; static PyObject *__pyx_kp_b__26; static PyObject *__pyx_kp_b__5; static PyObject *__pyx_kp_b__6; static PyObject *__pyx_n_s_baseclass; static PyObject *__pyx_n_s_borg_compress; static PyObject *__pyx_kp_s_borg_compress_pyx; static PyObject *__pyx_n_s_buf; static PyObject *__pyx_n_s_buffer; static PyObject *__pyx_n_s_check; static PyObject *__pyx_n_s_check_ok; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_cls; static PyObject *__pyx_n_s_cmf; static PyObject *__pyx_n_s_compress; static PyObject *__pyx_kp_s_compresses_using_a_compressor_w; static PyObject *__pyx_n_s_compressor; static PyObject *__pyx_n_s_data; static PyObject *__pyx_n_s_decompress; static PyObject *__pyx_n_s_dest; static PyObject *__pyx_n_s_detect; static PyObject *__pyx_n_s_dict; static PyObject *__pyx_n_s_doc; static PyObject *__pyx_n_s_e; static PyObject *__pyx_n_s_error; static PyObject *__pyx_n_s_flg; static PyObject *__pyx_n_s_get; static PyObject *__pyx_n_s_get_compressor; static PyObject *__pyx_n_s_getstate; static PyObject *__pyx_n_s_hdr; static PyObject *__pyx_n_s_helpers; static PyObject *__pyx_n_s_idata; static PyObject *__pyx_n_s_import; static PyObject *__pyx_n_s_init; static PyObject *__pyx_n_s_is_deflate; static PyObject *__pyx_n_s_isize; static PyObject *__pyx_n_s_kwargs; static PyObject *__pyx_n_s_level; static PyObject *__pyx_n_s_lz4; static PyObject *__pyx_kp_s_lz4_compress_failed; static PyObject *__pyx_kp_s_lz4_decompress_failed; static PyObject *__pyx_n_s_lzma; static PyObject *__pyx_kp_s_lzma_compression_decompression; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_metaclass; static PyObject *__pyx_n_s_module; static PyObject *__pyx_n_s_name; static PyObject *__pyx_n_s_name_2; static PyObject *__pyx_n_s_new; static PyObject *__pyx_n_s_none; static PyObject *__pyx_kp_s_none_no_compression_just_pass_t; static PyObject *__pyx_n_s_null; static PyObject *__pyx_n_s_osize; static PyObject *__pyx_n_s_params; static PyObject *__pyx_n_s_pickle; static PyObject *__pyx_n_s_prepare; static PyObject *__pyx_n_s_preset; static PyObject *__pyx_n_s_pyx_PickleError; static PyObject *__pyx_n_s_pyx_checksum; static PyObject *__pyx_n_s_pyx_result; static PyObject *__pyx_n_s_pyx_state; static PyObject *__pyx_n_s_pyx_type; static PyObject *__pyx_n_s_pyx_unpickle_CompressorBase; static PyObject *__pyx_n_s_qualname; static PyObject *__pyx_kp_s_raw_LZ4_compression_decompressi; static PyObject *__pyx_n_s_reduce; static PyObject *__pyx_n_s_reduce_cython; static PyObject *__pyx_n_s_reduce_ex; static PyObject *__pyx_n_s_rsize; static PyObject *__pyx_n_s_self; static PyObject *__pyx_n_s_setstate; static PyObject *__pyx_n_s_setstate_cython; static PyObject *__pyx_n_s_size; static PyObject *__pyx_n_s_source; static PyObject *__pyx_n_s_startswith; static PyObject *__pyx_kp_s_stringsource; static PyObject *__pyx_n_s_super; static PyObject *__pyx_n_s_test; static PyObject *__pyx_n_s_update; static PyObject *__pyx_n_s_zlib; static PyObject *__pyx_kp_s_zlib_compression_decompression; static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_detect(PyTypeObject *__pyx_v_cls, PyObject *__pyx_v_data); /* proto */ static int __pyx_pf_4borg_8compress_14CompressorBase_2__init__(CYTHON_UNUSED struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v_kwargs); /* proto */ static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_4compress(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_6decompress(CYTHON_UNUSED struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_8__reduce_cython__(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_10__setstate_cython__(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_pf_4borg_8compress_5CNONE_compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_5CNONE_2decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_3LZ4___init__(CYTHON_UNUSED PyObject *__pyx_self, CYTHON_UNUSED PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v_kwargs); /* proto */ static PyObject *__pyx_pf_4borg_8compress_3LZ4_2compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_idata); /* proto */ static PyObject *__pyx_pf_4borg_8compress_3LZ4_4decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_idata); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4LZMA___init__(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_level, PyObject *__pyx_v_kwargs); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4LZMA_2compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4LZMA_4decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4ZLIB_detect(CYTHON_UNUSED PyObject *__pyx_self, CYTHON_UNUSED PyObject *__pyx_v_cls, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4ZLIB_2__init__(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_level, PyObject *__pyx_v_kwargs); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4ZLIB_4compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_4ZLIB_6decompress(CYTHON_UNUSED PyObject *__pyx_self, CYTHON_UNUSED PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_get_compressor(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_name, PyObject *__pyx_v_kwargs); /* proto */ static PyObject *__pyx_pf_4borg_8compress_10Compressor___init__(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_name, PyObject *__pyx_v_kwargs); /* proto */ static PyObject *__pyx_pf_4borg_8compress_10Compressor_2compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_10Compressor_4decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_8compress_2__pyx_unpickle_CompressorBase(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v___pyx_type, long __pyx_v___pyx_checksum, PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_tp_new_4borg_8compress_CompressorBase(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_int_0; static PyObject *__pyx_int_2; static PyObject *__pyx_int_6; static PyObject *__pyx_int_8; static PyObject *__pyx_int_15; static PyObject *__pyx_int_31; static PyObject *__pyx_int_256; static PyObject *__pyx_int_222419149; static PyObject *__pyx_slice_; static PyObject *__pyx_slice__4; static PyObject *__pyx_tuple__2; static PyObject *__pyx_tuple__3; static PyObject *__pyx_tuple__7; static PyObject *__pyx_tuple__9; static PyObject *__pyx_tuple__12; static PyObject *__pyx_tuple__14; static PyObject *__pyx_tuple__16; static PyObject *__pyx_tuple__19; static PyObject *__pyx_tuple__21; static PyObject *__pyx_tuple__22; static PyObject *__pyx_tuple__24; static PyObject *__pyx_tuple__27; static PyObject *__pyx_tuple__29; static PyObject *__pyx_tuple__31; static PyObject *__pyx_tuple__32; static PyObject *__pyx_tuple__34; static PyObject *__pyx_tuple__36; static PyObject *__pyx_tuple__38; static PyObject *__pyx_tuple__40; static PyObject *__pyx_tuple__41; static PyObject *__pyx_tuple__43; static PyObject *__pyx_tuple__45; static PyObject *__pyx_codeobj__8; static PyObject *__pyx_codeobj__10; static PyObject *__pyx_codeobj__13; static PyObject *__pyx_codeobj__15; static PyObject *__pyx_codeobj__17; static PyObject *__pyx_codeobj__20; static PyObject *__pyx_codeobj__23; static PyObject *__pyx_codeobj__25; static PyObject *__pyx_codeobj__28; static PyObject *__pyx_codeobj__30; static PyObject *__pyx_codeobj__33; static PyObject *__pyx_codeobj__35; static PyObject *__pyx_codeobj__37; static PyObject *__pyx_codeobj__39; static PyObject *__pyx_codeobj__42; static PyObject *__pyx_codeobj__44; static PyObject *__pyx_codeobj__46; /* Late includes */ /* "borg/compress.pyx":31 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * return data.startswith(cls.ID) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_1detect(PyObject *__pyx_v_cls, PyObject *__pyx_v_data); /*proto*/ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_1detect(PyObject *__pyx_v_cls, PyObject *__pyx_v_data) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("detect (wrapper)", 0); __pyx_r = __pyx_pf_4borg_8compress_14CompressorBase_detect(((PyTypeObject*)__pyx_v_cls), ((PyObject *)__pyx_v_data)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_detect(PyTypeObject *__pyx_v_cls, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("detect", 0); /* "borg/compress.pyx":32 * @classmethod * def detect(cls, data): * return data.startswith(cls.ID) # <<<<<<<<<<<<<< * * def __init__(self, **kwargs): */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_data, __pyx_n_s_startswith); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 32, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_cls), __pyx_n_s_ID); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 32, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_t_3) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 32, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":31 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * return data.startswith(cls.ID) * */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.compress.CompressorBase.detect", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":34 * return data.startswith(cls.ID) * * def __init__(self, **kwargs): # <<<<<<<<<<<<<< * pass * */ /* Python wrapper */ static int __pyx_pw_4borg_8compress_14CompressorBase_3__init__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static int __pyx_pw_4borg_8compress_14CompressorBase_3__init__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { CYTHON_UNUSED PyObject *__pyx_v_kwargs = 0; int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__ (wrapper)", 0); if (unlikely(PyTuple_GET_SIZE(__pyx_args) > 0)) { __Pyx_RaiseArgtupleInvalid("__init__", 1, 0, 0, PyTuple_GET_SIZE(__pyx_args)); return -1;} if (unlikely(__pyx_kwds) && unlikely(!__Pyx_CheckKeywordStrings(__pyx_kwds, "__init__", 1))) return -1; __pyx_r = __pyx_pf_4borg_8compress_14CompressorBase_2__init__(((struct __pyx_obj_4borg_8compress_CompressorBase *)__pyx_v_self), __pyx_v_kwargs); /* function exit code */ __Pyx_XDECREF(__pyx_v_kwargs); __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_8compress_14CompressorBase_2__init__(CYTHON_UNUSED struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v_kwargs) { int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__", 0); /* function exit code */ __pyx_r = 0; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":37 * pass * * def compress(self, data): # <<<<<<<<<<<<<< * # add ID bytes * return self.ID + data */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_5compress(PyObject *__pyx_v_self, PyObject *__pyx_v_data); /*proto*/ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_5compress(PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("compress (wrapper)", 0); __pyx_r = __pyx_pf_4borg_8compress_14CompressorBase_4compress(((struct __pyx_obj_4borg_8compress_CompressorBase *)__pyx_v_self), ((PyObject *)__pyx_v_data)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_4compress(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannySetupContext("compress", 0); /* "borg/compress.pyx":39 * def compress(self, data): * # add ID bytes * return self.ID + data # <<<<<<<<<<<<<< * * def decompress(self, data): */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_ID); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyNumber_Add(__pyx_t_1, __pyx_v_data); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_2; __pyx_t_2 = 0; goto __pyx_L0; /* "borg/compress.pyx":37 * pass * * def compress(self, data): # <<<<<<<<<<<<<< * # add ID bytes * return self.ID + data */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.compress.CompressorBase.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":41 * return self.ID + data * * def decompress(self, data): # <<<<<<<<<<<<<< * # strip ID bytes * return data[2:] */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_7decompress(PyObject *__pyx_v_self, PyObject *__pyx_v_data); /*proto*/ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_7decompress(PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decompress (wrapper)", 0); __pyx_r = __pyx_pf_4borg_8compress_14CompressorBase_6decompress(((struct __pyx_obj_4borg_8compress_CompressorBase *)__pyx_v_self), ((PyObject *)__pyx_v_data)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_6decompress(CYTHON_UNUSED struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("decompress", 0); /* "borg/compress.pyx":43 * def decompress(self, data): * # strip ID bytes * return data[2:] # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = __Pyx_PyObject_GetSlice(__pyx_v_data, 2, 0, NULL, NULL, &__pyx_slice_, 1, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 43, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":41 * return self.ID + data * * def decompress(self, data): # <<<<<<<<<<<<<< * # strip ID bytes * return data[2:] */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.compress.CompressorBase.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * cdef bint use_setstate * state = () */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_9__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_9__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_8compress_14CompressorBase_8__reduce_cython__(((struct __pyx_obj_4borg_8compress_CompressorBase *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_8__reduce_cython__(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self) { int __pyx_v_use_setstate; PyObject *__pyx_v_state = NULL; PyObject *__pyx_v__dict = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; int __pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":3 * def __reduce_cython__(self): * cdef bint use_setstate * state = () # <<<<<<<<<<<<<< * _dict = getattr(self, '__dict__', None) * if _dict is not None: */ __Pyx_INCREF(__pyx_empty_tuple); __pyx_v_state = __pyx_empty_tuple; /* "(tree fragment)":4 * cdef bint use_setstate * state = () * _dict = getattr(self, '__dict__', None) # <<<<<<<<<<<<<< * if _dict is not None: * state += (_dict,) */ __pyx_t_1 = __Pyx_GetAttr3(((PyObject *)__pyx_v_self), __pyx_n_s_dict, Py_None); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v__dict = __pyx_t_1; __pyx_t_1 = 0; /* "(tree fragment)":5 * state = () * _dict = getattr(self, '__dict__', None) * if _dict is not None: # <<<<<<<<<<<<<< * state += (_dict,) * use_setstate = True */ __pyx_t_2 = (__pyx_v__dict != Py_None); __pyx_t_3 = (__pyx_t_2 != 0); if (__pyx_t_3) { /* "(tree fragment)":6 * _dict = getattr(self, '__dict__', None) * if _dict is not None: * state += (_dict,) # <<<<<<<<<<<<<< * use_setstate = True * else: */ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 6, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_v__dict); __Pyx_GIVEREF(__pyx_v__dict); PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_v__dict); __pyx_t_4 = PyNumber_InPlaceAdd(__pyx_v_state, __pyx_t_1); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 6, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF_SET(__pyx_v_state, ((PyObject*)__pyx_t_4)); __pyx_t_4 = 0; /* "(tree fragment)":7 * if _dict is not None: * state += (_dict,) * use_setstate = True # <<<<<<<<<<<<<< * else: * use_setstate = False */ __pyx_v_use_setstate = 1; /* "(tree fragment)":5 * state = () * _dict = getattr(self, '__dict__', None) * if _dict is not None: # <<<<<<<<<<<<<< * state += (_dict,) * use_setstate = True */ goto __pyx_L3; } /* "(tree fragment)":9 * use_setstate = True * else: * use_setstate = False # <<<<<<<<<<<<<< * if use_setstate: * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, None), state */ /*else*/ { __pyx_v_use_setstate = 0; } __pyx_L3:; /* "(tree fragment)":10 * else: * use_setstate = False * if use_setstate: # <<<<<<<<<<<<<< * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, None), state * else: */ __pyx_t_3 = (__pyx_v_use_setstate != 0); if (__pyx_t_3) { /* "(tree fragment)":11 * use_setstate = False * if use_setstate: * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, None), state # <<<<<<<<<<<<<< * else: * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, state) */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_4, __pyx_n_s_pyx_unpickle_CompressorBase); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_1 = PyTuple_New(3); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self)))); __Pyx_GIVEREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self)))); PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self)))); __Pyx_INCREF(__pyx_int_222419149); __Pyx_GIVEREF(__pyx_int_222419149); PyTuple_SET_ITEM(__pyx_t_1, 1, __pyx_int_222419149); __Pyx_INCREF(Py_None); __Pyx_GIVEREF(Py_None); PyTuple_SET_ITEM(__pyx_t_1, 2, Py_None); __pyx_t_5 = PyTuple_New(3); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_1); PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_1); __Pyx_INCREF(__pyx_v_state); __Pyx_GIVEREF(__pyx_v_state); PyTuple_SET_ITEM(__pyx_t_5, 2, __pyx_v_state); __pyx_t_4 = 0; __pyx_t_1 = 0; __pyx_r = __pyx_t_5; __pyx_t_5 = 0; goto __pyx_L0; /* "(tree fragment)":10 * else: * use_setstate = False * if use_setstate: # <<<<<<<<<<<<<< * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, None), state * else: */ } /* "(tree fragment)":13 * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, None), state * else: * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, state) # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * __pyx_unpickle_CompressorBase__set_state(self, __pyx_state) */ /*else*/ { __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_pyx_unpickle_CompressorBase); if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 13, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_1 = PyTuple_New(3); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 13, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self)))); __Pyx_GIVEREF(((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self)))); PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)Py_TYPE(((PyObject *)__pyx_v_self)))); __Pyx_INCREF(__pyx_int_222419149); __Pyx_GIVEREF(__pyx_int_222419149); PyTuple_SET_ITEM(__pyx_t_1, 1, __pyx_int_222419149); __Pyx_INCREF(__pyx_v_state); __Pyx_GIVEREF(__pyx_v_state); PyTuple_SET_ITEM(__pyx_t_1, 2, __pyx_v_state); __pyx_t_4 = PyTuple_New(2); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 13, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_4, 0, __pyx_t_5); __Pyx_GIVEREF(__pyx_t_1); PyTuple_SET_ITEM(__pyx_t_4, 1, __pyx_t_1); __pyx_t_5 = 0; __pyx_t_1 = 0; __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L0; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * cdef bint use_setstate * state = () */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.compress.CompressorBase.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_state); __Pyx_XDECREF(__pyx_v__dict); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":14 * else: * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, state) * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * __pyx_unpickle_CompressorBase__set_state(self, __pyx_state) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_11__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_8compress_14CompressorBase_11__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_8compress_14CompressorBase_10__setstate_cython__(((struct __pyx_obj_4borg_8compress_CompressorBase *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_14CompressorBase_10__setstate_cython__(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":15 * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, state) * def __setstate_cython__(self, __pyx_state): * __pyx_unpickle_CompressorBase__set_state(self, __pyx_state) # <<<<<<<<<<<<<< */ if (!(likely(PyTuple_CheckExact(__pyx_v___pyx_state))||((__pyx_v___pyx_state) == Py_None)||(PyErr_Format(PyExc_TypeError, "Expected %.16s, got %.200s", "tuple", Py_TYPE(__pyx_v___pyx_state)->tp_name), 0))) __PYX_ERR(1, 15, __pyx_L1_error) __pyx_t_1 = __pyx_f_4borg_8compress___pyx_unpickle_CompressorBase__set_state(__pyx_v_self, ((PyObject*)__pyx_v___pyx_state)); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 15, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "(tree fragment)":14 * else: * return __pyx_unpickle_CompressorBase, (type(self), 0xd41d8cd, state) * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * __pyx_unpickle_CompressorBase__set_state(self, __pyx_state) */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.compress.CompressorBase.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":53 * name = 'none' * * def compress(self, data): # <<<<<<<<<<<<<< * return super().compress(data) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_5CNONE_1compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_5CNONE_1compress = {"compress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_5CNONE_1compress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_5CNONE_1compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("compress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, 1); __PYX_ERR(0, 53, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "compress") < 0)) __PYX_ERR(0, 53, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 53, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.CNONE.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_5CNONE_compress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_5CNONE_compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; __Pyx_RefNannySetupContext("compress", 0); /* "borg/compress.pyx":54 * * def compress(self, data): * return super().compress(data) # <<<<<<<<<<<<<< * * def decompress(self, data): */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_2) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 54, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_2); __pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_t_2); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_v_self); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_3, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_compress); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_data); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":53 * name = 'none' * * def compress(self, data): # <<<<<<<<<<<<<< * return super().compress(data) * */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.compress.CNONE.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":56 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * if not isinstance(data, bytes): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_5CNONE_3decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_5CNONE_3decompress = {"decompress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_5CNONE_3decompress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_5CNONE_3decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decompress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, 1); __PYX_ERR(0, 56, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "decompress") < 0)) __PYX_ERR(0, 56, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 56, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.CNONE.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_5CNONE_2decompress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_5CNONE_2decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; int __pyx_t_5; __Pyx_RefNannySetupContext("decompress", 0); __Pyx_INCREF(__pyx_v_data); /* "borg/compress.pyx":57 * * def decompress(self, data): * data = super().decompress(data) # <<<<<<<<<<<<<< * if not isinstance(data, bytes): * data = bytes(data) */ __pyx_t_2 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_2) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 57, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_2); __pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_t_2); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_v_self); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_3, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_decompress); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_data); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF_SET(__pyx_v_data, __pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":58 * def decompress(self, data): * data = super().decompress(data) * if not isinstance(data, bytes): # <<<<<<<<<<<<<< * data = bytes(data) * return data */ __pyx_t_4 = PyBytes_Check(__pyx_v_data); __pyx_t_5 = ((!(__pyx_t_4 != 0)) != 0); if (__pyx_t_5) { /* "borg/compress.pyx":59 * data = super().decompress(data) * if not isinstance(data, bytes): * data = bytes(data) # <<<<<<<<<<<<<< * return data * */ __pyx_t_1 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyBytes_Type)), __pyx_v_data); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF_SET(__pyx_v_data, __pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":58 * def decompress(self, data): * data = super().decompress(data) * if not isinstance(data, bytes): # <<<<<<<<<<<<<< * data = bytes(data) * return data */ } /* "borg/compress.pyx":60 * if not isinstance(data, bytes): * data = bytes(data) * return data # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(__pyx_v_data); __pyx_r = __pyx_v_data; goto __pyx_L0; /* "borg/compress.pyx":56 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * if not isinstance(data, bytes): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.compress.CNONE.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_data); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":75 * name = 'lz4' * * def __init__(self, **kwargs): # <<<<<<<<<<<<<< * pass * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_3LZ4_1__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_3LZ4_1__init__ = {"__init__", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_3LZ4_1__init__, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_3LZ4_1__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { CYTHON_UNUSED PyObject *__pyx_v_self = 0; CYTHON_UNUSED PyObject *__pyx_v_kwargs = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__ (wrapper)", 0); __pyx_v_kwargs = PyDict_New(); if (unlikely(!__pyx_v_kwargs)) return NULL; __Pyx_GOTREF(__pyx_v_kwargs); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,0}; PyObject* values[1] = {0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, __pyx_v_kwargs, values, pos_args, "__init__") < 0)) __PYX_ERR(0, 75, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 1) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); } __pyx_v_self = values[0]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__init__", 1, 1, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 75, __pyx_L3_error) __pyx_L3_error:; __Pyx_DECREF(__pyx_v_kwargs); __pyx_v_kwargs = 0; __Pyx_AddTraceback("borg.compress.LZ4.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_3LZ4___init__(__pyx_self, __pyx_v_self, __pyx_v_kwargs); /* function exit code */ __Pyx_XDECREF(__pyx_v_kwargs); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_3LZ4___init__(CYTHON_UNUSED PyObject *__pyx_self, CYTHON_UNUSED PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v_kwargs) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__", 0); /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":78 * pass * * def compress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_3LZ4_3compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_3LZ4_3compress = {"compress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_3LZ4_3compress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_3LZ4_3compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_idata = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("compress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_idata,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_idata)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, 1); __PYX_ERR(0, 78, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "compress") < 0)) __PYX_ERR(0, 78, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_idata = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 78, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.LZ4.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_3LZ4_2compress(__pyx_self, __pyx_v_self, __pyx_v_idata); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_3LZ4_2compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_idata) { int __pyx_v_isize; int __pyx_v_osize; char *__pyx_v_source; char *__pyx_v_dest; PyObject *__pyx_v_buf = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; int __pyx_t_2; PyObject *__pyx_t_3 = NULL; Py_ssize_t __pyx_t_4; char *__pyx_t_5; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; __Pyx_RefNannySetupContext("compress", 0); __Pyx_INCREF(__pyx_v_idata); /* "borg/compress.pyx":79 * * def compress(self, idata): * if not isinstance(idata, bytes): # <<<<<<<<<<<<<< * idata = bytes(idata) # code below does not work with memoryview * cdef int isize = len(idata) */ __pyx_t_1 = PyBytes_Check(__pyx_v_idata); __pyx_t_2 = ((!(__pyx_t_1 != 0)) != 0); if (__pyx_t_2) { /* "borg/compress.pyx":80 * def compress(self, idata): * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview # <<<<<<<<<<<<<< * cdef int isize = len(idata) * cdef int osize */ __pyx_t_3 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyBytes_Type)), __pyx_v_idata); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF_SET(__pyx_v_idata, __pyx_t_3); __pyx_t_3 = 0; /* "borg/compress.pyx":79 * * def compress(self, idata): * if not isinstance(idata, bytes): # <<<<<<<<<<<<<< * idata = bytes(idata) # code below does not work with memoryview * cdef int isize = len(idata) */ } /* "borg/compress.pyx":81 * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview * cdef int isize = len(idata) # <<<<<<<<<<<<<< * cdef int osize * cdef char *source = idata */ __pyx_t_4 = PyObject_Length(__pyx_v_idata); if (unlikely(__pyx_t_4 == ((Py_ssize_t)-1))) __PYX_ERR(0, 81, __pyx_L1_error) __pyx_v_isize = __pyx_t_4; /* "borg/compress.pyx":83 * cdef int isize = len(idata) * cdef int osize * cdef char *source = idata # <<<<<<<<<<<<<< * cdef char *dest * osize = LZ4_compressBound(isize) */ __pyx_t_5 = __Pyx_PyObject_AsWritableString(__pyx_v_idata); if (unlikely((!__pyx_t_5) && PyErr_Occurred())) __PYX_ERR(0, 83, __pyx_L1_error) __pyx_v_source = __pyx_t_5; /* "borg/compress.pyx":85 * cdef char *source = idata * cdef char *dest * osize = LZ4_compressBound(isize) # <<<<<<<<<<<<<< * buf = buffer.get(osize) * dest = buf */ __pyx_v_osize = LZ4_compressBound(__pyx_v_isize); /* "borg/compress.pyx":86 * cdef char *dest * osize = LZ4_compressBound(isize) * buf = buffer.get(osize) # <<<<<<<<<<<<<< * dest = buf * with nogil: */ __Pyx_GetModuleGlobalName(__pyx_t_6, __pyx_n_s_buffer); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 86, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_6, __pyx_n_s_get); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 86, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_t_6 = __Pyx_PyInt_From_int(__pyx_v_osize); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 86, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_7))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); } } __pyx_t_3 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_7, __pyx_t_8, __pyx_t_6) : __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_6); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 86, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_v_buf = __pyx_t_3; __pyx_t_3 = 0; /* "borg/compress.pyx":87 * osize = LZ4_compressBound(isize) * buf = buffer.get(osize) * dest = buf # <<<<<<<<<<<<<< * with nogil: * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) */ __pyx_t_5 = __Pyx_PyObject_AsWritableString(__pyx_v_buf); if (unlikely((!__pyx_t_5) && PyErr_Occurred())) __PYX_ERR(0, 87, __pyx_L1_error) __pyx_v_dest = ((char *)__pyx_t_5); /* "borg/compress.pyx":88 * buf = buffer.get(osize) * dest = buf * with nogil: # <<<<<<<<<<<<<< * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) * if not osize: */ { #ifdef WITH_THREAD PyThreadState *_save; Py_UNBLOCK_THREADS __Pyx_FastGIL_Remember(); #endif /*try:*/ { /* "borg/compress.pyx":89 * dest = buf * with nogil: * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) # <<<<<<<<<<<<<< * if not osize: * raise Exception('lz4 compress failed') */ __pyx_v_osize = LZ4_compress_limitedOutput(__pyx_v_source, __pyx_v_dest, __pyx_v_isize, __pyx_v_osize); } /* "borg/compress.pyx":88 * buf = buffer.get(osize) * dest = buf * with nogil: # <<<<<<<<<<<<<< * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) * if not osize: */ /*finally:*/ { /*normal exit:*/{ #ifdef WITH_THREAD __Pyx_FastGIL_Forget(); Py_BLOCK_THREADS #endif goto __pyx_L6; } __pyx_L6:; } } /* "borg/compress.pyx":90 * with nogil: * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) * if not osize: # <<<<<<<<<<<<<< * raise Exception('lz4 compress failed') * return super().compress(dest[:osize]) */ __pyx_t_2 = ((!(__pyx_v_osize != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/compress.pyx":91 * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) * if not osize: * raise Exception('lz4 compress failed') # <<<<<<<<<<<<<< * return super().compress(dest[:osize]) * */ __pyx_t_3 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_Raise(__pyx_t_3, 0, 0, 0); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __PYX_ERR(0, 91, __pyx_L1_error) /* "borg/compress.pyx":90 * with nogil: * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) * if not osize: # <<<<<<<<<<<<<< * raise Exception('lz4 compress failed') * return super().compress(dest[:osize]) */ } /* "borg/compress.pyx":92 * if not osize: * raise Exception('lz4 compress failed') * return super().compress(dest[:osize]) # <<<<<<<<<<<<<< * * def decompress(self, idata): */ __Pyx_XDECREF(__pyx_r); __pyx_t_7 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_7) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 92, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_7); __pyx_t_6 = PyTuple_New(2); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 92, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_GIVEREF(__pyx_t_7); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_7); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_6, 1, __pyx_v_self); __pyx_t_7 = 0; __pyx_t_7 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_6, NULL); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 92, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_t_7, __pyx_n_s_compress); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 92, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_7 = __Pyx_PyBytes_FromStringAndSize(__pyx_v_dest + 0, __pyx_v_osize - 0); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 92, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_6))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); } } __pyx_t_3 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_6, __pyx_t_8, __pyx_t_7) : __Pyx_PyObject_CallOneArg(__pyx_t_6, __pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 92, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* "borg/compress.pyx":78 * pass * * def compress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.compress.LZ4.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_buf); __Pyx_XDECREF(__pyx_v_idata); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":94 * return super().compress(dest[:osize]) * * def decompress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_3LZ4_5decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_3LZ4_5decompress = {"decompress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_3LZ4_5decompress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_3LZ4_5decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_idata = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decompress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_idata,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_idata)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, 1); __PYX_ERR(0, 94, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "decompress") < 0)) __PYX_ERR(0, 94, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_idata = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 94, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.LZ4.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_3LZ4_4decompress(__pyx_self, __pyx_v_self, __pyx_v_idata); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_3LZ4_4decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_idata) { int __pyx_v_isize; int __pyx_v_osize; int __pyx_v_rsize; char *__pyx_v_source; char *__pyx_v_dest; PyObject *__pyx_v_buf = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; int __pyx_t_2; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; Py_ssize_t __pyx_t_6; char *__pyx_t_7; long __pyx_t_8; PyObject *__pyx_t_9 = NULL; int __pyx_t_10; __Pyx_RefNannySetupContext("decompress", 0); __Pyx_INCREF(__pyx_v_idata); /* "borg/compress.pyx":95 * * def decompress(self, idata): * if not isinstance(idata, bytes): # <<<<<<<<<<<<<< * idata = bytes(idata) # code below does not work with memoryview * idata = super().decompress(idata) */ __pyx_t_1 = PyBytes_Check(__pyx_v_idata); __pyx_t_2 = ((!(__pyx_t_1 != 0)) != 0); if (__pyx_t_2) { /* "borg/compress.pyx":96 * def decompress(self, idata): * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview # <<<<<<<<<<<<<< * idata = super().decompress(idata) * cdef int isize = len(idata) */ __pyx_t_3 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyBytes_Type)), __pyx_v_idata); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 96, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF_SET(__pyx_v_idata, __pyx_t_3); __pyx_t_3 = 0; /* "borg/compress.pyx":95 * * def decompress(self, idata): * if not isinstance(idata, bytes): # <<<<<<<<<<<<<< * idata = bytes(idata) # code below does not work with memoryview * idata = super().decompress(idata) */ } /* "borg/compress.pyx":97 * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview * idata = super().decompress(idata) # <<<<<<<<<<<<<< * cdef int isize = len(idata) * cdef int osize */ __pyx_t_4 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_4) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 97, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_4); __pyx_t_5 = PyTuple_New(2); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_4); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_v_self); __pyx_t_4 = 0; __pyx_t_4 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_5, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_t_4, __pyx_n_s_decompress); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_5))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_3 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_4, __pyx_v_idata) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_idata); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_DECREF_SET(__pyx_v_idata, __pyx_t_3); __pyx_t_3 = 0; /* "borg/compress.pyx":98 * idata = bytes(idata) # code below does not work with memoryview * idata = super().decompress(idata) * cdef int isize = len(idata) # <<<<<<<<<<<<<< * cdef int osize * cdef int rsize */ __pyx_t_6 = PyObject_Length(__pyx_v_idata); if (unlikely(__pyx_t_6 == ((Py_ssize_t)-1))) __PYX_ERR(0, 98, __pyx_L1_error) __pyx_v_isize = __pyx_t_6; /* "borg/compress.pyx":101 * cdef int osize * cdef int rsize * cdef char *source = idata # <<<<<<<<<<<<<< * cdef char *dest * # a bit more than 8MB is enough for the usual data sizes yielded by the chunker. */ __pyx_t_7 = __Pyx_PyObject_AsWritableString(__pyx_v_idata); if (unlikely((!__pyx_t_7) && PyErr_Occurred())) __PYX_ERR(0, 101, __pyx_L1_error) __pyx_v_source = __pyx_t_7; /* "borg/compress.pyx":105 * # a bit more than 8MB is enough for the usual data sizes yielded by the chunker. * # allocate more if isize * 3 is already bigger, to avoid having to resize often. * osize = max(int(1.1 * 2**23), isize * 3) # <<<<<<<<<<<<<< * while True: * buf = buffer.get(osize) */ __pyx_t_8 = (__pyx_v_isize * 3); __pyx_t_3 = __Pyx_PyInt_FromDouble((1.1 * 8388608.0)); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyInt_From_long(__pyx_t_8); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_9 = PyObject_RichCompare(__pyx_t_4, __pyx_t_3, Py_GT); __Pyx_XGOTREF(__pyx_t_9); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_2 = __Pyx_PyObject_IsTrue(__pyx_t_9); if (unlikely(__pyx_t_2 < 0)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; if (__pyx_t_2) { __pyx_t_9 = __Pyx_PyInt_From_long(__pyx_t_8); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __pyx_t_5 = __pyx_t_9; __pyx_t_9 = 0; } else { __Pyx_INCREF(__pyx_t_3); __pyx_t_5 = __pyx_t_3; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_10 = __Pyx_PyInt_As_int(__pyx_t_5); if (unlikely((__pyx_t_10 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_v_osize = __pyx_t_10; /* "borg/compress.pyx":106 * # allocate more if isize * 3 is already bigger, to avoid having to resize often. * osize = max(int(1.1 * 2**23), isize * 3) * while True: # <<<<<<<<<<<<<< * buf = buffer.get(osize) * dest = buf */ while (1) { /* "borg/compress.pyx":107 * osize = max(int(1.1 * 2**23), isize * 3) * while True: * buf = buffer.get(osize) # <<<<<<<<<<<<<< * dest = buf * with nogil: */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_buffer); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 107, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_t_3, __pyx_n_s_get); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 107, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyInt_From_int(__pyx_v_osize); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 107, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_9))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_9); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_9); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_9, function); } } __pyx_t_5 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_9, __pyx_t_4, __pyx_t_3) : __Pyx_PyObject_CallOneArg(__pyx_t_9, __pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 107, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_XDECREF_SET(__pyx_v_buf, __pyx_t_5); __pyx_t_5 = 0; /* "borg/compress.pyx":108 * while True: * buf = buffer.get(osize) * dest = buf # <<<<<<<<<<<<<< * with nogil: * rsize = LZ4_decompress_safe(source, dest, isize, osize) */ __pyx_t_7 = __Pyx_PyObject_AsWritableString(__pyx_v_buf); if (unlikely((!__pyx_t_7) && PyErr_Occurred())) __PYX_ERR(0, 108, __pyx_L1_error) __pyx_v_dest = ((char *)__pyx_t_7); /* "borg/compress.pyx":109 * buf = buffer.get(osize) * dest = buf * with nogil: # <<<<<<<<<<<<<< * rsize = LZ4_decompress_safe(source, dest, isize, osize) * if rsize >= 0: */ { #ifdef WITH_THREAD PyThreadState *_save; Py_UNBLOCK_THREADS __Pyx_FastGIL_Remember(); #endif /*try:*/ { /* "borg/compress.pyx":110 * dest = buf * with nogil: * rsize = LZ4_decompress_safe(source, dest, isize, osize) # <<<<<<<<<<<<<< * if rsize >= 0: * break */ __pyx_v_rsize = LZ4_decompress_safe(__pyx_v_source, __pyx_v_dest, __pyx_v_isize, __pyx_v_osize); } /* "borg/compress.pyx":109 * buf = buffer.get(osize) * dest = buf * with nogil: # <<<<<<<<<<<<<< * rsize = LZ4_decompress_safe(source, dest, isize, osize) * if rsize >= 0: */ /*finally:*/ { /*normal exit:*/{ #ifdef WITH_THREAD __Pyx_FastGIL_Forget(); Py_BLOCK_THREADS #endif goto __pyx_L10; } __pyx_L10:; } } /* "borg/compress.pyx":111 * with nogil: * rsize = LZ4_decompress_safe(source, dest, isize, osize) * if rsize >= 0: # <<<<<<<<<<<<<< * break * if osize > 2 ** 30: */ __pyx_t_2 = ((__pyx_v_rsize >= 0) != 0); if (__pyx_t_2) { /* "borg/compress.pyx":112 * rsize = LZ4_decompress_safe(source, dest, isize, osize) * if rsize >= 0: * break # <<<<<<<<<<<<<< * if osize > 2 ** 30: * # this is insane, get out of here */ goto __pyx_L5_break; /* "borg/compress.pyx":111 * with nogil: * rsize = LZ4_decompress_safe(source, dest, isize, osize) * if rsize >= 0: # <<<<<<<<<<<<<< * break * if osize > 2 ** 30: */ } /* "borg/compress.pyx":113 * if rsize >= 0: * break * if osize > 2 ** 30: # <<<<<<<<<<<<<< * # this is insane, get out of here * raise DecompressionError('lz4 decompress failed') */ __pyx_t_2 = ((__pyx_v_osize > 0x40000000) != 0); if (unlikely(__pyx_t_2)) { /* "borg/compress.pyx":115 * if osize > 2 ** 30: * # this is insane, get out of here * raise DecompressionError('lz4 decompress failed') # <<<<<<<<<<<<<< * # likely the buffer was too small, get a bigger one: * osize = int(1.5 * osize) */ __Pyx_GetModuleGlobalName(__pyx_t_9, __pyx_n_s_DecompressionError); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 115, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __pyx_t_3 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_9))) { __pyx_t_3 = PyMethod_GET_SELF(__pyx_t_9); if (likely(__pyx_t_3)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_9); __Pyx_INCREF(__pyx_t_3); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_9, function); } } __pyx_t_5 = (__pyx_t_3) ? __Pyx_PyObject_Call2Args(__pyx_t_9, __pyx_t_3, __pyx_kp_s_lz4_decompress_failed) : __Pyx_PyObject_CallOneArg(__pyx_t_9, __pyx_kp_s_lz4_decompress_failed); __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 115, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_Raise(__pyx_t_5, 0, 0, 0); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __PYX_ERR(0, 115, __pyx_L1_error) /* "borg/compress.pyx":113 * if rsize >= 0: * break * if osize > 2 ** 30: # <<<<<<<<<<<<<< * # this is insane, get out of here * raise DecompressionError('lz4 decompress failed') */ } /* "borg/compress.pyx":117 * raise DecompressionError('lz4 decompress failed') * # likely the buffer was too small, get a bigger one: * osize = int(1.5 * osize) # <<<<<<<<<<<<<< * return dest[:rsize] * */ __pyx_v_osize = ((int)(1.5 * __pyx_v_osize)); } __pyx_L5_break:; /* "borg/compress.pyx":118 * # likely the buffer was too small, get a bigger one: * osize = int(1.5 * osize) * return dest[:rsize] # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_5 = __Pyx_PyBytes_FromStringAndSize(__pyx_v_dest + 0, __pyx_v_rsize - 0); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 118, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_r = __pyx_t_5; __pyx_t_5 = 0; goto __pyx_L0; /* "borg/compress.pyx":94 * return super().compress(dest[:osize]) * * def decompress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_9); __Pyx_AddTraceback("borg.compress.LZ4.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_buf); __Pyx_XDECREF(__pyx_v_idata); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":128 * name = 'lzma' * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4LZMA_1__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4LZMA_1__init__ = {"__init__", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4LZMA_1__init__, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4LZMA_1__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_level = 0; PyObject *__pyx_v_kwargs = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__ (wrapper)", 0); __pyx_v_kwargs = PyDict_New(); if (unlikely(!__pyx_v_kwargs)) return NULL; __Pyx_GOTREF(__pyx_v_kwargs); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_level,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)((PyObject *)__pyx_int_6)); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_level); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, __pyx_v_kwargs, values, pos_args, "__init__") < 0)) __PYX_ERR(0, 128, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_self = values[0]; __pyx_v_level = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__init__", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 128, __pyx_L3_error) __pyx_L3_error:; __Pyx_DECREF(__pyx_v_kwargs); __pyx_v_kwargs = 0; __Pyx_AddTraceback("borg.compress.LZMA.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4LZMA___init__(__pyx_self, __pyx_v_self, __pyx_v_level, __pyx_v_kwargs); /* function exit code */ __Pyx_XDECREF(__pyx_v_kwargs); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4LZMA___init__(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_level, PyObject *__pyx_v_kwargs) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; int __pyx_t_3; int __pyx_t_4; __Pyx_RefNannySetupContext("__init__", 0); /* "borg/compress.pyx":129 * * def __init__(self, level=6, **kwargs): * super().__init__(**kwargs) # <<<<<<<<<<<<<< * self.level = level * if lzma is None: */ __pyx_t_1 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_1) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 129, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_1); __pyx_t_2 = PyTuple_New(2); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_GIVEREF(__pyx_t_1); PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_1); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_2, 1, __pyx_v_self); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_init); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_empty_tuple, __pyx_v_kwargs); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":130 * def __init__(self, level=6, **kwargs): * super().__init__(**kwargs) * self.level = level # <<<<<<<<<<<<<< * if lzma is None: * raise ValueError('No lzma support found.') */ if (__Pyx_PyObject_SetAttrStr(__pyx_v_self, __pyx_n_s_level, __pyx_v_level) < 0) __PYX_ERR(0, 130, __pyx_L1_error) /* "borg/compress.pyx":131 * super().__init__(**kwargs) * self.level = level * if lzma is None: # <<<<<<<<<<<<<< * raise ValueError('No lzma support found.') * */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_lzma); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 131, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = (__pyx_t_1 == Py_None); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_4 = (__pyx_t_3 != 0); if (unlikely(__pyx_t_4)) { /* "borg/compress.pyx":132 * self.level = level * if lzma is None: * raise ValueError('No lzma support found.') # <<<<<<<<<<<<<< * * def compress(self, data): */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_tuple__3, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(0, 132, __pyx_L1_error) /* "borg/compress.pyx":131 * super().__init__(**kwargs) * self.level = level * if lzma is None: # <<<<<<<<<<<<<< * raise ValueError('No lzma support found.') * */ } /* "borg/compress.pyx":128 * name = 'lzma' * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.compress.LZMA.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":134 * raise ValueError('No lzma support found.') * * def compress(self, data): # <<<<<<<<<<<<<< * # we do not need integrity checks in lzma, we do that already * data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4LZMA_3compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4LZMA_3compress = {"compress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4LZMA_3compress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4LZMA_3compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("compress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, 1); __PYX_ERR(0, 134, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "compress") < 0)) __PYX_ERR(0, 134, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 134, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.LZMA.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4LZMA_2compress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4LZMA_2compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("compress", 0); __Pyx_INCREF(__pyx_v_data); /* "borg/compress.pyx":136 * def compress(self, data): * # we do not need integrity checks in lzma, we do that already * data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) # <<<<<<<<<<<<<< * return super().compress(data) * */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_lzma); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_compress); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_v_data); __Pyx_GIVEREF(__pyx_v_data); PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_v_data); __pyx_t_3 = __Pyx_PyDict_NewPresized(2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyObject_GetAttrStr(__pyx_v_self, __pyx_n_s_level); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); if (PyDict_SetItem(__pyx_t_3, __pyx_n_s_preset, __pyx_t_4) < 0) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_GetModuleGlobalName(__pyx_t_4, __pyx_n_s_lzma); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_t_4, __pyx_n_s_CHECK_NONE); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (PyDict_SetItem(__pyx_t_3, __pyx_n_s_check, __pyx_t_5) < 0) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_t_1, __pyx_t_3); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 136, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF_SET(__pyx_v_data, __pyx_t_5); __pyx_t_5 = 0; /* "borg/compress.pyx":137 * # we do not need integrity checks in lzma, we do that already * data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) * return super().compress(data) # <<<<<<<<<<<<<< * * def decompress(self, data): */ __Pyx_XDECREF(__pyx_r); __pyx_t_3 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_3) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 137, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_3); __pyx_t_1 = PyTuple_New(2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 137, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_t_3); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_1, 1, __pyx_v_self); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_1, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 137, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_t_3, __pyx_n_s_compress); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 137, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_1))) { __pyx_t_3 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_3)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_3); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_5 = (__pyx_t_3) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_3, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_v_data); __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 137, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_5; __pyx_t_5 = 0; goto __pyx_L0; /* "borg/compress.pyx":134 * raise ValueError('No lzma support found.') * * def compress(self, data): # <<<<<<<<<<<<<< * # we do not need integrity checks in lzma, we do that already * data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.compress.LZMA.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_data); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":139 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * try: */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4LZMA_5decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4LZMA_5decompress = {"decompress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4LZMA_5decompress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4LZMA_5decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decompress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, 1); __PYX_ERR(0, 139, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "decompress") < 0)) __PYX_ERR(0, 139, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 139, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.LZMA.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4LZMA_4decompress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4LZMA_4decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_v_e = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; int __pyx_t_9; PyObject *__pyx_t_10 = NULL; PyObject *__pyx_t_11 = NULL; __Pyx_RefNannySetupContext("decompress", 0); __Pyx_INCREF(__pyx_v_data); /* "borg/compress.pyx":140 * * def decompress(self, data): * data = super().decompress(data) # <<<<<<<<<<<<<< * try: * return lzma.decompress(data) */ __pyx_t_2 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_2) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 140, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_2); __pyx_t_3 = PyTuple_New(2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 140, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_3, 0, __pyx_t_2); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_3, 1, __pyx_v_self); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_3, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 140, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_decompress); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 140, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_data); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 140, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF_SET(__pyx_v_data, __pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":141 * def decompress(self, data): * data = super().decompress(data) * try: # <<<<<<<<<<<<<< * return lzma.decompress(data) * except lzma.LZMAError as e: */ { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ExceptionSave(&__pyx_t_4, &__pyx_t_5, &__pyx_t_6); __Pyx_XGOTREF(__pyx_t_4); __Pyx_XGOTREF(__pyx_t_5); __Pyx_XGOTREF(__pyx_t_6); /*try:*/ { /* "borg/compress.pyx":142 * data = super().decompress(data) * try: * return lzma.decompress(data) # <<<<<<<<<<<<<< * except lzma.LZMAError as e: * raise DecompressionError(str(e)) from None */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_lzma); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 142, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_3, __pyx_n_s_decompress); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 142, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_3 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_3)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_3); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_3) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_3, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_v_data); __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 142, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L7_try_return; /* "borg/compress.pyx":141 * def decompress(self, data): * data = super().decompress(data) * try: # <<<<<<<<<<<<<< * return lzma.decompress(data) * except lzma.LZMAError as e: */ } __pyx_L3_error:; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":143 * try: * return lzma.decompress(data) * except lzma.LZMAError as e: # <<<<<<<<<<<<<< * raise DecompressionError(str(e)) from None * */ __Pyx_ErrFetch(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3); __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_lzma); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 143, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = __Pyx_PyObject_GetAttrStr(__pyx_t_7, __pyx_n_s_LZMAError); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 143, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_9 = __Pyx_PyErr_GivenExceptionMatches(__pyx_t_1, __pyx_t_8); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_ErrRestore(__pyx_t_1, __pyx_t_2, __pyx_t_3); __pyx_t_1 = 0; __pyx_t_2 = 0; __pyx_t_3 = 0; if (__pyx_t_9) { __Pyx_AddTraceback("borg.compress.LZMA.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); if (__Pyx_GetException(&__pyx_t_3, &__pyx_t_2, &__pyx_t_1) < 0) __PYX_ERR(0, 143, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_GOTREF(__pyx_t_2); __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_t_2); __pyx_v_e = __pyx_t_2; /* "borg/compress.pyx":144 * return lzma.decompress(data) * except lzma.LZMAError as e: * raise DecompressionError(str(e)) from None # <<<<<<<<<<<<<< * * */ __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_DecompressionError); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 144, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_10 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_v_e); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 144, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_10); __pyx_t_11 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_7))) { __pyx_t_11 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_11)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_11); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); } } __pyx_t_8 = (__pyx_t_11) ? __Pyx_PyObject_Call2Args(__pyx_t_7, __pyx_t_11, __pyx_t_10) : __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_10); __Pyx_XDECREF(__pyx_t_11); __pyx_t_11 = 0; __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 144, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_Raise(__pyx_t_8, 0, 0, Py_None); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __PYX_ERR(0, 144, __pyx_L5_except_error) } goto __pyx_L5_except_error; __pyx_L5_except_error:; /* "borg/compress.pyx":141 * def decompress(self, data): * data = super().decompress(data) * try: # <<<<<<<<<<<<<< * return lzma.decompress(data) * except lzma.LZMAError as e: */ __Pyx_XGIVEREF(__pyx_t_4); __Pyx_XGIVEREF(__pyx_t_5); __Pyx_XGIVEREF(__pyx_t_6); __Pyx_ExceptionReset(__pyx_t_4, __pyx_t_5, __pyx_t_6); goto __pyx_L1_error; __pyx_L7_try_return:; __Pyx_XGIVEREF(__pyx_t_4); __Pyx_XGIVEREF(__pyx_t_5); __Pyx_XGIVEREF(__pyx_t_6); __Pyx_ExceptionReset(__pyx_t_4, __pyx_t_5, __pyx_t_6); goto __pyx_L0; } /* "borg/compress.pyx":139 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * try: */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_XDECREF(__pyx_t_10); __Pyx_XDECREF(__pyx_t_11); __Pyx_AddTraceback("borg.compress.LZMA.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_e); __Pyx_XDECREF(__pyx_v_data); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":156 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4ZLIB_1detect(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4ZLIB_1detect = {"detect", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4ZLIB_1detect, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4ZLIB_1detect(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { CYTHON_UNUSED PyObject *__pyx_v_cls = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("detect (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_cls,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_cls)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("detect", 1, 2, 2, 1); __PYX_ERR(0, 156, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "detect") < 0)) __PYX_ERR(0, 156, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_cls = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("detect", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 156, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.ZLIB.detect", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4ZLIB_detect(__pyx_self, __pyx_v_cls, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4ZLIB_detect(CYTHON_UNUSED PyObject *__pyx_self, CYTHON_UNUSED PyObject *__pyx_v_cls, PyObject *__pyx_v_data) { PyObject *__pyx_v_cmf = NULL; PyObject *__pyx_v_flg = NULL; PyObject *__pyx_v_is_deflate = NULL; PyObject *__pyx_v_check_ok = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *(*__pyx_t_5)(PyObject *); int __pyx_t_6; __Pyx_RefNannySetupContext("detect", 0); /* "borg/compress.pyx":158 * def detect(cls, data): * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] # <<<<<<<<<<<<<< * is_deflate = cmf & 0x0f == 8 * check_ok = (cmf * 256 + flg) % 31 == 0 */ __pyx_t_1 = __Pyx_PyObject_GetSlice(__pyx_v_data, 0, 2, NULL, NULL, &__pyx_slice__4, 0, 1, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 158, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if ((likely(PyTuple_CheckExact(__pyx_t_1))) || (PyList_CheckExact(__pyx_t_1))) { PyObject* sequence = __pyx_t_1; Py_ssize_t size = __Pyx_PySequence_SIZE(sequence); if (unlikely(size != 2)) { if (size > 2) __Pyx_RaiseTooManyValuesError(2); else if (size >= 0) __Pyx_RaiseNeedMoreValuesError(size); __PYX_ERR(0, 158, __pyx_L1_error) } #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS if (likely(PyTuple_CheckExact(sequence))) { __pyx_t_2 = PyTuple_GET_ITEM(sequence, 0); __pyx_t_3 = PyTuple_GET_ITEM(sequence, 1); } else { __pyx_t_2 = PyList_GET_ITEM(sequence, 0); __pyx_t_3 = PyList_GET_ITEM(sequence, 1); } __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(__pyx_t_3); #else __pyx_t_2 = PySequence_ITEM(sequence, 0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 158, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = PySequence_ITEM(sequence, 1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 158, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); #endif __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; } else { Py_ssize_t index = -1; __pyx_t_4 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 158, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_5 = Py_TYPE(__pyx_t_4)->tp_iternext; index = 0; __pyx_t_2 = __pyx_t_5(__pyx_t_4); if (unlikely(!__pyx_t_2)) goto __pyx_L3_unpacking_failed; __Pyx_GOTREF(__pyx_t_2); index = 1; __pyx_t_3 = __pyx_t_5(__pyx_t_4); if (unlikely(!__pyx_t_3)) goto __pyx_L3_unpacking_failed; __Pyx_GOTREF(__pyx_t_3); if (__Pyx_IternextUnpackEndCheck(__pyx_t_5(__pyx_t_4), 2) < 0) __PYX_ERR(0, 158, __pyx_L1_error) __pyx_t_5 = NULL; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; goto __pyx_L4_unpacking_done; __pyx_L3_unpacking_failed:; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_5 = NULL; if (__Pyx_IterFinish() == 0) __Pyx_RaiseNeedMoreValuesError(index); __PYX_ERR(0, 158, __pyx_L1_error) __pyx_L4_unpacking_done:; } __pyx_v_cmf = __pyx_t_2; __pyx_t_2 = 0; __pyx_v_flg = __pyx_t_3; __pyx_t_3 = 0; /* "borg/compress.pyx":159 * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] * is_deflate = cmf & 0x0f == 8 # <<<<<<<<<<<<<< * check_ok = (cmf * 256 + flg) % 31 == 0 * return check_ok and is_deflate */ __pyx_t_1 = __Pyx_PyInt_AndObjC(__pyx_v_cmf, __pyx_int_15, 0x0f, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 159, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __Pyx_PyInt_EqObjC(__pyx_t_1, __pyx_int_8, 8, 0); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 159, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_is_deflate = __pyx_t_3; __pyx_t_3 = 0; /* "borg/compress.pyx":160 * cmf, flg = data[:2] * is_deflate = cmf & 0x0f == 8 * check_ok = (cmf * 256 + flg) % 31 == 0 # <<<<<<<<<<<<<< * return check_ok and is_deflate * */ __pyx_t_3 = PyNumber_Multiply(__pyx_v_cmf, __pyx_int_256); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 160, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_1 = PyNumber_Add(__pyx_t_3, __pyx_v_flg); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 160, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyInt_RemainderObjC(__pyx_t_1, __pyx_int_31, 31, 0); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 160, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyInt_EqObjC(__pyx_t_3, __pyx_int_0, 0, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 160, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_v_check_ok = __pyx_t_1; __pyx_t_1 = 0; /* "borg/compress.pyx":161 * is_deflate = cmf & 0x0f == 8 * check_ok = (cmf * 256 + flg) % 31 == 0 * return check_ok and is_deflate # <<<<<<<<<<<<<< * * def __init__(self, level=6, **kwargs): */ __Pyx_XDECREF(__pyx_r); __pyx_t_6 = __Pyx_PyObject_IsTrue(__pyx_v_check_ok); if (unlikely(__pyx_t_6 < 0)) __PYX_ERR(0, 161, __pyx_L1_error) if (__pyx_t_6) { } else { __Pyx_INCREF(__pyx_v_check_ok); __pyx_t_1 = __pyx_v_check_ok; goto __pyx_L5_bool_binop_done; } __Pyx_INCREF(__pyx_v_is_deflate); __pyx_t_1 = __pyx_v_is_deflate; __pyx_L5_bool_binop_done:; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":156 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.compress.ZLIB.detect", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_cmf); __Pyx_XDECREF(__pyx_v_flg); __Pyx_XDECREF(__pyx_v_is_deflate); __Pyx_XDECREF(__pyx_v_check_ok); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":163 * return check_ok and is_deflate * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4ZLIB_3__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4ZLIB_3__init__ = {"__init__", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4ZLIB_3__init__, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4ZLIB_3__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_level = 0; PyObject *__pyx_v_kwargs = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__ (wrapper)", 0); __pyx_v_kwargs = PyDict_New(); if (unlikely(!__pyx_v_kwargs)) return NULL; __Pyx_GOTREF(__pyx_v_kwargs); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_level,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)((PyObject *)__pyx_int_6)); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_level); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, __pyx_v_kwargs, values, pos_args, "__init__") < 0)) __PYX_ERR(0, 163, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_self = values[0]; __pyx_v_level = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__init__", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 163, __pyx_L3_error) __pyx_L3_error:; __Pyx_DECREF(__pyx_v_kwargs); __pyx_v_kwargs = 0; __Pyx_AddTraceback("borg.compress.ZLIB.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4ZLIB_2__init__(__pyx_self, __pyx_v_self, __pyx_v_level, __pyx_v_kwargs); /* function exit code */ __Pyx_XDECREF(__pyx_v_kwargs); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4ZLIB_2__init__(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_level, PyObject *__pyx_v_kwargs) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannySetupContext("__init__", 0); /* "borg/compress.pyx":164 * * def __init__(self, level=6, **kwargs): * super().__init__(**kwargs) # <<<<<<<<<<<<<< * self.level = level * */ __pyx_t_1 = __Pyx_CyFunction_GetClassObj(__pyx_self); if (!__pyx_t_1) { PyErr_SetString(PyExc_SystemError, "super(): empty __class__ cell"); __PYX_ERR(0, 164, __pyx_L1_error) } __Pyx_INCREF(__pyx_t_1); __pyx_t_2 = PyTuple_New(2); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 164, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_GIVEREF(__pyx_t_1); PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_1); __Pyx_INCREF(__pyx_v_self); __Pyx_GIVEREF(__pyx_v_self); PyTuple_SET_ITEM(__pyx_t_2, 1, __pyx_v_self); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_super, __pyx_t_2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 164, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_init); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 164, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_empty_tuple, __pyx_v_kwargs); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 164, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":165 * def __init__(self, level=6, **kwargs): * super().__init__(**kwargs) * self.level = level # <<<<<<<<<<<<<< * * def compress(self, data): */ if (__Pyx_PyObject_SetAttrStr(__pyx_v_self, __pyx_n_s_level, __pyx_v_level) < 0) __PYX_ERR(0, 165, __pyx_L1_error) /* "borg/compress.pyx":163 * return check_ok and is_deflate * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.compress.ZLIB.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":167 * self.level = level * * def compress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not add ID bytes * return zlib.compress(data, self.level) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4ZLIB_5compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4ZLIB_5compress = {"compress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4ZLIB_5compress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4ZLIB_5compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("compress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, 1); __PYX_ERR(0, 167, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "compress") < 0)) __PYX_ERR(0, 167, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 167, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.ZLIB.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4ZLIB_4compress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4ZLIB_4compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; int __pyx_t_5; PyObject *__pyx_t_6 = NULL; __Pyx_RefNannySetupContext("compress", 0); /* "borg/compress.pyx":169 * def compress(self, data): * # note: for compatibility no super call, do not add ID bytes * return zlib.compress(data, self.level) # <<<<<<<<<<<<<< * * def decompress(self, data): */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_zlib); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_compress); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_self, __pyx_n_s_level); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; __pyx_t_5 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_5 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_4, __pyx_v_data, __pyx_t_2}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_5, 2+__pyx_t_5); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_4, __pyx_v_data, __pyx_t_2}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_5, 2+__pyx_t_5); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; } else #endif { __pyx_t_6 = PyTuple_New(2+__pyx_t_5); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (__pyx_t_4) { __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_4); __pyx_t_4 = NULL; } __Pyx_INCREF(__pyx_v_data); __Pyx_GIVEREF(__pyx_v_data); PyTuple_SET_ITEM(__pyx_t_6, 0+__pyx_t_5, __pyx_v_data); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_6, 1+__pyx_t_5, __pyx_t_2); __pyx_t_2 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_6, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 169, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":167 * self.level = level * * def compress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not add ID bytes * return zlib.compress(data, self.level) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.compress.ZLIB.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":171 * return zlib.compress(data, self.level) * * def decompress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not strip ID bytes * try: */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_4ZLIB_7decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_4ZLIB_7decompress = {"decompress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_4ZLIB_7decompress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_4ZLIB_7decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { CYTHON_UNUSED PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decompress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, 1); __PYX_ERR(0, 171, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "decompress") < 0)) __PYX_ERR(0, 171, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 171, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.ZLIB.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_4ZLIB_6decompress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_4ZLIB_6decompress(CYTHON_UNUSED PyObject *__pyx_self, CYTHON_UNUSED PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_v_e = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; int __pyx_t_9; PyObject *__pyx_t_10 = NULL; PyObject *__pyx_t_11 = NULL; __Pyx_RefNannySetupContext("decompress", 0); /* "borg/compress.pyx":173 * def decompress(self, data): * # note: for compatibility no super call, do not strip ID bytes * try: # <<<<<<<<<<<<<< * return zlib.decompress(data) * except zlib.error as e: */ { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3); __Pyx_XGOTREF(__pyx_t_1); __Pyx_XGOTREF(__pyx_t_2); __Pyx_XGOTREF(__pyx_t_3); /*try:*/ { /* "borg/compress.pyx":174 * # note: for compatibility no super call, do not strip ID bytes * try: * return zlib.decompress(data) # <<<<<<<<<<<<<< * except zlib.error as e: * raise DecompressionError(str(e)) from None */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_zlib); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 174, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_t_5, __pyx_n_s_decompress); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 174, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_6))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); } } __pyx_t_4 = (__pyx_t_5) ? __Pyx_PyObject_Call2Args(__pyx_t_6, __pyx_t_5, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_6, __pyx_v_data); __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 174, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L7_try_return; /* "borg/compress.pyx":173 * def decompress(self, data): * # note: for compatibility no super call, do not strip ID bytes * try: # <<<<<<<<<<<<<< * return zlib.decompress(data) * except zlib.error as e: */ } __pyx_L3_error:; __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/compress.pyx":175 * try: * return zlib.decompress(data) * except zlib.error as e: # <<<<<<<<<<<<<< * raise DecompressionError(str(e)) from None * */ __Pyx_ErrFetch(&__pyx_t_4, &__pyx_t_6, &__pyx_t_5); __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_zlib); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 175, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = __Pyx_PyObject_GetAttrStr(__pyx_t_7, __pyx_n_s_error); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 175, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_9 = __Pyx_PyErr_GivenExceptionMatches(__pyx_t_4, __pyx_t_8); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_ErrRestore(__pyx_t_4, __pyx_t_6, __pyx_t_5); __pyx_t_4 = 0; __pyx_t_6 = 0; __pyx_t_5 = 0; if (__pyx_t_9) { __Pyx_AddTraceback("borg.compress.ZLIB.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); if (__Pyx_GetException(&__pyx_t_5, &__pyx_t_6, &__pyx_t_4) < 0) __PYX_ERR(0, 175, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_GOTREF(__pyx_t_6); __Pyx_GOTREF(__pyx_t_4); __Pyx_INCREF(__pyx_t_6); __pyx_v_e = __pyx_t_6; /* "borg/compress.pyx":176 * return zlib.decompress(data) * except zlib.error as e: * raise DecompressionError(str(e)) from None # <<<<<<<<<<<<<< * * */ __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_DecompressionError); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 176, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_10 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_v_e); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 176, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_10); __pyx_t_11 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_7))) { __pyx_t_11 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_11)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_11); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); } } __pyx_t_8 = (__pyx_t_11) ? __Pyx_PyObject_Call2Args(__pyx_t_7, __pyx_t_11, __pyx_t_10) : __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_10); __Pyx_XDECREF(__pyx_t_11); __pyx_t_11 = 0; __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 176, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_Raise(__pyx_t_8, 0, 0, Py_None); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __PYX_ERR(0, 176, __pyx_L5_except_error) } goto __pyx_L5_except_error; __pyx_L5_except_error:; /* "borg/compress.pyx":173 * def decompress(self, data): * # note: for compatibility no super call, do not strip ID bytes * try: # <<<<<<<<<<<<<< * return zlib.decompress(data) * except zlib.error as e: */ __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L1_error; __pyx_L7_try_return:; __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L0; } /* "borg/compress.pyx":171 * return zlib.compress(data, self.level) * * def decompress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not strip ID bytes * try: */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_XDECREF(__pyx_t_10); __Pyx_XDECREF(__pyx_t_11); __Pyx_AddTraceback("borg.compress.ZLIB.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_e); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":187 * COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first * * def get_compressor(name, **kwargs): # <<<<<<<<<<<<<< * cls = COMPRESSOR_TABLE[name] * return cls(**kwargs) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_1get_compressor(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_1get_compressor = {"get_compressor", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_1get_compressor, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_1get_compressor(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_name = 0; PyObject *__pyx_v_kwargs = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("get_compressor (wrapper)", 0); __pyx_v_kwargs = PyDict_New(); if (unlikely(!__pyx_v_kwargs)) return NULL; __Pyx_GOTREF(__pyx_v_kwargs); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_name,0}; PyObject* values[1] = {0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_name)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, __pyx_v_kwargs, values, pos_args, "get_compressor") < 0)) __PYX_ERR(0, 187, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 1) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); } __pyx_v_name = values[0]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("get_compressor", 1, 1, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 187, __pyx_L3_error) __pyx_L3_error:; __Pyx_DECREF(__pyx_v_kwargs); __pyx_v_kwargs = 0; __Pyx_AddTraceback("borg.compress.get_compressor", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_get_compressor(__pyx_self, __pyx_v_name, __pyx_v_kwargs); /* function exit code */ __Pyx_XDECREF(__pyx_v_kwargs); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_get_compressor(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_name, PyObject *__pyx_v_kwargs) { PyObject *__pyx_v_cls = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannySetupContext("get_compressor", 0); /* "borg/compress.pyx":188 * * def get_compressor(name, **kwargs): * cls = COMPRESSOR_TABLE[name] # <<<<<<<<<<<<<< * return cls(**kwargs) * */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_COMPRESSOR_TABLE); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 188, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_GetItem(__pyx_t_1, __pyx_v_name); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 188, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_cls = __pyx_t_2; __pyx_t_2 = 0; /* "borg/compress.pyx":189 * def get_compressor(name, **kwargs): * cls = COMPRESSOR_TABLE[name] * return cls(**kwargs) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyObject_Call(__pyx_v_cls, __pyx_empty_tuple, __pyx_v_kwargs); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 189, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_r = __pyx_t_2; __pyx_t_2 = 0; goto __pyx_L0; /* "borg/compress.pyx":187 * COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first * * def get_compressor(name, **kwargs): # <<<<<<<<<<<<<< * cls = COMPRESSOR_TABLE[name] * return cls(**kwargs) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.compress.get_compressor", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_cls); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":197 * decompresses everything we can handle (autodetect) * """ * def __init__(self, name='null', **kwargs): # <<<<<<<<<<<<<< * self.params = kwargs * self.compressor = get_compressor(name, **self.params) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_10Compressor_1__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_10Compressor_1__init__ = {"__init__", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_10Compressor_1__init__, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_10Compressor_1__init__(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_name = 0; PyObject *__pyx_v_kwargs = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__init__ (wrapper)", 0); __pyx_v_kwargs = PyDict_New(); if (unlikely(!__pyx_v_kwargs)) return NULL; __Pyx_GOTREF(__pyx_v_kwargs); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_name,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)((PyObject*)__pyx_n_s_null)); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_name); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, __pyx_v_kwargs, values, pos_args, "__init__") < 0)) __PYX_ERR(0, 197, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_self = values[0]; __pyx_v_name = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__init__", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 197, __pyx_L3_error) __pyx_L3_error:; __Pyx_DECREF(__pyx_v_kwargs); __pyx_v_kwargs = 0; __Pyx_AddTraceback("borg.compress.Compressor.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_10Compressor___init__(__pyx_self, __pyx_v_self, __pyx_v_name, __pyx_v_kwargs); /* function exit code */ __Pyx_XDECREF(__pyx_v_kwargs); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_10Compressor___init__(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_name, PyObject *__pyx_v_kwargs) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("__init__", 0); /* "borg/compress.pyx":198 * """ * def __init__(self, name='null', **kwargs): * self.params = kwargs # <<<<<<<<<<<<<< * self.compressor = get_compressor(name, **self.params) * */ if (__Pyx_PyObject_SetAttrStr(__pyx_v_self, __pyx_n_s_params, __pyx_v_kwargs) < 0) __PYX_ERR(0, 198, __pyx_L1_error) /* "borg/compress.pyx":199 * def __init__(self, name='null', **kwargs): * self.params = kwargs * self.compressor = get_compressor(name, **self.params) # <<<<<<<<<<<<<< * * def compress(self, data): */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_get_compressor); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyTuple_New(1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_v_name); __pyx_t_4 = __Pyx_PyObject_GetAttrStr(__pyx_v_self, __pyx_n_s_params); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); if (unlikely(__pyx_t_4 == Py_None)) { PyErr_SetString(PyExc_TypeError, "argument after ** must be a mapping, not NoneType"); __PYX_ERR(0, 199, __pyx_L1_error) } if (likely(PyDict_CheckExact(__pyx_t_4))) { __pyx_t_3 = PyDict_Copy(__pyx_t_4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; } else { __pyx_t_3 = PyObject_CallFunctionObjArgs((PyObject*)&PyDict_Type, __pyx_t_4, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; } __pyx_t_4 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_t_2, __pyx_t_3); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (__Pyx_PyObject_SetAttrStr(__pyx_v_self, __pyx_n_s_compressor, __pyx_t_4) < 0) __PYX_ERR(0, 199, __pyx_L1_error) __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/compress.pyx":197 * decompresses everything we can handle (autodetect) * """ * def __init__(self, name='null', **kwargs): # <<<<<<<<<<<<<< * self.params = kwargs * self.compressor = get_compressor(name, **self.params) */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.compress.Compressor.__init__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":201 * self.compressor = get_compressor(name, **self.params) * * def compress(self, data): # <<<<<<<<<<<<<< * return self.compressor.compress(data) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_10Compressor_3compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_10Compressor_3compress = {"compress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_10Compressor_3compress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_10Compressor_3compress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("compress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, 1); __PYX_ERR(0, 201, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "compress") < 0)) __PYX_ERR(0, 201, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("compress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 201, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.Compressor.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_10Compressor_2compress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_10Compressor_2compress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; __Pyx_RefNannySetupContext("compress", 0); /* "borg/compress.pyx":202 * * def compress(self, data): * return self.compressor.compress(data) # <<<<<<<<<<<<<< * * def decompress(self, data): */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_self, __pyx_n_s_compressor); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_compress); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_data); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":201 * self.compressor = get_compressor(name, **self.params) * * def compress(self, data): # <<<<<<<<<<<<<< * return self.compressor.compress(data) * */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.compress.Compressor.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/compress.pyx":204 * return self.compressor.compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_10Compressor_5decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_10Compressor_5decompress = {"decompress", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_10Compressor_5decompress, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_10Compressor_5decompress(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_self = 0; PyObject *__pyx_v_data = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decompress (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_self,&__pyx_n_s_data,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_self)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_data)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, 1); __PYX_ERR(0, 204, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "decompress") < 0)) __PYX_ERR(0, 204, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_self = values[0]; __pyx_v_data = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("decompress", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 204, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.Compressor.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_10Compressor_4decompress(__pyx_self, __pyx_v_self, __pyx_v_data); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_10Compressor_4decompress(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_v_hdr = NULL; PyObject *__pyx_v_cls = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; Py_ssize_t __pyx_t_3; PyObject *(*__pyx_t_4)(PyObject *); PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; int __pyx_t_7; __Pyx_RefNannySetupContext("decompress", 0); /* "borg/compress.pyx":205 * * def decompress(self, data): * hdr = bytes(data[:2]) # detect() does not work with memoryview # <<<<<<<<<<<<<< * for cls in COMPRESSOR_LIST: * if cls.detect(hdr): */ __pyx_t_1 = __Pyx_PyObject_GetSlice(__pyx_v_data, 0, 2, NULL, NULL, &__pyx_slice__4, 0, 1, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 205, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyBytes_Type)), __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 205, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_hdr = ((PyObject*)__pyx_t_2); __pyx_t_2 = 0; /* "borg/compress.pyx":206 * def decompress(self, data): * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: # <<<<<<<<<<<<<< * if cls.detect(hdr): * return cls(**self.params).decompress(data) */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_COMPRESSOR_LIST); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 206, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (likely(PyList_CheckExact(__pyx_t_2)) || PyTuple_CheckExact(__pyx_t_2)) { __pyx_t_1 = __pyx_t_2; __Pyx_INCREF(__pyx_t_1); __pyx_t_3 = 0; __pyx_t_4 = NULL; } else { __pyx_t_3 = -1; __pyx_t_1 = PyObject_GetIter(__pyx_t_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 206, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_4 = Py_TYPE(__pyx_t_1)->tp_iternext; if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 206, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; for (;;) { if (likely(!__pyx_t_4)) { if (likely(PyList_CheckExact(__pyx_t_1))) { if (__pyx_t_3 >= PyList_GET_SIZE(__pyx_t_1)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_2 = PyList_GET_ITEM(__pyx_t_1, __pyx_t_3); __Pyx_INCREF(__pyx_t_2); __pyx_t_3++; if (unlikely(0 < 0)) __PYX_ERR(0, 206, __pyx_L1_error) #else __pyx_t_2 = PySequence_ITEM(__pyx_t_1, __pyx_t_3); __pyx_t_3++; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 206, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); #endif } else { if (__pyx_t_3 >= PyTuple_GET_SIZE(__pyx_t_1)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_2 = PyTuple_GET_ITEM(__pyx_t_1, __pyx_t_3); __Pyx_INCREF(__pyx_t_2); __pyx_t_3++; if (unlikely(0 < 0)) __PYX_ERR(0, 206, __pyx_L1_error) #else __pyx_t_2 = PySequence_ITEM(__pyx_t_1, __pyx_t_3); __pyx_t_3++; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 206, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); #endif } } else { __pyx_t_2 = __pyx_t_4(__pyx_t_1); if (unlikely(!__pyx_t_2)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 206, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_2); } __Pyx_XDECREF_SET(__pyx_v_cls, __pyx_t_2); __pyx_t_2 = 0; /* "borg/compress.pyx":207 * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: * if cls.detect(hdr): # <<<<<<<<<<<<<< * return cls(**self.params).decompress(data) * else: */ __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_v_cls, __pyx_n_s_detect); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 207, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_5))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_2 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_6, __pyx_v_hdr) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_hdr); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 207, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_7 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_7 < 0)) __PYX_ERR(0, 207, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (__pyx_t_7) { /* "borg/compress.pyx":208 * for cls in COMPRESSOR_LIST: * if cls.detect(hdr): * return cls(**self.params).decompress(data) # <<<<<<<<<<<<<< * else: * raise ValueError('No decompressor for this data found: %r.', data[:2]) */ __Pyx_XDECREF(__pyx_r); __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_v_self, __pyx_n_s_params); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 208, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (unlikely(__pyx_t_6 == Py_None)) { PyErr_SetString(PyExc_TypeError, "argument after ** must be a mapping, not NoneType"); __PYX_ERR(0, 208, __pyx_L1_error) } if (likely(PyDict_CheckExact(__pyx_t_6))) { __pyx_t_5 = PyDict_Copy(__pyx_t_6); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 208, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } else { __pyx_t_5 = PyObject_CallFunctionObjArgs((PyObject*)&PyDict_Type, __pyx_t_6, NULL); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 208, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __pyx_t_6 = __Pyx_PyObject_Call(__pyx_v_cls, __pyx_empty_tuple, __pyx_t_5); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 208, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_t_6, __pyx_n_s_decompress); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 208, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_5))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_2 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_6, __pyx_v_data) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_data); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 208, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_r = __pyx_t_2; __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; goto __pyx_L0; /* "borg/compress.pyx":207 * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: * if cls.detect(hdr): # <<<<<<<<<<<<<< * return cls(**self.params).decompress(data) * else: */ } /* "borg/compress.pyx":206 * def decompress(self, data): * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: # <<<<<<<<<<<<<< * if cls.detect(hdr): * return cls(**self.params).decompress(data) */ } /*else*/ { /* "borg/compress.pyx":210 * return cls(**self.params).decompress(data) * else: * raise ValueError('No decompressor for this data found: %r.', data[:2]) # <<<<<<<<<<<<<< */ __pyx_t_2 = __Pyx_PyObject_GetSlice(__pyx_v_data, 0, 2, NULL, NULL, &__pyx_slice__4, 0, 1, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 210, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_5 = PyTuple_New(2); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 210, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_INCREF(__pyx_kp_s_No_decompressor_for_this_data_fo); __Pyx_GIVEREF(__pyx_kp_s_No_decompressor_for_this_data_fo); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_kp_s_No_decompressor_for_this_data_fo); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_builtin_ValueError, __pyx_t_5, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 210, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_Raise(__pyx_t_2, 0, 0, 0); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __PYX_ERR(0, 210, __pyx_L1_error) } /* "borg/compress.pyx":206 * def decompress(self, data): * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: # <<<<<<<<<<<<<< * if cls.detect(hdr): * return cls(**self.params).decompress(data) */ __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":204 * return self.compressor.compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.compress.Compressor.decompress", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_hdr); __Pyx_XDECREF(__pyx_v_cls); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): # <<<<<<<<<<<<<< * if __pyx_checksum != 0xd41d8cd: * from pickle import PickleError as __pyx_PickleError */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_8compress_3__pyx_unpickle_CompressorBase(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_8compress_3__pyx_unpickle_CompressorBase = {"__pyx_unpickle_CompressorBase", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_8compress_3__pyx_unpickle_CompressorBase, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_8compress_3__pyx_unpickle_CompressorBase(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v___pyx_type = 0; long __pyx_v___pyx_checksum; PyObject *__pyx_v___pyx_state = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__pyx_unpickle_CompressorBase (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_pyx_type,&__pyx_n_s_pyx_checksum,&__pyx_n_s_pyx_state,0}; PyObject* values[3] = {0,0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_pyx_type)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_pyx_checksum)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__pyx_unpickle_CompressorBase", 1, 3, 3, 1); __PYX_ERR(1, 1, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_pyx_state)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__pyx_unpickle_CompressorBase", 1, 3, 3, 2); __PYX_ERR(1, 1, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__pyx_unpickle_CompressorBase") < 0)) __PYX_ERR(1, 1, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 3) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[2] = PyTuple_GET_ITEM(__pyx_args, 2); } __pyx_v___pyx_type = values[0]; __pyx_v___pyx_checksum = __Pyx_PyInt_As_long(values[1]); if (unlikely((__pyx_v___pyx_checksum == (long)-1) && PyErr_Occurred())) __PYX_ERR(1, 1, __pyx_L3_error) __pyx_v___pyx_state = values[2]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__pyx_unpickle_CompressorBase", 1, 3, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(1, 1, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.compress.__pyx_unpickle_CompressorBase", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_8compress_2__pyx_unpickle_CompressorBase(__pyx_self, __pyx_v___pyx_type, __pyx_v___pyx_checksum, __pyx_v___pyx_state); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_8compress_2__pyx_unpickle_CompressorBase(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v___pyx_type, long __pyx_v___pyx_checksum, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_v___pyx_PickleError = NULL; PyObject *__pyx_v___pyx_result = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; int __pyx_t_6; __Pyx_RefNannySetupContext("__pyx_unpickle_CompressorBase", 0); /* "(tree fragment)":2 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): * if __pyx_checksum != 0xd41d8cd: # <<<<<<<<<<<<<< * from pickle import PickleError as __pyx_PickleError * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) */ __pyx_t_1 = ((__pyx_v___pyx_checksum != 0xd41d8cd) != 0); if (__pyx_t_1) { /* "(tree fragment)":3 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): * if __pyx_checksum != 0xd41d8cd: * from pickle import PickleError as __pyx_PickleError # <<<<<<<<<<<<<< * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) * __pyx_result = CompressorBase.__new__(__pyx_type) */ __pyx_t_2 = PyList_New(1); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 3, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_INCREF(__pyx_n_s_PickleError); __Pyx_GIVEREF(__pyx_n_s_PickleError); PyList_SET_ITEM(__pyx_t_2, 0, __pyx_n_s_PickleError); __pyx_t_3 = __Pyx_Import(__pyx_n_s_pickle, __pyx_t_2, -1); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 3, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_3, __pyx_n_s_PickleError); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 3, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_INCREF(__pyx_t_2); __pyx_v___pyx_PickleError = __pyx_t_2; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "(tree fragment)":4 * if __pyx_checksum != 0xd41d8cd: * from pickle import PickleError as __pyx_PickleError * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) # <<<<<<<<<<<<<< * __pyx_result = CompressorBase.__new__(__pyx_type) * if __pyx_state is not None: */ __pyx_t_2 = __Pyx_PyInt_From_long(__pyx_v___pyx_checksum); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = __Pyx_PyString_Format(__pyx_kp_s_Incompatible_checksums_s_vs_0xd4, __pyx_t_2); if (unlikely(!__pyx_t_4)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_INCREF(__pyx_v___pyx_PickleError); __pyx_t_2 = __pyx_v___pyx_PickleError; __pyx_t_5 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_3 = (__pyx_t_5) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_5, __pyx_t_4) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_Raise(__pyx_t_3, 0, 0, 0); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":2 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): * if __pyx_checksum != 0xd41d8cd: # <<<<<<<<<<<<<< * from pickle import PickleError as __pyx_PickleError * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) */ } /* "(tree fragment)":5 * from pickle import PickleError as __pyx_PickleError * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) * __pyx_result = CompressorBase.__new__(__pyx_type) # <<<<<<<<<<<<<< * if __pyx_state is not None: * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase), __pyx_n_s_new); if (unlikely(!__pyx_t_2)) __PYX_ERR(1, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_3 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_v___pyx_type) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_v___pyx_type); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v___pyx_result = __pyx_t_3; __pyx_t_3 = 0; /* "(tree fragment)":6 * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) * __pyx_result = CompressorBase.__new__(__pyx_type) * if __pyx_state is not None: # <<<<<<<<<<<<<< * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) * return __pyx_result */ __pyx_t_1 = (__pyx_v___pyx_state != Py_None); __pyx_t_6 = (__pyx_t_1 != 0); if (__pyx_t_6) { /* "(tree fragment)":7 * __pyx_result = CompressorBase.__new__(__pyx_type) * if __pyx_state is not None: * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) # <<<<<<<<<<<<<< * return __pyx_result * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): */ if (!(likely(PyTuple_CheckExact(__pyx_v___pyx_state))||((__pyx_v___pyx_state) == Py_None)||(PyErr_Format(PyExc_TypeError, "Expected %.16s, got %.200s", "tuple", Py_TYPE(__pyx_v___pyx_state)->tp_name), 0))) __PYX_ERR(1, 7, __pyx_L1_error) __pyx_t_3 = __pyx_f_4borg_8compress___pyx_unpickle_CompressorBase__set_state(((struct __pyx_obj_4borg_8compress_CompressorBase *)__pyx_v___pyx_result), ((PyObject*)__pyx_v___pyx_state)); if (unlikely(!__pyx_t_3)) __PYX_ERR(1, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "(tree fragment)":6 * raise __pyx_PickleError("Incompatible checksums (%s vs 0xd41d8cd = ())" % __pyx_checksum) * __pyx_result = CompressorBase.__new__(__pyx_type) * if __pyx_state is not None: # <<<<<<<<<<<<<< * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) * return __pyx_result */ } /* "(tree fragment)":8 * if __pyx_state is not None: * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) * return __pyx_result # <<<<<<<<<<<<<< * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): * if len(__pyx_state) > 0 and hasattr(__pyx_result, '__dict__'): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(__pyx_v___pyx_result); __pyx_r = __pyx_v___pyx_result; goto __pyx_L0; /* "(tree fragment)":1 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): # <<<<<<<<<<<<<< * if __pyx_checksum != 0xd41d8cd: * from pickle import PickleError as __pyx_PickleError */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.compress.__pyx_unpickle_CompressorBase", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v___pyx_PickleError); __Pyx_XDECREF(__pyx_v___pyx_result); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":9 * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) * return __pyx_result * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): # <<<<<<<<<<<<<< * if len(__pyx_state) > 0 and hasattr(__pyx_result, '__dict__'): * __pyx_result.__dict__.update(__pyx_state[0]) */ static PyObject *__pyx_f_4borg_8compress___pyx_unpickle_CompressorBase__set_state(struct __pyx_obj_4borg_8compress_CompressorBase *__pyx_v___pyx_result, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; Py_ssize_t __pyx_t_2; int __pyx_t_3; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; __Pyx_RefNannySetupContext("__pyx_unpickle_CompressorBase__set_state", 0); /* "(tree fragment)":10 * return __pyx_result * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): * if len(__pyx_state) > 0 and hasattr(__pyx_result, '__dict__'): # <<<<<<<<<<<<<< * __pyx_result.__dict__.update(__pyx_state[0]) */ if (unlikely(__pyx_v___pyx_state == Py_None)) { PyErr_SetString(PyExc_TypeError, "object of type 'NoneType' has no len()"); __PYX_ERR(1, 10, __pyx_L1_error) } __pyx_t_2 = PyTuple_GET_SIZE(__pyx_v___pyx_state); if (unlikely(__pyx_t_2 == ((Py_ssize_t)-1))) __PYX_ERR(1, 10, __pyx_L1_error) __pyx_t_3 = ((__pyx_t_2 > 0) != 0); if (__pyx_t_3) { } else { __pyx_t_1 = __pyx_t_3; goto __pyx_L4_bool_binop_done; } __pyx_t_3 = __Pyx_HasAttr(((PyObject *)__pyx_v___pyx_result), __pyx_n_s_dict); if (unlikely(__pyx_t_3 == ((int)-1))) __PYX_ERR(1, 10, __pyx_L1_error) __pyx_t_4 = (__pyx_t_3 != 0); __pyx_t_1 = __pyx_t_4; __pyx_L4_bool_binop_done:; if (__pyx_t_1) { /* "(tree fragment)":11 * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): * if len(__pyx_state) > 0 and hasattr(__pyx_result, '__dict__'): * __pyx_result.__dict__.update(__pyx_state[0]) # <<<<<<<<<<<<<< */ __pyx_t_6 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v___pyx_result), __pyx_n_s_dict); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_6, __pyx_n_s_update); if (unlikely(!__pyx_t_7)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(__pyx_v___pyx_state == Py_None)) { PyErr_SetString(PyExc_TypeError, "'NoneType' object is not subscriptable"); __PYX_ERR(1, 11, __pyx_L1_error) } __pyx_t_6 = __Pyx_GetItemInt_Tuple(__pyx_v___pyx_state, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_7))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); } } __pyx_t_5 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_7, __pyx_t_8, __pyx_t_6) : __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_6); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(1, 11, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; /* "(tree fragment)":10 * return __pyx_result * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): * if len(__pyx_state) > 0 and hasattr(__pyx_result, '__dict__'): # <<<<<<<<<<<<<< * __pyx_result.__dict__.update(__pyx_state[0]) */ } /* "(tree fragment)":9 * __pyx_unpickle_CompressorBase__set_state( __pyx_result, __pyx_state) * return __pyx_result * cdef __pyx_unpickle_CompressorBase__set_state(CompressorBase __pyx_result, tuple __pyx_state): # <<<<<<<<<<<<<< * if len(__pyx_state) > 0 and hasattr(__pyx_result, '__dict__'): * __pyx_result.__dict__.update(__pyx_state[0]) */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.compress.__pyx_unpickle_CompressorBase__set_state", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_tp_new_4borg_8compress_CompressorBase(PyTypeObject *t, CYTHON_UNUSED PyObject *a, CYTHON_UNUSED PyObject *k) { PyObject *o; if (likely((t->tp_flags & Py_TPFLAGS_IS_ABSTRACT) == 0)) { o = (*t->tp_alloc)(t, 0); } else { o = (PyObject *) PyBaseObject_Type.tp_new(t, __pyx_empty_tuple, 0); } if (unlikely(!o)) return 0; return o; } static void __pyx_tp_dealloc_4borg_8compress_CompressorBase(PyObject *o) { #if CYTHON_USE_TP_FINALIZE if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && (!PyType_IS_GC(Py_TYPE(o)) || !_PyGC_FINALIZED(o))) { if (PyObject_CallFinalizerFromDealloc(o)) return; } #endif (*Py_TYPE(o)->tp_free)(o); } static PyMethodDef __pyx_methods_4borg_8compress_CompressorBase[] = { {"detect", (PyCFunction)__pyx_pw_4borg_8compress_14CompressorBase_1detect, METH_O, 0}, {"compress", (PyCFunction)__pyx_pw_4borg_8compress_14CompressorBase_5compress, METH_O, 0}, {"decompress", (PyCFunction)__pyx_pw_4borg_8compress_14CompressorBase_7decompress, METH_O, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_8compress_14CompressorBase_9__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_8compress_14CompressorBase_11__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PyTypeObject __pyx_type_4borg_8compress_CompressorBase = { PyVarObject_HEAD_INIT(0, 0) "borg.compress.CompressorBase", /*tp_name*/ sizeof(struct __pyx_obj_4borg_8compress_CompressorBase), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_8compress_CompressorBase, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE, /*tp_flags*/ "\n base class for all (de)compression classes,\n also handles compression format auto detection and\n adding/stripping the ID header (which enable auto detection).\n ", /*tp_doc*/ 0, /*tp_traverse*/ 0, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ 0, /*tp_iter*/ 0, /*tp_iternext*/ __pyx_methods_4borg_8compress_CompressorBase, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ __pyx_pw_4borg_8compress_14CompressorBase_3__init__, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_8compress_CompressorBase, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_compress(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_compress}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "compress", 0, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_n_s_Buffer, __pyx_k_Buffer, sizeof(__pyx_k_Buffer), 0, 0, 1, 1}, {&__pyx_n_s_CHECK_NONE, __pyx_k_CHECK_NONE, sizeof(__pyx_k_CHECK_NONE), 0, 0, 1, 1}, {&__pyx_n_s_CNONE, __pyx_k_CNONE, sizeof(__pyx_k_CNONE), 0, 0, 1, 1}, {&__pyx_n_s_CNONE_compress, __pyx_k_CNONE_compress, sizeof(__pyx_k_CNONE_compress), 0, 0, 1, 1}, {&__pyx_n_s_CNONE_decompress, __pyx_k_CNONE_decompress, sizeof(__pyx_k_CNONE_decompress), 0, 0, 1, 1}, {&__pyx_n_s_COMPRESSOR_LIST, __pyx_k_COMPRESSOR_LIST, sizeof(__pyx_k_COMPRESSOR_LIST), 0, 0, 1, 1}, {&__pyx_n_s_COMPRESSOR_TABLE, __pyx_k_COMPRESSOR_TABLE, sizeof(__pyx_k_COMPRESSOR_TABLE), 0, 0, 1, 1}, {&__pyx_n_s_Compressor, __pyx_k_Compressor, sizeof(__pyx_k_Compressor), 0, 0, 1, 1}, {&__pyx_n_s_CompressorBase, __pyx_k_CompressorBase, sizeof(__pyx_k_CompressorBase), 0, 0, 1, 1}, {&__pyx_n_s_Compressor___init, __pyx_k_Compressor___init, sizeof(__pyx_k_Compressor___init), 0, 0, 1, 1}, {&__pyx_n_s_Compressor_compress, __pyx_k_Compressor_compress, sizeof(__pyx_k_Compressor_compress), 0, 0, 1, 1}, {&__pyx_n_s_Compressor_decompress, __pyx_k_Compressor_decompress, sizeof(__pyx_k_Compressor_decompress), 0, 0, 1, 1}, {&__pyx_n_s_DecompressionError, __pyx_k_DecompressionError, sizeof(__pyx_k_DecompressionError), 0, 0, 1, 1}, {&__pyx_n_s_ID, __pyx_k_ID, sizeof(__pyx_k_ID), 0, 0, 1, 1}, {&__pyx_n_s_ImportError, __pyx_k_ImportError, sizeof(__pyx_k_ImportError), 0, 0, 1, 1}, {&__pyx_kp_s_Incompatible_checksums_s_vs_0xd4, __pyx_k_Incompatible_checksums_s_vs_0xd4, sizeof(__pyx_k_Incompatible_checksums_s_vs_0xd4), 0, 0, 1, 0}, {&__pyx_n_s_LZ4, __pyx_k_LZ4, sizeof(__pyx_k_LZ4), 0, 0, 1, 1}, {&__pyx_n_s_LZ4___init, __pyx_k_LZ4___init, sizeof(__pyx_k_LZ4___init), 0, 0, 1, 1}, {&__pyx_n_s_LZ4_compress, __pyx_k_LZ4_compress, sizeof(__pyx_k_LZ4_compress), 0, 0, 1, 1}, {&__pyx_n_s_LZ4_decompress, __pyx_k_LZ4_decompress, sizeof(__pyx_k_LZ4_decompress), 0, 0, 1, 1}, {&__pyx_n_s_LZMA, __pyx_k_LZMA, sizeof(__pyx_k_LZMA), 0, 0, 1, 1}, {&__pyx_n_s_LZMAError, __pyx_k_LZMAError, sizeof(__pyx_k_LZMAError), 0, 0, 1, 1}, {&__pyx_n_s_LZMA___init, __pyx_k_LZMA___init, sizeof(__pyx_k_LZMA___init), 0, 0, 1, 1}, {&__pyx_n_s_LZMA_compress, __pyx_k_LZMA_compress, sizeof(__pyx_k_LZMA_compress), 0, 0, 1, 1}, {&__pyx_n_s_LZMA_decompress, __pyx_k_LZMA_decompress, sizeof(__pyx_k_LZMA_decompress), 0, 0, 1, 1}, {&__pyx_kp_s_No_decompressor_for_this_data_fo, __pyx_k_No_decompressor_for_this_data_fo, sizeof(__pyx_k_No_decompressor_for_this_data_fo), 0, 0, 1, 0}, {&__pyx_kp_s_No_lzma_support_found, __pyx_k_No_lzma_support_found, sizeof(__pyx_k_No_lzma_support_found), 0, 0, 1, 0}, {&__pyx_n_s_PickleError, __pyx_k_PickleError, sizeof(__pyx_k_PickleError), 0, 0, 1, 1}, {&__pyx_n_s_ValueError, __pyx_k_ValueError, sizeof(__pyx_k_ValueError), 0, 0, 1, 1}, {&__pyx_n_s_ZLIB, __pyx_k_ZLIB, sizeof(__pyx_k_ZLIB), 0, 0, 1, 1}, {&__pyx_n_s_ZLIB___init, __pyx_k_ZLIB___init, sizeof(__pyx_k_ZLIB___init), 0, 0, 1, 1}, {&__pyx_n_s_ZLIB_compress, __pyx_k_ZLIB_compress, sizeof(__pyx_k_ZLIB_compress), 0, 0, 1, 1}, {&__pyx_n_s_ZLIB_decompress, __pyx_k_ZLIB_decompress, sizeof(__pyx_k_ZLIB_decompress), 0, 0, 1, 1}, {&__pyx_n_s_ZLIB_detect, __pyx_k_ZLIB_detect, sizeof(__pyx_k_ZLIB_detect), 0, 0, 1, 1}, {&__pyx_kp_b__11, __pyx_k__11, sizeof(__pyx_k__11), 0, 0, 0, 0}, {&__pyx_kp_b__18, __pyx_k__18, sizeof(__pyx_k__18), 0, 0, 0, 0}, {&__pyx_kp_b__26, __pyx_k__26, sizeof(__pyx_k__26), 0, 0, 0, 0}, {&__pyx_kp_b__5, __pyx_k__5, sizeof(__pyx_k__5), 0, 0, 0, 0}, {&__pyx_kp_b__6, __pyx_k__6, sizeof(__pyx_k__6), 0, 0, 0, 0}, {&__pyx_n_s_baseclass, __pyx_k_baseclass, sizeof(__pyx_k_baseclass), 0, 0, 1, 1}, {&__pyx_n_s_borg_compress, __pyx_k_borg_compress, sizeof(__pyx_k_borg_compress), 0, 0, 1, 1}, {&__pyx_kp_s_borg_compress_pyx, __pyx_k_borg_compress_pyx, sizeof(__pyx_k_borg_compress_pyx), 0, 0, 1, 0}, {&__pyx_n_s_buf, __pyx_k_buf, sizeof(__pyx_k_buf), 0, 0, 1, 1}, {&__pyx_n_s_buffer, __pyx_k_buffer, sizeof(__pyx_k_buffer), 0, 0, 1, 1}, {&__pyx_n_s_check, __pyx_k_check, sizeof(__pyx_k_check), 0, 0, 1, 1}, {&__pyx_n_s_check_ok, __pyx_k_check_ok, sizeof(__pyx_k_check_ok), 0, 0, 1, 1}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_cls, __pyx_k_cls, sizeof(__pyx_k_cls), 0, 0, 1, 1}, {&__pyx_n_s_cmf, __pyx_k_cmf, sizeof(__pyx_k_cmf), 0, 0, 1, 1}, {&__pyx_n_s_compress, __pyx_k_compress, sizeof(__pyx_k_compress), 0, 0, 1, 1}, {&__pyx_kp_s_compresses_using_a_compressor_w, __pyx_k_compresses_using_a_compressor_w, sizeof(__pyx_k_compresses_using_a_compressor_w), 0, 0, 1, 0}, {&__pyx_n_s_compressor, __pyx_k_compressor, sizeof(__pyx_k_compressor), 0, 0, 1, 1}, {&__pyx_n_s_data, __pyx_k_data, sizeof(__pyx_k_data), 0, 0, 1, 1}, {&__pyx_n_s_decompress, __pyx_k_decompress, sizeof(__pyx_k_decompress), 0, 0, 1, 1}, {&__pyx_n_s_dest, __pyx_k_dest, sizeof(__pyx_k_dest), 0, 0, 1, 1}, {&__pyx_n_s_detect, __pyx_k_detect, sizeof(__pyx_k_detect), 0, 0, 1, 1}, {&__pyx_n_s_dict, __pyx_k_dict, sizeof(__pyx_k_dict), 0, 0, 1, 1}, {&__pyx_n_s_doc, __pyx_k_doc, sizeof(__pyx_k_doc), 0, 0, 1, 1}, {&__pyx_n_s_e, __pyx_k_e, sizeof(__pyx_k_e), 0, 0, 1, 1}, {&__pyx_n_s_error, __pyx_k_error, sizeof(__pyx_k_error), 0, 0, 1, 1}, {&__pyx_n_s_flg, __pyx_k_flg, sizeof(__pyx_k_flg), 0, 0, 1, 1}, {&__pyx_n_s_get, __pyx_k_get, sizeof(__pyx_k_get), 0, 0, 1, 1}, {&__pyx_n_s_get_compressor, __pyx_k_get_compressor, sizeof(__pyx_k_get_compressor), 0, 0, 1, 1}, {&__pyx_n_s_getstate, __pyx_k_getstate, sizeof(__pyx_k_getstate), 0, 0, 1, 1}, {&__pyx_n_s_hdr, __pyx_k_hdr, sizeof(__pyx_k_hdr), 0, 0, 1, 1}, {&__pyx_n_s_helpers, __pyx_k_helpers, sizeof(__pyx_k_helpers), 0, 0, 1, 1}, {&__pyx_n_s_idata, __pyx_k_idata, sizeof(__pyx_k_idata), 0, 0, 1, 1}, {&__pyx_n_s_import, __pyx_k_import, sizeof(__pyx_k_import), 0, 0, 1, 1}, {&__pyx_n_s_init, __pyx_k_init, sizeof(__pyx_k_init), 0, 0, 1, 1}, {&__pyx_n_s_is_deflate, __pyx_k_is_deflate, sizeof(__pyx_k_is_deflate), 0, 0, 1, 1}, {&__pyx_n_s_isize, __pyx_k_isize, sizeof(__pyx_k_isize), 0, 0, 1, 1}, {&__pyx_n_s_kwargs, __pyx_k_kwargs, sizeof(__pyx_k_kwargs), 0, 0, 1, 1}, {&__pyx_n_s_level, __pyx_k_level, sizeof(__pyx_k_level), 0, 0, 1, 1}, {&__pyx_n_s_lz4, __pyx_k_lz4, sizeof(__pyx_k_lz4), 0, 0, 1, 1}, {&__pyx_kp_s_lz4_compress_failed, __pyx_k_lz4_compress_failed, sizeof(__pyx_k_lz4_compress_failed), 0, 0, 1, 0}, {&__pyx_kp_s_lz4_decompress_failed, __pyx_k_lz4_decompress_failed, sizeof(__pyx_k_lz4_decompress_failed), 0, 0, 1, 0}, {&__pyx_n_s_lzma, __pyx_k_lzma, sizeof(__pyx_k_lzma), 0, 0, 1, 1}, {&__pyx_kp_s_lzma_compression_decompression, __pyx_k_lzma_compression_decompression, sizeof(__pyx_k_lzma_compression_decompression), 0, 0, 1, 0}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_metaclass, __pyx_k_metaclass, sizeof(__pyx_k_metaclass), 0, 0, 1, 1}, {&__pyx_n_s_module, __pyx_k_module, sizeof(__pyx_k_module), 0, 0, 1, 1}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_n_s_name_2, __pyx_k_name_2, sizeof(__pyx_k_name_2), 0, 0, 1, 1}, {&__pyx_n_s_new, __pyx_k_new, sizeof(__pyx_k_new), 0, 0, 1, 1}, {&__pyx_n_s_none, __pyx_k_none, sizeof(__pyx_k_none), 0, 0, 1, 1}, {&__pyx_kp_s_none_no_compression_just_pass_t, __pyx_k_none_no_compression_just_pass_t, sizeof(__pyx_k_none_no_compression_just_pass_t), 0, 0, 1, 0}, {&__pyx_n_s_null, __pyx_k_null, sizeof(__pyx_k_null), 0, 0, 1, 1}, {&__pyx_n_s_osize, __pyx_k_osize, sizeof(__pyx_k_osize), 0, 0, 1, 1}, {&__pyx_n_s_params, __pyx_k_params, sizeof(__pyx_k_params), 0, 0, 1, 1}, {&__pyx_n_s_pickle, __pyx_k_pickle, sizeof(__pyx_k_pickle), 0, 0, 1, 1}, {&__pyx_n_s_prepare, __pyx_k_prepare, sizeof(__pyx_k_prepare), 0, 0, 1, 1}, {&__pyx_n_s_preset, __pyx_k_preset, sizeof(__pyx_k_preset), 0, 0, 1, 1}, {&__pyx_n_s_pyx_PickleError, __pyx_k_pyx_PickleError, sizeof(__pyx_k_pyx_PickleError), 0, 0, 1, 1}, {&__pyx_n_s_pyx_checksum, __pyx_k_pyx_checksum, sizeof(__pyx_k_pyx_checksum), 0, 0, 1, 1}, {&__pyx_n_s_pyx_result, __pyx_k_pyx_result, sizeof(__pyx_k_pyx_result), 0, 0, 1, 1}, {&__pyx_n_s_pyx_state, __pyx_k_pyx_state, sizeof(__pyx_k_pyx_state), 0, 0, 1, 1}, {&__pyx_n_s_pyx_type, __pyx_k_pyx_type, sizeof(__pyx_k_pyx_type), 0, 0, 1, 1}, {&__pyx_n_s_pyx_unpickle_CompressorBase, __pyx_k_pyx_unpickle_CompressorBase, sizeof(__pyx_k_pyx_unpickle_CompressorBase), 0, 0, 1, 1}, {&__pyx_n_s_qualname, __pyx_k_qualname, sizeof(__pyx_k_qualname), 0, 0, 1, 1}, {&__pyx_kp_s_raw_LZ4_compression_decompressi, __pyx_k_raw_LZ4_compression_decompressi, sizeof(__pyx_k_raw_LZ4_compression_decompressi), 0, 0, 1, 0}, {&__pyx_n_s_reduce, __pyx_k_reduce, sizeof(__pyx_k_reduce), 0, 0, 1, 1}, {&__pyx_n_s_reduce_cython, __pyx_k_reduce_cython, sizeof(__pyx_k_reduce_cython), 0, 0, 1, 1}, {&__pyx_n_s_reduce_ex, __pyx_k_reduce_ex, sizeof(__pyx_k_reduce_ex), 0, 0, 1, 1}, {&__pyx_n_s_rsize, __pyx_k_rsize, sizeof(__pyx_k_rsize), 0, 0, 1, 1}, {&__pyx_n_s_self, __pyx_k_self, sizeof(__pyx_k_self), 0, 0, 1, 1}, {&__pyx_n_s_setstate, __pyx_k_setstate, sizeof(__pyx_k_setstate), 0, 0, 1, 1}, {&__pyx_n_s_setstate_cython, __pyx_k_setstate_cython, sizeof(__pyx_k_setstate_cython), 0, 0, 1, 1}, {&__pyx_n_s_size, __pyx_k_size, sizeof(__pyx_k_size), 0, 0, 1, 1}, {&__pyx_n_s_source, __pyx_k_source, sizeof(__pyx_k_source), 0, 0, 1, 1}, {&__pyx_n_s_startswith, __pyx_k_startswith, sizeof(__pyx_k_startswith), 0, 0, 1, 1}, {&__pyx_kp_s_stringsource, __pyx_k_stringsource, sizeof(__pyx_k_stringsource), 0, 0, 1, 0}, {&__pyx_n_s_super, __pyx_k_super, sizeof(__pyx_k_super), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {&__pyx_n_s_update, __pyx_k_update, sizeof(__pyx_k_update), 0, 0, 1, 1}, {&__pyx_n_s_zlib, __pyx_k_zlib, sizeof(__pyx_k_zlib), 0, 0, 1, 1}, {&__pyx_kp_s_zlib_compression_decompression, __pyx_k_zlib_compression_decompression, sizeof(__pyx_k_zlib_compression_decompression), 0, 0, 1, 0}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { __pyx_builtin_ImportError = __Pyx_GetBuiltinName(__pyx_n_s_ImportError); if (!__pyx_builtin_ImportError) __PYX_ERR(0, 4, __pyx_L1_error) __pyx_builtin_super = __Pyx_GetBuiltinName(__pyx_n_s_super); if (!__pyx_builtin_super) __PYX_ERR(0, 54, __pyx_L1_error) __pyx_builtin_ValueError = __Pyx_GetBuiltinName(__pyx_n_s_ValueError); if (!__pyx_builtin_ValueError) __PYX_ERR(0, 132, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "borg/compress.pyx":43 * def decompress(self, data): * # strip ID bytes * return data[2:] # <<<<<<<<<<<<<< * * */ __pyx_slice_ = PySlice_New(__pyx_int_2, Py_None, Py_None); if (unlikely(!__pyx_slice_)) __PYX_ERR(0, 43, __pyx_L1_error) __Pyx_GOTREF(__pyx_slice_); __Pyx_GIVEREF(__pyx_slice_); /* "borg/compress.pyx":91 * osize = LZ4_compress_limitedOutput(source, dest, isize, osize) * if not osize: * raise Exception('lz4 compress failed') # <<<<<<<<<<<<<< * return super().compress(dest[:osize]) * */ __pyx_tuple__2 = PyTuple_Pack(1, __pyx_kp_s_lz4_compress_failed); if (unlikely(!__pyx_tuple__2)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__2); __Pyx_GIVEREF(__pyx_tuple__2); /* "borg/compress.pyx":132 * self.level = level * if lzma is None: * raise ValueError('No lzma support found.') # <<<<<<<<<<<<<< * * def compress(self, data): */ __pyx_tuple__3 = PyTuple_Pack(1, __pyx_kp_s_No_lzma_support_found); if (unlikely(!__pyx_tuple__3)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__3); __Pyx_GIVEREF(__pyx_tuple__3); /* "borg/compress.pyx":158 * def detect(cls, data): * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] # <<<<<<<<<<<<<< * is_deflate = cmf & 0x0f == 8 * check_ok = (cmf * 256 + flg) % 31 == 0 */ __pyx_slice__4 = PySlice_New(Py_None, __pyx_int_2, Py_None); if (unlikely(!__pyx_slice__4)) __PYX_ERR(0, 158, __pyx_L1_error) __Pyx_GOTREF(__pyx_slice__4); __Pyx_GIVEREF(__pyx_slice__4); /* "borg/compress.pyx":205 * * def decompress(self, data): * hdr = bytes(data[:2]) # detect() does not work with memoryview # <<<<<<<<<<<<<< * for cls in COMPRESSOR_LIST: * if cls.detect(hdr): */ __pyx_slice__4 = PySlice_New(Py_None, __pyx_int_2, Py_None); if (unlikely(!__pyx_slice__4)) __PYX_ERR(0, 205, __pyx_L1_error) __Pyx_GOTREF(__pyx_slice__4); __Pyx_GIVEREF(__pyx_slice__4); /* "borg/compress.pyx":210 * return cls(**self.params).decompress(data) * else: * raise ValueError('No decompressor for this data found: %r.', data[:2]) # <<<<<<<<<<<<<< */ __pyx_slice__4 = PySlice_New(Py_None, __pyx_int_2, Py_None); if (unlikely(!__pyx_slice__4)) __PYX_ERR(0, 210, __pyx_L1_error) __Pyx_GOTREF(__pyx_slice__4); __Pyx_GIVEREF(__pyx_slice__4); /* "borg/compress.pyx":53 * name = 'none' * * def compress(self, data): # <<<<<<<<<<<<<< * return super().compress(data) * */ __pyx_tuple__7 = PyTuple_Pack(2, __pyx_n_s_self, __pyx_n_s_data); if (unlikely(!__pyx_tuple__7)) __PYX_ERR(0, 53, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__7); __Pyx_GIVEREF(__pyx_tuple__7); __pyx_codeobj__8 = (PyObject*)__Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__7, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_compress, 53, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__8)) __PYX_ERR(0, 53, __pyx_L1_error) /* "borg/compress.pyx":56 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * if not isinstance(data, bytes): */ __pyx_tuple__9 = PyTuple_Pack(2, __pyx_n_s_self, __pyx_n_s_data); if (unlikely(!__pyx_tuple__9)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__9); __Pyx_GIVEREF(__pyx_tuple__9); __pyx_codeobj__10 = (PyObject*)__Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__9, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_decompress, 56, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__10)) __PYX_ERR(0, 56, __pyx_L1_error) /* "borg/compress.pyx":75 * name = 'lz4' * * def __init__(self, **kwargs): # <<<<<<<<<<<<<< * pass * */ __pyx_tuple__12 = PyTuple_Pack(2, __pyx_n_s_self, __pyx_n_s_kwargs); if (unlikely(!__pyx_tuple__12)) __PYX_ERR(0, 75, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__12); __Pyx_GIVEREF(__pyx_tuple__12); __pyx_codeobj__13 = (PyObject*)__Pyx_PyCode_New(1, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS|CO_VARKEYWORDS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__12, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_init, 75, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__13)) __PYX_ERR(0, 75, __pyx_L1_error) /* "borg/compress.pyx":78 * pass * * def compress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ __pyx_tuple__14 = PyTuple_Pack(7, __pyx_n_s_self, __pyx_n_s_idata, __pyx_n_s_isize, __pyx_n_s_osize, __pyx_n_s_source, __pyx_n_s_dest, __pyx_n_s_buf); if (unlikely(!__pyx_tuple__14)) __PYX_ERR(0, 78, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__14); __Pyx_GIVEREF(__pyx_tuple__14); __pyx_codeobj__15 = (PyObject*)__Pyx_PyCode_New(2, 0, 7, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__14, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_compress, 78, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__15)) __PYX_ERR(0, 78, __pyx_L1_error) /* "borg/compress.pyx":94 * return super().compress(dest[:osize]) * * def decompress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ __pyx_tuple__16 = PyTuple_Pack(8, __pyx_n_s_self, __pyx_n_s_idata, __pyx_n_s_isize, __pyx_n_s_osize, __pyx_n_s_rsize, __pyx_n_s_source, __pyx_n_s_dest, __pyx_n_s_buf); if (unlikely(!__pyx_tuple__16)) __PYX_ERR(0, 94, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__16); __Pyx_GIVEREF(__pyx_tuple__16); __pyx_codeobj__17 = (PyObject*)__Pyx_PyCode_New(2, 0, 8, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__16, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_decompress, 94, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__17)) __PYX_ERR(0, 94, __pyx_L1_error) /* "borg/compress.pyx":128 * name = 'lzma' * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ __pyx_tuple__19 = PyTuple_Pack(3, __pyx_n_s_self, __pyx_n_s_level, __pyx_n_s_kwargs); if (unlikely(!__pyx_tuple__19)) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__19); __Pyx_GIVEREF(__pyx_tuple__19); __pyx_codeobj__20 = (PyObject*)__Pyx_PyCode_New(2, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS|CO_VARKEYWORDS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__19, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_init, 128, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__20)) __PYX_ERR(0, 128, __pyx_L1_error) __pyx_tuple__21 = PyTuple_Pack(1, ((PyObject *)__pyx_int_6)); if (unlikely(!__pyx_tuple__21)) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__21); __Pyx_GIVEREF(__pyx_tuple__21); /* "borg/compress.pyx":134 * raise ValueError('No lzma support found.') * * def compress(self, data): # <<<<<<<<<<<<<< * # we do not need integrity checks in lzma, we do that already * data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) */ __pyx_tuple__22 = PyTuple_Pack(2, __pyx_n_s_self, __pyx_n_s_data); if (unlikely(!__pyx_tuple__22)) __PYX_ERR(0, 134, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__22); __Pyx_GIVEREF(__pyx_tuple__22); __pyx_codeobj__23 = (PyObject*)__Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__22, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_compress, 134, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__23)) __PYX_ERR(0, 134, __pyx_L1_error) /* "borg/compress.pyx":139 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * try: */ __pyx_tuple__24 = PyTuple_Pack(3, __pyx_n_s_self, __pyx_n_s_data, __pyx_n_s_e); if (unlikely(!__pyx_tuple__24)) __PYX_ERR(0, 139, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__24); __Pyx_GIVEREF(__pyx_tuple__24); __pyx_codeobj__25 = (PyObject*)__Pyx_PyCode_New(2, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__24, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_decompress, 139, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__25)) __PYX_ERR(0, 139, __pyx_L1_error) /* "borg/compress.pyx":156 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] */ __pyx_tuple__27 = PyTuple_Pack(6, __pyx_n_s_cls, __pyx_n_s_data, __pyx_n_s_cmf, __pyx_n_s_flg, __pyx_n_s_is_deflate, __pyx_n_s_check_ok); if (unlikely(!__pyx_tuple__27)) __PYX_ERR(0, 156, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__27); __Pyx_GIVEREF(__pyx_tuple__27); __pyx_codeobj__28 = (PyObject*)__Pyx_PyCode_New(2, 0, 6, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__27, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_detect, 156, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__28)) __PYX_ERR(0, 156, __pyx_L1_error) /* "borg/compress.pyx":163 * return check_ok and is_deflate * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ __pyx_tuple__29 = PyTuple_Pack(3, __pyx_n_s_self, __pyx_n_s_level, __pyx_n_s_kwargs); if (unlikely(!__pyx_tuple__29)) __PYX_ERR(0, 163, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__29); __Pyx_GIVEREF(__pyx_tuple__29); __pyx_codeobj__30 = (PyObject*)__Pyx_PyCode_New(2, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS|CO_VARKEYWORDS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__29, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_init, 163, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__30)) __PYX_ERR(0, 163, __pyx_L1_error) __pyx_tuple__31 = PyTuple_Pack(1, ((PyObject *)__pyx_int_6)); if (unlikely(!__pyx_tuple__31)) __PYX_ERR(0, 163, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__31); __Pyx_GIVEREF(__pyx_tuple__31); /* "borg/compress.pyx":167 * self.level = level * * def compress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not add ID bytes * return zlib.compress(data, self.level) */ __pyx_tuple__32 = PyTuple_Pack(2, __pyx_n_s_self, __pyx_n_s_data); if (unlikely(!__pyx_tuple__32)) __PYX_ERR(0, 167, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__32); __Pyx_GIVEREF(__pyx_tuple__32); __pyx_codeobj__33 = (PyObject*)__Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__32, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_compress, 167, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__33)) __PYX_ERR(0, 167, __pyx_L1_error) /* "borg/compress.pyx":171 * return zlib.compress(data, self.level) * * def decompress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not strip ID bytes * try: */ __pyx_tuple__34 = PyTuple_Pack(3, __pyx_n_s_self, __pyx_n_s_data, __pyx_n_s_e); if (unlikely(!__pyx_tuple__34)) __PYX_ERR(0, 171, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__34); __Pyx_GIVEREF(__pyx_tuple__34); __pyx_codeobj__35 = (PyObject*)__Pyx_PyCode_New(2, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__34, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_decompress, 171, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__35)) __PYX_ERR(0, 171, __pyx_L1_error) /* "borg/compress.pyx":187 * COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first * * def get_compressor(name, **kwargs): # <<<<<<<<<<<<<< * cls = COMPRESSOR_TABLE[name] * return cls(**kwargs) */ __pyx_tuple__36 = PyTuple_Pack(3, __pyx_n_s_name, __pyx_n_s_kwargs, __pyx_n_s_cls); if (unlikely(!__pyx_tuple__36)) __PYX_ERR(0, 187, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__36); __Pyx_GIVEREF(__pyx_tuple__36); __pyx_codeobj__37 = (PyObject*)__Pyx_PyCode_New(1, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS|CO_VARKEYWORDS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__36, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_get_compressor, 187, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__37)) __PYX_ERR(0, 187, __pyx_L1_error) /* "borg/compress.pyx":197 * decompresses everything we can handle (autodetect) * """ * def __init__(self, name='null', **kwargs): # <<<<<<<<<<<<<< * self.params = kwargs * self.compressor = get_compressor(name, **self.params) */ __pyx_tuple__38 = PyTuple_Pack(3, __pyx_n_s_self, __pyx_n_s_name, __pyx_n_s_kwargs); if (unlikely(!__pyx_tuple__38)) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__38); __Pyx_GIVEREF(__pyx_tuple__38); __pyx_codeobj__39 = (PyObject*)__Pyx_PyCode_New(2, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS|CO_VARKEYWORDS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__38, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_init, 197, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__39)) __PYX_ERR(0, 197, __pyx_L1_error) __pyx_tuple__40 = PyTuple_Pack(1, ((PyObject*)__pyx_n_s_null)); if (unlikely(!__pyx_tuple__40)) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__40); __Pyx_GIVEREF(__pyx_tuple__40); /* "borg/compress.pyx":201 * self.compressor = get_compressor(name, **self.params) * * def compress(self, data): # <<<<<<<<<<<<<< * return self.compressor.compress(data) * */ __pyx_tuple__41 = PyTuple_Pack(2, __pyx_n_s_self, __pyx_n_s_data); if (unlikely(!__pyx_tuple__41)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__41); __Pyx_GIVEREF(__pyx_tuple__41); __pyx_codeobj__42 = (PyObject*)__Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__41, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_compress, 201, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__42)) __PYX_ERR(0, 201, __pyx_L1_error) /* "borg/compress.pyx":204 * return self.compressor.compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: */ __pyx_tuple__43 = PyTuple_Pack(4, __pyx_n_s_self, __pyx_n_s_data, __pyx_n_s_hdr, __pyx_n_s_cls); if (unlikely(!__pyx_tuple__43)) __PYX_ERR(0, 204, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__43); __Pyx_GIVEREF(__pyx_tuple__43); __pyx_codeobj__44 = (PyObject*)__Pyx_PyCode_New(2, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__43, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_compress_pyx, __pyx_n_s_decompress, 204, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__44)) __PYX_ERR(0, 204, __pyx_L1_error) /* "(tree fragment)":1 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): # <<<<<<<<<<<<<< * if __pyx_checksum != 0xd41d8cd: * from pickle import PickleError as __pyx_PickleError */ __pyx_tuple__45 = PyTuple_Pack(5, __pyx_n_s_pyx_type, __pyx_n_s_pyx_checksum, __pyx_n_s_pyx_state, __pyx_n_s_pyx_PickleError, __pyx_n_s_pyx_result); if (unlikely(!__pyx_tuple__45)) __PYX_ERR(1, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__45); __Pyx_GIVEREF(__pyx_tuple__45); __pyx_codeobj__46 = (PyObject*)__Pyx_PyCode_New(3, 0, 5, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__45, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_stringsource, __pyx_n_s_pyx_unpickle_CompressorBase, 1, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__46)) __PYX_ERR(1, 1, __pyx_L1_error) __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(0, 1, __pyx_L1_error); __pyx_int_0 = PyInt_FromLong(0); if (unlikely(!__pyx_int_0)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_2 = PyInt_FromLong(2); if (unlikely(!__pyx_int_2)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_6 = PyInt_FromLong(6); if (unlikely(!__pyx_int_6)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_8 = PyInt_FromLong(8); if (unlikely(!__pyx_int_8)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_15 = PyInt_FromLong(15); if (unlikely(!__pyx_int_15)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_31 = PyInt_FromLong(31); if (unlikely(!__pyx_int_31)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_256 = PyInt_FromLong(256); if (unlikely(!__pyx_int_256)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_222419149 = PyInt_FromLong(222419149L); if (unlikely(!__pyx_int_222419149)) __PYX_ERR(0, 1, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ if (PyType_Ready(&__pyx_type_4borg_8compress_CompressorBase) < 0) __PYX_ERR(0, 20, __pyx_L1_error) __pyx_type_4borg_8compress_CompressorBase.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_8compress_CompressorBase.tp_dictoffset && __pyx_type_4borg_8compress_CompressorBase.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_8compress_CompressorBase.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (PyObject_SetAttr(__pyx_m, __pyx_n_s_CompressorBase, (PyObject *)&__pyx_type_4borg_8compress_CompressorBase) < 0) __PYX_ERR(0, 20, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_8compress_CompressorBase) < 0) __PYX_ERR(0, 20, __pyx_L1_error) __pyx_ptype_4borg_8compress_CompressorBase = &__pyx_type_4borg_8compress_CompressorBase; __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC initcompress(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC initcompress(void) #else __Pyx_PyMODINIT_FUNC PyInit_compress(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_compress(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_compress(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; int __pyx_t_5; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; PyObject *__pyx_t_9 = NULL; PyObject *__pyx_t_10 = NULL; PyObject *__pyx_t_11 = NULL; PyObject *__pyx_t_12 = NULL; PyObject *__pyx_t_13 = NULL; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'compress' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_compress(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("compress", __pyx_methods, 0, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(0, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(0, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(0, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(0, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__compress) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name_2, __pyx_n_s_main) < 0) __PYX_ERR(0, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(0, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.compress")) { if (unlikely(PyDict_SetItemString(modules, "borg.compress", __pyx_m) < 0)) __PYX_ERR(0, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); if (unlikely(__Pyx_modinit_type_init_code() != 0)) goto __pyx_L1_error; (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /* "borg/compress.pyx":1 * import zlib # <<<<<<<<<<<<<< * try: * import lzma */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_zlib, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_zlib, __pyx_t_1) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":2 * import zlib * try: # <<<<<<<<<<<<<< * import lzma * except ImportError: */ { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ExceptionSave(&__pyx_t_2, &__pyx_t_3, &__pyx_t_4); __Pyx_XGOTREF(__pyx_t_2); __Pyx_XGOTREF(__pyx_t_3); __Pyx_XGOTREF(__pyx_t_4); /*try:*/ { /* "borg/compress.pyx":3 * import zlib * try: * import lzma # <<<<<<<<<<<<<< * except ImportError: * lzma = None */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_lzma, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L2_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_lzma, __pyx_t_1) < 0) __PYX_ERR(0, 3, __pyx_L2_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":2 * import zlib * try: # <<<<<<<<<<<<<< * import lzma * except ImportError: */ } __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; goto __pyx_L7_try_end; __pyx_L2_error:; __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":4 * try: * import lzma * except ImportError: # <<<<<<<<<<<<<< * lzma = None * */ __pyx_t_5 = __Pyx_PyErr_ExceptionMatches(__pyx_builtin_ImportError); if (__pyx_t_5) { __Pyx_AddTraceback("borg.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); if (__Pyx_GetException(&__pyx_t_1, &__pyx_t_6, &__pyx_t_7) < 0) __PYX_ERR(0, 4, __pyx_L4_except_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_GOTREF(__pyx_t_6); __Pyx_GOTREF(__pyx_t_7); /* "borg/compress.pyx":5 * import lzma * except ImportError: * lzma = None # <<<<<<<<<<<<<< * * from .helpers import Buffer, DecompressionError */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_lzma, Py_None) < 0) __PYX_ERR(0, 5, __pyx_L4_except_error) __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0; goto __pyx_L3_exception_handled; } goto __pyx_L4_except_error; __pyx_L4_except_error:; /* "borg/compress.pyx":2 * import zlib * try: # <<<<<<<<<<<<<< * import lzma * except ImportError: */ __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_XGIVEREF(__pyx_t_4); __Pyx_ExceptionReset(__pyx_t_2, __pyx_t_3, __pyx_t_4); goto __pyx_L1_error; __pyx_L3_exception_handled:; __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_XGIVEREF(__pyx_t_4); __Pyx_ExceptionReset(__pyx_t_2, __pyx_t_3, __pyx_t_4); __pyx_L7_try_end:; } /* "borg/compress.pyx":7 * lzma = None * * from .helpers import Buffer, DecompressionError # <<<<<<<<<<<<<< * * API_VERSION = '1.0_01' */ __pyx_t_7 = PyList_New(2); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_INCREF(__pyx_n_s_Buffer); __Pyx_GIVEREF(__pyx_n_s_Buffer); PyList_SET_ITEM(__pyx_t_7, 0, __pyx_n_s_Buffer); __Pyx_INCREF(__pyx_n_s_DecompressionError); __Pyx_GIVEREF(__pyx_n_s_DecompressionError); PyList_SET_ITEM(__pyx_t_7, 1, __pyx_n_s_DecompressionError); __pyx_t_6 = __Pyx_Import(__pyx_n_s_helpers, __pyx_t_7, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_7 = __Pyx_ImportFrom(__pyx_t_6, __pyx_n_s_Buffer); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); if (PyDict_SetItem(__pyx_d, __pyx_n_s_Buffer, __pyx_t_7) < 0) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_7 = __Pyx_ImportFrom(__pyx_t_6, __pyx_n_s_DecompressionError); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); if (PyDict_SetItem(__pyx_d, __pyx_n_s_DecompressionError, __pyx_t_7) < 0) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; /* "borg/compress.pyx":9 * from .helpers import Buffer, DecompressionError * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * cdef extern from "lz4.h": */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(0, 9, __pyx_L1_error) /* "borg/compress.pyx":17 * * * buffer = Buffer(bytearray, size=0) # <<<<<<<<<<<<<< * * */ __Pyx_GetModuleGlobalName(__pyx_t_6, __pyx_n_s_Buffer); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 17, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = PyTuple_New(1); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 17, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_INCREF(((PyObject *)(&PyByteArray_Type))); __Pyx_GIVEREF(((PyObject *)(&PyByteArray_Type))); PyTuple_SET_ITEM(__pyx_t_7, 0, ((PyObject *)(&PyByteArray_Type))); __pyx_t_1 = __Pyx_PyDict_NewPresized(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 17, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_t_1, __pyx_n_s_size, __pyx_int_0) < 0) __PYX_ERR(0, 17, __pyx_L1_error) __pyx_t_8 = __Pyx_PyObject_Call(__pyx_t_6, __pyx_t_7, __pyx_t_1); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 17, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_buffer, __pyx_t_8) < 0) __PYX_ERR(0, 17, __pyx_L1_error) __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; /* "borg/compress.pyx":26 * adding/stripping the ID header (which enable auto detection). * """ * ID = b'\xFF\xFF' # reserved and not used # <<<<<<<<<<<<<< * # overwrite with a unique 2-bytes bytestring in child classes * name = 'baseclass' */ if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase->tp_dict, __pyx_n_s_ID, __pyx_kp_b__5) < 0) __PYX_ERR(0, 26, __pyx_L1_error) PyType_Modified(__pyx_ptype_4borg_8compress_CompressorBase); /* "borg/compress.pyx":28 * ID = b'\xFF\xFF' # reserved and not used * # overwrite with a unique 2-bytes bytestring in child classes * name = 'baseclass' # <<<<<<<<<<<<<< * * @classmethod */ if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase->tp_dict, __pyx_n_s_name, __pyx_n_s_baseclass) < 0) __PYX_ERR(0, 28, __pyx_L1_error) PyType_Modified(__pyx_ptype_4borg_8compress_CompressorBase); /* "borg/compress.pyx":31 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * return data.startswith(cls.ID) * */ __Pyx_GetNameInClass(__pyx_t_8, (PyObject *)__pyx_ptype_4borg_8compress_CompressorBase, __pyx_n_s_detect); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 31, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); /* "borg/compress.pyx":30 * name = 'baseclass' * * @classmethod # <<<<<<<<<<<<<< * def detect(cls, data): * return data.startswith(cls.ID) */ __pyx_t_1 = __Pyx_Method_ClassMethod(__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 30, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase->tp_dict, __pyx_n_s_detect, __pyx_t_1) < 0) __PYX_ERR(0, 31, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; PyType_Modified(__pyx_ptype_4borg_8compress_CompressorBase); /* "borg/compress.pyx":46 * * * class CNONE(CompressorBase): # <<<<<<<<<<<<<< * """ * none - no compression, just pass through data */ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __Pyx_GIVEREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __pyx_t_8 = __Pyx_CalculateMetaclass(NULL, __pyx_t_1); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_Py3MetaclassPrepare(__pyx_t_8, __pyx_t_1, __pyx_n_s_CNONE, __pyx_n_s_CNONE, (PyObject *) NULL, __pyx_n_s_borg_compress, __pyx_kp_s_none_no_compression_just_pass_t); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_6 = PyList_New(0); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); /* "borg/compress.pyx":50 * none - no compression, just pass through data * """ * ID = b'\x00\x00' # <<<<<<<<<<<<<< * name = 'none' * */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_ID, __pyx_kp_b__6) < 0) __PYX_ERR(0, 50, __pyx_L1_error) /* "borg/compress.pyx":51 * """ * ID = b'\x00\x00' * name = 'none' # <<<<<<<<<<<<<< * * def compress(self, data): */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_name, __pyx_n_s_none) < 0) __PYX_ERR(0, 51, __pyx_L1_error) /* "borg/compress.pyx":53 * name = 'none' * * def compress(self, data): # <<<<<<<<<<<<<< * return super().compress(data) * */ __pyx_t_9 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_5CNONE_1compress, 0, __pyx_n_s_CNONE_compress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__8)); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 53, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_INCREF(__pyx_t_9); PyList_Append(__pyx_t_6, __pyx_t_9); __Pyx_GIVEREF(__pyx_t_9); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_compress, __pyx_t_9) < 0) __PYX_ERR(0, 53, __pyx_L1_error) __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; /* "borg/compress.pyx":56 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * if not isinstance(data, bytes): */ __pyx_t_9 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_5CNONE_3decompress, 0, __pyx_n_s_CNONE_decompress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__10)); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_INCREF(__pyx_t_9); PyList_Append(__pyx_t_6, __pyx_t_9); __Pyx_GIVEREF(__pyx_t_9); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_decompress, __pyx_t_9) < 0) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; /* "borg/compress.pyx":46 * * * class CNONE(CompressorBase): # <<<<<<<<<<<<<< * """ * none - no compression, just pass through data */ __pyx_t_9 = __Pyx_Py3ClassCreate(__pyx_t_8, __pyx_n_s_CNONE, __pyx_t_1, __pyx_t_7, NULL, 0, 1); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); if (__Pyx_CyFunction_InitClassCell(__pyx_t_6, __pyx_t_9) < 0) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_CNONE, __pyx_t_9) < 0) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":63 * * * class LZ4(CompressorBase): # <<<<<<<<<<<<<< * """ * raw LZ4 compression / decompression (liblz4). */ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __Pyx_GIVEREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __pyx_t_8 = __Pyx_CalculateMetaclass(NULL, __pyx_t_1); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_Py3MetaclassPrepare(__pyx_t_8, __pyx_t_1, __pyx_n_s_LZ4, __pyx_n_s_LZ4, (PyObject *) NULL, __pyx_n_s_borg_compress, __pyx_kp_s_raw_LZ4_compression_decompressi); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_9 = PyList_New(0); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); /* "borg/compress.pyx":72 * - uses safe lz4 methods that never go beyond the end of the output buffer * """ * ID = b'\x01\x00' # <<<<<<<<<<<<<< * name = 'lz4' * */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_ID, __pyx_kp_b__11) < 0) __PYX_ERR(0, 72, __pyx_L1_error) /* "borg/compress.pyx":73 * """ * ID = b'\x01\x00' * name = 'lz4' # <<<<<<<<<<<<<< * * def __init__(self, **kwargs): */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_name, __pyx_n_s_lz4) < 0) __PYX_ERR(0, 73, __pyx_L1_error) /* "borg/compress.pyx":75 * name = 'lz4' * * def __init__(self, **kwargs): # <<<<<<<<<<<<<< * pass * */ __pyx_t_10 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_3LZ4_1__init__, 0, __pyx_n_s_LZ4___init, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__13)); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 75, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_10); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_init, __pyx_t_10) < 0) __PYX_ERR(0, 75, __pyx_L1_error) __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; /* "borg/compress.pyx":78 * pass * * def compress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ __pyx_t_10 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_3LZ4_3compress, 0, __pyx_n_s_LZ4_compress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__15)); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 78, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_10); __Pyx_INCREF(__pyx_t_10); PyList_Append(__pyx_t_9, __pyx_t_10); __Pyx_GIVEREF(__pyx_t_10); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_compress, __pyx_t_10) < 0) __PYX_ERR(0, 78, __pyx_L1_error) __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; /* "borg/compress.pyx":94 * return super().compress(dest[:osize]) * * def decompress(self, idata): # <<<<<<<<<<<<<< * if not isinstance(idata, bytes): * idata = bytes(idata) # code below does not work with memoryview */ __pyx_t_10 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_3LZ4_5decompress, 0, __pyx_n_s_LZ4_decompress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__17)); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 94, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_10); __Pyx_INCREF(__pyx_t_10); PyList_Append(__pyx_t_9, __pyx_t_10); __Pyx_GIVEREF(__pyx_t_10); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_decompress, __pyx_t_10) < 0) __PYX_ERR(0, 94, __pyx_L1_error) __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; /* "borg/compress.pyx":63 * * * class LZ4(CompressorBase): # <<<<<<<<<<<<<< * """ * raw LZ4 compression / decompression (liblz4). */ __pyx_t_10 = __Pyx_Py3ClassCreate(__pyx_t_8, __pyx_n_s_LZ4, __pyx_t_1, __pyx_t_7, NULL, 0, 1); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_10); if (__Pyx_CyFunction_InitClassCell(__pyx_t_9, __pyx_t_10) < 0) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_LZ4, __pyx_t_10) < 0) __PYX_ERR(0, 63, __pyx_L1_error) __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":121 * * * class LZMA(CompressorBase): # <<<<<<<<<<<<<< * """ * lzma compression / decompression */ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __Pyx_GIVEREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __pyx_t_8 = __Pyx_CalculateMetaclass(NULL, __pyx_t_1); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_Py3MetaclassPrepare(__pyx_t_8, __pyx_t_1, __pyx_n_s_LZMA, __pyx_n_s_LZMA, (PyObject *) NULL, __pyx_n_s_borg_compress, __pyx_kp_s_lzma_compression_decompression); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_10 = PyList_New(0); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_10); /* "borg/compress.pyx":125 * lzma compression / decompression * """ * ID = b'\x02\x00' # <<<<<<<<<<<<<< * name = 'lzma' * */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_ID, __pyx_kp_b__18) < 0) __PYX_ERR(0, 125, __pyx_L1_error) /* "borg/compress.pyx":126 * """ * ID = b'\x02\x00' * name = 'lzma' # <<<<<<<<<<<<<< * * def __init__(self, level=6, **kwargs): */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_name, __pyx_n_s_lzma) < 0) __PYX_ERR(0, 126, __pyx_L1_error) /* "borg/compress.pyx":128 * name = 'lzma' * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ __pyx_t_11 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4LZMA_1__init__, 0, __pyx_n_s_LZMA___init, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__20)); if (unlikely(!__pyx_t_11)) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_11); __Pyx_INCREF(__pyx_t_11); PyList_Append(__pyx_t_10, __pyx_t_11); __Pyx_GIVEREF(__pyx_t_11); __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_11, __pyx_tuple__21); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_init, __pyx_t_11) < 0) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_DECREF(__pyx_t_11); __pyx_t_11 = 0; /* "borg/compress.pyx":134 * raise ValueError('No lzma support found.') * * def compress(self, data): # <<<<<<<<<<<<<< * # we do not need integrity checks in lzma, we do that already * data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) */ __pyx_t_11 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4LZMA_3compress, 0, __pyx_n_s_LZMA_compress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__23)); if (unlikely(!__pyx_t_11)) __PYX_ERR(0, 134, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_11); __Pyx_INCREF(__pyx_t_11); PyList_Append(__pyx_t_10, __pyx_t_11); __Pyx_GIVEREF(__pyx_t_11); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_compress, __pyx_t_11) < 0) __PYX_ERR(0, 134, __pyx_L1_error) __Pyx_DECREF(__pyx_t_11); __pyx_t_11 = 0; /* "borg/compress.pyx":139 * return super().compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * data = super().decompress(data) * try: */ __pyx_t_11 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4LZMA_5decompress, 0, __pyx_n_s_LZMA_decompress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__25)); if (unlikely(!__pyx_t_11)) __PYX_ERR(0, 139, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_11); __Pyx_INCREF(__pyx_t_11); PyList_Append(__pyx_t_10, __pyx_t_11); __Pyx_GIVEREF(__pyx_t_11); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_decompress, __pyx_t_11) < 0) __PYX_ERR(0, 139, __pyx_L1_error) __Pyx_DECREF(__pyx_t_11); __pyx_t_11 = 0; /* "borg/compress.pyx":121 * * * class LZMA(CompressorBase): # <<<<<<<<<<<<<< * """ * lzma compression / decompression */ __pyx_t_11 = __Pyx_Py3ClassCreate(__pyx_t_8, __pyx_n_s_LZMA, __pyx_t_1, __pyx_t_7, NULL, 0, 1); if (unlikely(!__pyx_t_11)) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_11); if (__Pyx_CyFunction_InitClassCell(__pyx_t_10, __pyx_t_11) < 0) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_LZMA, __pyx_t_11) < 0) __PYX_ERR(0, 121, __pyx_L1_error) __Pyx_DECREF(__pyx_t_11); __pyx_t_11 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":147 * * * class ZLIB(CompressorBase): # <<<<<<<<<<<<<< * """ * zlib compression / decompression (python stdlib) */ __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __Pyx_GIVEREF(((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); PyTuple_SET_ITEM(__pyx_t_1, 0, ((PyObject *)__pyx_ptype_4borg_8compress_CompressorBase)); __pyx_t_8 = __Pyx_CalculateMetaclass(NULL, __pyx_t_1); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_Py3MetaclassPrepare(__pyx_t_8, __pyx_t_1, __pyx_n_s_ZLIB, __pyx_n_s_ZLIB, (PyObject *) NULL, __pyx_n_s_borg_compress, __pyx_kp_s_zlib_compression_decompression); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_11 = PyList_New(0); if (unlikely(!__pyx_t_11)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_11); /* "borg/compress.pyx":151 * zlib compression / decompression (python stdlib) * """ * ID = b'\x08\x00' # not used here, see detect() # <<<<<<<<<<<<<< * # avoid all 0x.8.. IDs elsewhere! * name = 'zlib' */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_ID, __pyx_kp_b__26) < 0) __PYX_ERR(0, 151, __pyx_L1_error) /* "borg/compress.pyx":153 * ID = b'\x08\x00' # not used here, see detect() * # avoid all 0x.8.. IDs elsewhere! * name = 'zlib' # <<<<<<<<<<<<<< * * @classmethod */ if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_name, __pyx_n_s_zlib) < 0) __PYX_ERR(0, 153, __pyx_L1_error) /* "borg/compress.pyx":156 * * @classmethod * def detect(cls, data): # <<<<<<<<<<<<<< * # matches misc. patterns 0x.8.. used by zlib * cmf, flg = data[:2] */ __pyx_t_12 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4ZLIB_1detect, __Pyx_CYFUNCTION_CLASSMETHOD, __pyx_n_s_ZLIB_detect, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__28)); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 156, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); /* "borg/compress.pyx":155 * name = 'zlib' * * @classmethod # <<<<<<<<<<<<<< * def detect(cls, data): * # matches misc. patterns 0x.8.. used by zlib */ __pyx_t_13 = __Pyx_Method_ClassMethod(__pyx_t_12); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 155, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_detect, __pyx_t_13) < 0) __PYX_ERR(0, 156, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":163 * return check_ok and is_deflate * * def __init__(self, level=6, **kwargs): # <<<<<<<<<<<<<< * super().__init__(**kwargs) * self.level = level */ __pyx_t_13 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4ZLIB_3__init__, 0, __pyx_n_s_ZLIB___init, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__30)); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 163, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); __Pyx_INCREF(__pyx_t_13); PyList_Append(__pyx_t_11, __pyx_t_13); __Pyx_GIVEREF(__pyx_t_13); __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_13, __pyx_tuple__31); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_init, __pyx_t_13) < 0) __PYX_ERR(0, 163, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":167 * self.level = level * * def compress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not add ID bytes * return zlib.compress(data, self.level) */ __pyx_t_13 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4ZLIB_5compress, 0, __pyx_n_s_ZLIB_compress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__33)); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 167, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_compress, __pyx_t_13) < 0) __PYX_ERR(0, 167, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":171 * return zlib.compress(data, self.level) * * def decompress(self, data): # <<<<<<<<<<<<<< * # note: for compatibility no super call, do not strip ID bytes * try: */ __pyx_t_13 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_4ZLIB_7decompress, 0, __pyx_n_s_ZLIB_decompress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__35)); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 171, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); if (__Pyx_SetNameInClass(__pyx_t_7, __pyx_n_s_decompress, __pyx_t_13) < 0) __PYX_ERR(0, 171, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":147 * * * class ZLIB(CompressorBase): # <<<<<<<<<<<<<< * """ * zlib compression / decompression (python stdlib) */ __pyx_t_13 = __Pyx_Py3ClassCreate(__pyx_t_8, __pyx_n_s_ZLIB, __pyx_t_1, __pyx_t_7, NULL, 0, 1); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); if (__Pyx_CyFunction_InitClassCell(__pyx_t_11, __pyx_t_13) < 0) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_DECREF(__pyx_t_11); __pyx_t_11 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_ZLIB, __pyx_t_13) < 0) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":180 * * COMPRESSOR_TABLE = { * CNONE.name: CNONE, # <<<<<<<<<<<<<< * LZ4.name: LZ4, * ZLIB.name: ZLIB, */ __pyx_t_1 = __Pyx_PyDict_NewPresized(4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_CNONE); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_8, __pyx_n_s_name); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_CNONE); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); if (PyDict_SetItem(__pyx_t_1, __pyx_t_7, __pyx_t_8) < 0) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; /* "borg/compress.pyx":181 * COMPRESSOR_TABLE = { * CNONE.name: CNONE, * LZ4.name: LZ4, # <<<<<<<<<<<<<< * ZLIB.name: ZLIB, * LZMA.name: LZMA, */ __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_LZ4); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 181, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_8, __pyx_n_s_name); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 181, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_LZ4); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 181, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); if (PyDict_SetItem(__pyx_t_1, __pyx_t_7, __pyx_t_8) < 0) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; /* "borg/compress.pyx":182 * CNONE.name: CNONE, * LZ4.name: LZ4, * ZLIB.name: ZLIB, # <<<<<<<<<<<<<< * LZMA.name: LZMA, * } */ __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_ZLIB); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 182, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_8, __pyx_n_s_name); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 182, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_ZLIB); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 182, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); if (PyDict_SetItem(__pyx_t_1, __pyx_t_7, __pyx_t_8) < 0) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; /* "borg/compress.pyx":183 * LZ4.name: LZ4, * ZLIB.name: ZLIB, * LZMA.name: LZMA, # <<<<<<<<<<<<<< * } * COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first */ __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_LZMA); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 183, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_8, __pyx_n_s_name); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 183, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_LZMA); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 183, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); if (PyDict_SetItem(__pyx_t_1, __pyx_t_7, __pyx_t_8) < 0) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_COMPRESSOR_TABLE, __pyx_t_1) < 0) __PYX_ERR(0, 179, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/compress.pyx":185 * LZMA.name: LZMA, * } * COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first # <<<<<<<<<<<<<< * * def get_compressor(name, **kwargs): */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_LZ4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 185, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_GetModuleGlobalName(__pyx_t_8, __pyx_n_s_CNONE); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 185, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_ZLIB); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 185, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_GetModuleGlobalName(__pyx_t_13, __pyx_n_s_LZMA); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 185, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); __pyx_t_12 = PyList_New(4); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 185, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); __Pyx_GIVEREF(__pyx_t_1); PyList_SET_ITEM(__pyx_t_12, 0, __pyx_t_1); __Pyx_GIVEREF(__pyx_t_8); PyList_SET_ITEM(__pyx_t_12, 1, __pyx_t_8); __Pyx_GIVEREF(__pyx_t_7); PyList_SET_ITEM(__pyx_t_12, 2, __pyx_t_7); __Pyx_GIVEREF(__pyx_t_13); PyList_SET_ITEM(__pyx_t_12, 3, __pyx_t_13); __pyx_t_1 = 0; __pyx_t_8 = 0; __pyx_t_7 = 0; __pyx_t_13 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_COMPRESSOR_LIST, __pyx_t_12) < 0) __PYX_ERR(0, 185, __pyx_L1_error) __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; /* "borg/compress.pyx":187 * COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first * * def get_compressor(name, **kwargs): # <<<<<<<<<<<<<< * cls = COMPRESSOR_TABLE[name] * return cls(**kwargs) */ __pyx_t_12 = PyCFunction_NewEx(&__pyx_mdef_4borg_8compress_1get_compressor, NULL, __pyx_n_s_borg_compress); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 187, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); if (PyDict_SetItem(__pyx_d, __pyx_n_s_get_compressor, __pyx_t_12) < 0) __PYX_ERR(0, 187, __pyx_L1_error) __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; /* "borg/compress.pyx":192 * * * class Compressor: # <<<<<<<<<<<<<< * """ * compresses using a compressor with given name and parameters */ __pyx_t_12 = __Pyx_Py3MetaclassPrepare((PyObject *) NULL, __pyx_empty_tuple, __pyx_n_s_Compressor, __pyx_n_s_Compressor, (PyObject *) NULL, __pyx_n_s_borg_compress, __pyx_kp_s_compresses_using_a_compressor_w); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 192, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); /* "borg/compress.pyx":197 * decompresses everything we can handle (autodetect) * """ * def __init__(self, name='null', **kwargs): # <<<<<<<<<<<<<< * self.params = kwargs * self.compressor = get_compressor(name, **self.params) */ __pyx_t_13 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_10Compressor_1__init__, 0, __pyx_n_s_Compressor___init, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__39)); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_13, __pyx_tuple__40); if (__Pyx_SetNameInClass(__pyx_t_12, __pyx_n_s_init, __pyx_t_13) < 0) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":201 * self.compressor = get_compressor(name, **self.params) * * def compress(self, data): # <<<<<<<<<<<<<< * return self.compressor.compress(data) * */ __pyx_t_13 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_10Compressor_3compress, 0, __pyx_n_s_Compressor_compress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__42)); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); if (__Pyx_SetNameInClass(__pyx_t_12, __pyx_n_s_compress, __pyx_t_13) < 0) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":204 * return self.compressor.compress(data) * * def decompress(self, data): # <<<<<<<<<<<<<< * hdr = bytes(data[:2]) # detect() does not work with memoryview * for cls in COMPRESSOR_LIST: */ __pyx_t_13 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_8compress_10Compressor_5decompress, 0, __pyx_n_s_Compressor_decompress, NULL, __pyx_n_s_borg_compress, __pyx_d, ((PyObject *)__pyx_codeobj__44)); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 204, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); if (__Pyx_SetNameInClass(__pyx_t_12, __pyx_n_s_decompress, __pyx_t_13) < 0) __PYX_ERR(0, 204, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; /* "borg/compress.pyx":192 * * * class Compressor: # <<<<<<<<<<<<<< * """ * compresses using a compressor with given name and parameters */ __pyx_t_13 = __Pyx_Py3ClassCreate(((PyObject*)&__Pyx_DefaultClassType), __pyx_n_s_Compressor, __pyx_empty_tuple, __pyx_t_12, NULL, 0, 1); if (unlikely(!__pyx_t_13)) __PYX_ERR(0, 192, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_13); if (PyDict_SetItem(__pyx_d, __pyx_n_s_Compressor, __pyx_t_13) < 0) __PYX_ERR(0, 192, __pyx_L1_error) __Pyx_DECREF(__pyx_t_13); __pyx_t_13 = 0; __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; /* "(tree fragment)":1 * def __pyx_unpickle_CompressorBase(__pyx_type, long __pyx_checksum, __pyx_state): # <<<<<<<<<<<<<< * if __pyx_checksum != 0xd41d8cd: * from pickle import PickleError as __pyx_PickleError */ __pyx_t_12 = PyCFunction_NewEx(&__pyx_mdef_4borg_8compress_3__pyx_unpickle_CompressorBase, NULL, __pyx_n_s_borg_compress); if (unlikely(!__pyx_t_12)) __PYX_ERR(1, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); if (PyDict_SetItem(__pyx_d, __pyx_n_s_pyx_unpickle_CompressorBase, __pyx_t_12) < 0) __PYX_ERR(1, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; /* "borg/compress.pyx":1 * import zlib # <<<<<<<<<<<<<< * try: * import lzma */ __pyx_t_12 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_12) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_XDECREF(__pyx_t_9); __Pyx_XDECREF(__pyx_t_10); __Pyx_XDECREF(__pyx_t_11); __Pyx_XDECREF(__pyx_t_12); __Pyx_XDECREF(__pyx_t_13); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.compress", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.compress"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* PyCFunctionFastCall */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, Py_ssize_t nargs) { PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); int flags = PyCFunction_GET_FLAGS(func); assert(PyCFunction_Check(func)); assert(METH_FASTCALL == (flags & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))); assert(nargs >= 0); assert(nargs == 0 || args != NULL); /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); if ((PY_VERSION_HEX < 0x030700A0) || unlikely(flags & METH_KEYWORDS)) { return (*((__Pyx_PyCFunctionFastWithKeywords)(void*)meth)) (self, args, nargs, NULL); } else { return (*((__Pyx_PyCFunctionFast)(void*)meth)) (self, args, nargs); } } #endif /* PyFunctionFastCall */ #if CYTHON_FAST_PYCALL static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject **fastlocals; Py_ssize_t i; PyObject *result; assert(globals != NULL); /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) { return NULL; } fastlocals = __Pyx_PyFrame_GetLocalsplus(f); for (i = 0; i < na; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } result = PyEval_EvalFrameEx(f,0); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; return result; } #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *closure; #if PY_MAJOR_VERSION >= 3 PyObject *kwdefs; #endif PyObject *kwtuple, **k; PyObject **d; Py_ssize_t nd; Py_ssize_t nk; PyObject *result; assert(kwargs == NULL || PyDict_Check(kwargs)); nk = kwargs ? PyDict_Size(kwargs) : 0; if (Py_EnterRecursiveCall((char*)" while calling a Python object")) { return NULL; } if ( #if PY_MAJOR_VERSION >= 3 co->co_kwonlyargcount == 0 && #endif likely(kwargs == NULL || nk == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { result = __Pyx_PyFunction_FastCallNoKw(co, args, nargs, globals); goto done; } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); result =__Pyx_PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), globals); goto done; } } if (kwargs != NULL) { Py_ssize_t pos, i; kwtuple = PyTuple_New(2 * nk); if (kwtuple == NULL) { result = NULL; goto done; } k = &PyTuple_GET_ITEM(kwtuple, 0); pos = i = 0; while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { Py_INCREF(k[i]); Py_INCREF(k[i+1]); i += 2; } nk = i / 2; } else { kwtuple = NULL; k = NULL; } closure = PyFunction_GET_CLOSURE(func); #if PY_MAJOR_VERSION >= 3 kwdefs = PyFunction_GET_KW_DEFAULTS(func); #endif if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } else { d = NULL; nd = 0; } #if PY_MAJOR_VERSION >= 3 result = PyEval_EvalCodeEx((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, kwdefs, closure); #else result = PyEval_EvalCodeEx(co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, closure); #endif Py_XDECREF(kwtuple); done: Py_LeaveRecursiveCall(); return result; } #endif #endif /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCall2Args */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2) { PyObject *args, *result = NULL; #if CYTHON_FAST_PYCALL if (PyFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyFunction_FastCall(function, args, 2); } #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyCFunction_FastCall(function, args, 2); } #endif args = PyTuple_New(2); if (unlikely(!args)) goto done; Py_INCREF(arg1); PyTuple_SET_ITEM(args, 0, arg1); Py_INCREF(arg2); PyTuple_SET_ITEM(args, 1, arg2); Py_INCREF(function); result = __Pyx_PyObject_Call(function, args, NULL); Py_DECREF(args); Py_DECREF(function); done: return result; } /* PyObjectCallMethO */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg) { PyObject *self, *result; PyCFunction cfunc; cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = cfunc(self, arg); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCallOneArg */ #if CYTHON_COMPILING_IN_CPYTHON static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_New(1); if (unlikely(!args)) return NULL; Py_INCREF(arg); PyTuple_SET_ITEM(args, 0, arg); result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, &arg, 1); } #endif if (likely(PyCFunction_Check(func))) { if (likely(PyCFunction_GET_FLAGS(func) & METH_O)) { return __Pyx_PyObject_CallMethO(func, arg); #if CYTHON_FAST_PYCCALL } else if (PyCFunction_GET_FLAGS(func) & METH_FASTCALL) { return __Pyx_PyCFunction_FastCall(func, &arg, 1); #endif } } return __Pyx__PyObject_CallOneArg(func, arg); } #else static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_Pack(1, arg); if (unlikely(!args)) return NULL; result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } #endif /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* KeywordStringCheck */ static int __Pyx_CheckKeywordStrings( PyObject *kwdict, const char* function_name, int kw_allowed) { PyObject* key = 0; Py_ssize_t pos = 0; #if CYTHON_COMPILING_IN_PYPY if (!kw_allowed && PyDict_Next(kwdict, &pos, &key, 0)) goto invalid_keyword; return 1; #else while (PyDict_Next(kwdict, &pos, &key, 0)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyString_Check(key))) #endif if (unlikely(!PyUnicode_Check(key))) goto invalid_keyword_type; } if ((!kw_allowed) && unlikely(key)) goto invalid_keyword; return 1; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); return 0; #endif invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif return 0; } /* SliceObject */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice(PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** _py_start, PyObject** _py_stop, PyObject** _py_slice, int has_cstart, int has_cstop, CYTHON_UNUSED int wraparound) { #if CYTHON_USE_TYPE_SLOTS PyMappingMethods* mp; #if PY_MAJOR_VERSION < 3 PySequenceMethods* ms = Py_TYPE(obj)->tp_as_sequence; if (likely(ms && ms->sq_slice)) { if (!has_cstart) { if (_py_start && (*_py_start != Py_None)) { cstart = __Pyx_PyIndex_AsSsize_t(*_py_start); if ((cstart == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstart = 0; } if (!has_cstop) { if (_py_stop && (*_py_stop != Py_None)) { cstop = __Pyx_PyIndex_AsSsize_t(*_py_stop); if ((cstop == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstop = PY_SSIZE_T_MAX; } if (wraparound && unlikely((cstart < 0) | (cstop < 0)) && likely(ms->sq_length)) { Py_ssize_t l = ms->sq_length(obj); if (likely(l >= 0)) { if (cstop < 0) { cstop += l; if (cstop < 0) cstop = 0; } if (cstart < 0) { cstart += l; if (cstart < 0) cstart = 0; } } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) goto bad; PyErr_Clear(); } } return ms->sq_slice(obj, cstart, cstop); } #endif mp = Py_TYPE(obj)->tp_as_mapping; if (likely(mp && mp->mp_subscript)) #endif { PyObject* result; PyObject *py_slice, *py_start, *py_stop; if (_py_slice) { py_slice = *_py_slice; } else { PyObject* owned_start = NULL; PyObject* owned_stop = NULL; if (_py_start) { py_start = *_py_start; } else { if (has_cstart) { owned_start = py_start = PyInt_FromSsize_t(cstart); if (unlikely(!py_start)) goto bad; } else py_start = Py_None; } if (_py_stop) { py_stop = *_py_stop; } else { if (has_cstop) { owned_stop = py_stop = PyInt_FromSsize_t(cstop); if (unlikely(!py_stop)) { Py_XDECREF(owned_start); goto bad; } } else py_stop = Py_None; } py_slice = PySlice_New(py_start, py_stop, Py_None); Py_XDECREF(owned_start); Py_XDECREF(owned_stop); if (unlikely(!py_slice)) goto bad; } #if CYTHON_USE_TYPE_SLOTS result = mp->mp_subscript(obj, py_slice); #else result = PyObject_GetItem(obj, py_slice); #endif if (!_py_slice) { Py_DECREF(py_slice); } return result; } PyErr_Format(PyExc_TypeError, "'%.200s' object is unsliceable", Py_TYPE(obj)->tp_name); bad: return NULL; } /* PyErrExceptionMatches */ #if CYTHON_FAST_THREAD_STATE static int __Pyx_PyErr_ExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; icurexc_type; if (exc_type == err) return 1; if (unlikely(!exc_type)) return 0; if (unlikely(PyTuple_Check(err))) return __Pyx_PyErr_ExceptionMatchesTuple(exc_type, err); return __Pyx_PyErr_GivenExceptionMatches(exc_type, err); } #endif /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* GetAttr */ static CYTHON_INLINE PyObject *__Pyx_GetAttr(PyObject *o, PyObject *n) { #if CYTHON_USE_TYPE_SLOTS #if PY_MAJOR_VERSION >= 3 if (likely(PyUnicode_Check(n))) #else if (likely(PyString_Check(n))) #endif return __Pyx_PyObject_GetAttrStr(o, n); #endif return PyObject_GetAttr(o, n); } /* GetAttr3 */ static PyObject *__Pyx_GetAttr3Default(PyObject *d) { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign if (unlikely(!__Pyx_PyErr_ExceptionMatches(PyExc_AttributeError))) return NULL; __Pyx_PyErr_Clear(); Py_INCREF(d); return d; } static CYTHON_INLINE PyObject *__Pyx_GetAttr3(PyObject *o, PyObject *n, PyObject *d) { PyObject *r = __Pyx_GetAttr(o, n); return (likely(r)) ? r : __Pyx_GetAttr3Default(d); } /* GetModuleGlobalName */ #if CYTHON_USE_DICT_VERSIONS static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value) #else static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name) #endif { PyObject *result; #if !CYTHON_AVOID_BORROWED_REFS #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 result = _PyDict_GetItem_KnownHash(__pyx_d, name, ((PyASCIIObject *) name)->hash); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } else if (unlikely(PyErr_Occurred())) { return NULL; } #else result = PyDict_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } #endif #else result = PyObject_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } PyErr_Clear(); #endif return __Pyx_GetBuiltinName(name); } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* RaiseException */ #if PY_MAJOR_VERSION < 3 static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, CYTHON_UNUSED PyObject *cause) { __Pyx_PyThreadState_declare Py_XINCREF(type); if (!value || value == Py_None) value = NULL; else Py_INCREF(value); if (!tb || tb == Py_None) tb = NULL; else { Py_INCREF(tb); if (!PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto raise_error; } } if (PyType_Check(type)) { #if CYTHON_COMPILING_IN_PYPY if (!value) { Py_INCREF(Py_None); value = Py_None; } #endif PyErr_NormalizeException(&type, &value, &tb); } else { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto raise_error; } value = type; type = (PyObject*) Py_TYPE(type); Py_INCREF(type); if (!PyType_IsSubtype((PyTypeObject *)type, (PyTypeObject *)PyExc_BaseException)) { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto raise_error; } } __Pyx_PyThreadState_assign __Pyx_ErrRestore(type, value, tb); return; raise_error: Py_XDECREF(value); Py_XDECREF(type); Py_XDECREF(tb); return; } #else static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause) { PyObject* owned_instance = NULL; if (tb == Py_None) { tb = 0; } else if (tb && !PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto bad; } if (value == Py_None) value = 0; if (PyExceptionInstance_Check(type)) { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto bad; } value = type; type = (PyObject*) Py_TYPE(value); } else if (PyExceptionClass_Check(type)) { PyObject *instance_class = NULL; if (value && PyExceptionInstance_Check(value)) { instance_class = (PyObject*) Py_TYPE(value); if (instance_class != type) { int is_subclass = PyObject_IsSubclass(instance_class, type); if (!is_subclass) { instance_class = NULL; } else if (unlikely(is_subclass == -1)) { goto bad; } else { type = instance_class; } } } if (!instance_class) { PyObject *args; if (!value) args = PyTuple_New(0); else if (PyTuple_Check(value)) { Py_INCREF(value); args = value; } else args = PyTuple_Pack(1, value); if (!args) goto bad; owned_instance = PyObject_Call(type, args, NULL); Py_DECREF(args); if (!owned_instance) goto bad; value = owned_instance; if (!PyExceptionInstance_Check(value)) { PyErr_Format(PyExc_TypeError, "calling %R should have returned an instance of " "BaseException, not %R", type, Py_TYPE(value)); goto bad; } } } else { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto bad; } if (cause) { PyObject *fixed_cause; if (cause == Py_None) { fixed_cause = NULL; } else if (PyExceptionClass_Check(cause)) { fixed_cause = PyObject_CallObject(cause, NULL); if (fixed_cause == NULL) goto bad; } else if (PyExceptionInstance_Check(cause)) { fixed_cause = cause; Py_INCREF(fixed_cause); } else { PyErr_SetString(PyExc_TypeError, "exception causes must derive from " "BaseException"); goto bad; } PyException_SetCause(value, fixed_cause); } PyErr_SetObject(type, value); if (tb) { #if CYTHON_COMPILING_IN_PYPY PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_Fetch(&tmp_type, &tmp_value, &tmp_tb); Py_INCREF(tb); PyErr_Restore(tmp_type, tmp_value, tb); Py_XDECREF(tmp_tb); #else PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* tmp_tb = tstate->curexc_traceback; if (tb != tmp_tb) { Py_INCREF(tb); tstate->curexc_traceback = tb; Py_XDECREF(tmp_tb); } #endif } bad: Py_XDECREF(owned_instance); return; } #endif /* PyIntFromDouble */ #if PY_MAJOR_VERSION < 3 static CYTHON_INLINE PyObject* __Pyx_PyInt_FromDouble(double value) { if (value >= (double)LONG_MIN && value <= (double)LONG_MAX) { return PyInt_FromLong((long)value); } return PyLong_FromDouble(value); } #endif /* PyObjectSetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE int __Pyx_PyObject_SetAttrStr(PyObject* obj, PyObject* attr_name, PyObject* value) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_setattro)) return tp->tp_setattro(obj, attr_name, value); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_setattr)) return tp->tp_setattr(obj, PyString_AS_STRING(attr_name), value); #endif return PyObject_SetAttr(obj, attr_name, value); } #endif /* GetTopmostException */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate) { _PyErr_StackItem *exc_info = tstate->exc_info; while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && exc_info->previous_item != NULL) { exc_info = exc_info->previous_item; } return exc_info; } #endif /* SaveResetException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = __Pyx_PyErr_GetTopmostException(tstate); *type = exc_info->exc_type; *value = exc_info->exc_value; *tb = exc_info->exc_traceback; #else *type = tstate->exc_type; *value = tstate->exc_value; *tb = tstate->exc_traceback; #endif Py_XINCREF(*type); Py_XINCREF(*value); Py_XINCREF(*tb); } static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = type; exc_info->exc_value = value; exc_info->exc_traceback = tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = type; tstate->exc_value = value; tstate->exc_traceback = tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } #endif /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; icurexc_type; local_value = tstate->curexc_value; local_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; #else PyErr_Fetch(&local_type, &local_value, &local_tb); #endif PyErr_NormalizeException(&local_type, &local_value, &local_tb); #if CYTHON_FAST_THREAD_STATE if (unlikely(tstate->curexc_type)) #else if (unlikely(PyErr_Occurred())) #endif goto bad; #if PY_MAJOR_VERSION >= 3 if (local_tb) { if (unlikely(PyException_SetTraceback(local_value, local_tb) < 0)) goto bad; } #endif Py_XINCREF(local_tb); Py_XINCREF(local_type); Py_XINCREF(local_value); *type = local_type; *value = local_value; *tb = local_tb; #if CYTHON_FAST_THREAD_STATE #if CYTHON_USE_EXC_INFO_STACK { _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = local_type; exc_info->exc_value = local_value; exc_info->exc_traceback = local_tb; } #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = local_type; tstate->exc_value = local_value; tstate->exc_traceback = local_tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); #else PyErr_SetExcInfo(local_type, local_value, local_tb); #endif return 0; bad: *type = 0; *value = 0; *tb = 0; Py_XDECREF(local_type); Py_XDECREF(local_value); Py_XDECREF(local_tb); return -1; } /* RaiseTooManyValuesToUnpack */ static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected) { PyErr_Format(PyExc_ValueError, "too many values to unpack (expected %" CYTHON_FORMAT_SSIZE_T "d)", expected); } /* RaiseNeedMoreValuesToUnpack */ static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index) { PyErr_Format(PyExc_ValueError, "need more than %" CYTHON_FORMAT_SSIZE_T "d value%.1s to unpack", index, (index == 1) ? "" : "s"); } /* IterFinish */ static CYTHON_INLINE int __Pyx_IterFinish(void) { #if CYTHON_FAST_THREAD_STATE PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* exc_type = tstate->curexc_type; if (unlikely(exc_type)) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) { PyObject *exc_value, *exc_tb; exc_value = tstate->curexc_value; exc_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; Py_DECREF(exc_type); Py_XDECREF(exc_value); Py_XDECREF(exc_tb); return 0; } else { return -1; } } return 0; #else if (unlikely(PyErr_Occurred())) { if (likely(PyErr_ExceptionMatches(PyExc_StopIteration))) { PyErr_Clear(); return 0; } else { return -1; } } return 0; #endif } /* UnpackItemEndCheck */ static int __Pyx_IternextUnpackEndCheck(PyObject *retval, Py_ssize_t expected) { if (unlikely(retval)) { Py_DECREF(retval); __Pyx_RaiseTooManyValuesError(expected); return -1; } else { return __Pyx_IterFinish(); } return 0; } /* PyIntCompare */ static CYTHON_INLINE PyObject* __Pyx_PyInt_EqObjC(PyObject *op1, PyObject *op2, CYTHON_UNUSED long intval, CYTHON_UNUSED long inplace) { if (op1 == op2) { Py_RETURN_TRUE; } #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(op1))) { const long b = intval; long a = PyInt_AS_LONG(op1); if (a == b) Py_RETURN_TRUE; else Py_RETURN_FALSE; } #endif #if CYTHON_USE_PYLONG_INTERNALS if (likely(PyLong_CheckExact(op1))) { int unequal; unsigned long uintval; Py_ssize_t size = Py_SIZE(op1); const digit* digits = ((PyLongObject*)op1)->ob_digit; if (intval == 0) { if (size == 0) Py_RETURN_TRUE; else Py_RETURN_FALSE; } else if (intval < 0) { if (size >= 0) Py_RETURN_FALSE; intval = -intval; size = -size; } else { if (size <= 0) Py_RETURN_FALSE; } uintval = (unsigned long) intval; #if PyLong_SHIFT * 4 < SIZEOF_LONG*8 if (uintval >> (PyLong_SHIFT * 4)) { unequal = (size != 5) || (digits[0] != (uintval & (unsigned long) PyLong_MASK)) | (digits[1] != ((uintval >> (1 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)) | (digits[2] != ((uintval >> (2 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)) | (digits[3] != ((uintval >> (3 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)) | (digits[4] != ((uintval >> (4 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)); } else #endif #if PyLong_SHIFT * 3 < SIZEOF_LONG*8 if (uintval >> (PyLong_SHIFT * 3)) { unequal = (size != 4) || (digits[0] != (uintval & (unsigned long) PyLong_MASK)) | (digits[1] != ((uintval >> (1 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)) | (digits[2] != ((uintval >> (2 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)) | (digits[3] != ((uintval >> (3 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)); } else #endif #if PyLong_SHIFT * 2 < SIZEOF_LONG*8 if (uintval >> (PyLong_SHIFT * 2)) { unequal = (size != 3) || (digits[0] != (uintval & (unsigned long) PyLong_MASK)) | (digits[1] != ((uintval >> (1 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)) | (digits[2] != ((uintval >> (2 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)); } else #endif #if PyLong_SHIFT * 1 < SIZEOF_LONG*8 if (uintval >> (PyLong_SHIFT * 1)) { unequal = (size != 2) || (digits[0] != (uintval & (unsigned long) PyLong_MASK)) | (digits[1] != ((uintval >> (1 * PyLong_SHIFT)) & (unsigned long) PyLong_MASK)); } else #endif unequal = (size != 1) || (((unsigned long) digits[0]) != (uintval & (unsigned long) PyLong_MASK)); if (unequal == 0) Py_RETURN_TRUE; else Py_RETURN_FALSE; } #endif if (PyFloat_CheckExact(op1)) { const long b = intval; double a = PyFloat_AS_DOUBLE(op1); if ((double)a == (double)b) Py_RETURN_TRUE; else Py_RETURN_FALSE; } return ( PyObject_RichCompare(op1, op2, Py_EQ)); } /* PyIntBinop */ #if !CYTHON_COMPILING_IN_PYPY static PyObject* __Pyx_PyInt_AndObjC(PyObject *op1, PyObject *op2, CYTHON_UNUSED long intval, CYTHON_UNUSED int inplace) { #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(op1))) { const long b = intval; long a = PyInt_AS_LONG(op1); return PyInt_FromLong(a & b); } #endif #if CYTHON_USE_PYLONG_INTERNALS if (likely(PyLong_CheckExact(op1))) { const long b = intval; long a, x; #ifdef HAVE_LONG_LONG const PY_LONG_LONG llb = intval; PY_LONG_LONG lla, llx; #endif const digit* digits = ((PyLongObject*)op1)->ob_digit; const Py_ssize_t size = Py_SIZE(op1); if (likely(__Pyx_sst_abs(size) <= 1)) { a = likely(size) ? digits[0] : 0; if (size == -1) a = -a; } else { switch (size) { case -2: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { a = -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 2: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { a = (long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case -3: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { a = -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((((unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 3: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { a = (long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((((unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case -4: if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { a = -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((((((unsigned PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 4: if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { a = (long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((((((unsigned PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; default: return PyLong_Type.tp_as_number->nb_and(op1, op2); } } x = a & b; return PyLong_FromLong(x); #ifdef HAVE_LONG_LONG long_long: llx = lla & llb; return PyLong_FromLongLong(llx); #endif } #endif return (inplace ? PyNumber_InPlaceAnd : PyNumber_And)(op1, op2); } #endif /* PyIntBinop */ #if !CYTHON_COMPILING_IN_PYPY static PyObject* __Pyx_PyInt_RemainderObjC(PyObject *op1, PyObject *op2, CYTHON_UNUSED long intval, CYTHON_UNUSED int inplace) { #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(op1))) { const long b = intval; long x; long a = PyInt_AS_LONG(op1); x = a % b; x += ((x != 0) & ((x ^ b) < 0)) * b; return PyInt_FromLong(x); } #endif #if CYTHON_USE_PYLONG_INTERNALS if (likely(PyLong_CheckExact(op1))) { const long b = intval; long a, x; #ifdef HAVE_LONG_LONG const PY_LONG_LONG llb = intval; PY_LONG_LONG lla, llx; #endif const digit* digits = ((PyLongObject*)op1)->ob_digit; const Py_ssize_t size = Py_SIZE(op1); if (likely(__Pyx_sst_abs(size) <= 1)) { a = likely(size) ? digits[0] : 0; if (size == -1) a = -a; } else { switch (size) { case -2: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { a = -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 2: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { a = (long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case -3: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { a = -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((((unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 3: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { a = (long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((((unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case -4: if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { a = -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((((((unsigned PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 4: if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { a = (long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((((((unsigned PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; default: return PyLong_Type.tp_as_number->nb_remainder(op1, op2); } } x = a % b; x += ((x != 0) & ((x ^ b) < 0)) * b; return PyLong_FromLong(x); #ifdef HAVE_LONG_LONG long_long: llx = lla % llb; llx += ((llx != 0) & ((llx ^ llb) < 0)) * llb; return PyLong_FromLongLong(llx); #endif } #endif return (inplace ? PyNumber_InPlaceRemainder : PyNumber_Remainder)(op1, op2); } #endif /* GetItemInt */ static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) { PyObject *r; if (!j) return NULL; r = PyObject_GetItem(o, j); Py_DECREF(j); return r; } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyList_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyList_GET_SIZE(o)))) { PyObject *r = PyList_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyTuple_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyList_GET_SIZE(o); if ((!boundscheck) || (likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o))))) { PyObject *r = PyList_GET_ITEM(o, n); Py_INCREF(r); return r; } } else if (PyTuple_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyTuple_GET_SIZE(o); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, n); Py_INCREF(r); return r; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return NULL; PyErr_Clear(); } } return m->sq_item(o, i); } } #else if (is_list || PySequence_Check(o)) { return PySequence_GetItem(o, i); } #endif return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); } /* ObjectGetItem */ #if CYTHON_USE_TYPE_SLOTS static PyObject *__Pyx_PyObject_GetIndex(PyObject *obj, PyObject* index) { PyObject *runerr; Py_ssize_t key_value; PySequenceMethods *m = Py_TYPE(obj)->tp_as_sequence; if (unlikely(!(m && m->sq_item))) { PyErr_Format(PyExc_TypeError, "'%.200s' object is not subscriptable", Py_TYPE(obj)->tp_name); return NULL; } key_value = __Pyx_PyIndex_AsSsize_t(index); if (likely(key_value != -1 || !(runerr = PyErr_Occurred()))) { return __Pyx_GetItemInt_Fast(obj, key_value, 0, 1, 1); } if (PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError)) { PyErr_Clear(); PyErr_Format(PyExc_IndexError, "cannot fit '%.200s' into an index-sized integer", Py_TYPE(index)->tp_name); } return NULL; } static PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key) { PyMappingMethods *m = Py_TYPE(obj)->tp_as_mapping; if (likely(m && m->mp_subscript)) { return m->mp_subscript(obj, key); } return __Pyx_PyObject_GetIndex(obj, key); } #endif /* Import */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level) { PyObject *empty_list = 0; PyObject *module = 0; PyObject *global_dict = 0; PyObject *empty_dict = 0; PyObject *list; #if PY_MAJOR_VERSION < 3 PyObject *py_import; py_import = __Pyx_PyObject_GetAttrStr(__pyx_b, __pyx_n_s_import); if (!py_import) goto bad; #endif if (from_list) list = from_list; else { empty_list = PyList_New(0); if (!empty_list) goto bad; list = empty_list; } global_dict = PyModule_GetDict(__pyx_m); if (!global_dict) goto bad; empty_dict = PyDict_New(); if (!empty_dict) goto bad; { #if PY_MAJOR_VERSION >= 3 if (level == -1) { if (strchr(__Pyx_MODULE_NAME, '.')) { module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, 1); if (!module) { if (!PyErr_ExceptionMatches(PyExc_ImportError)) goto bad; PyErr_Clear(); } } level = 0; } #endif if (!module) { #if PY_MAJOR_VERSION < 3 PyObject *py_level = PyInt_FromLong(level); if (!py_level) goto bad; module = PyObject_CallFunctionObjArgs(py_import, name, global_dict, empty_dict, list, py_level, (PyObject *)NULL); Py_DECREF(py_level); #else module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, level); #endif } } bad: #if PY_MAJOR_VERSION < 3 Py_XDECREF(py_import); #endif Py_XDECREF(empty_list); Py_XDECREF(empty_dict); return module; } /* ImportFrom */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name) { PyObject* value = __Pyx_PyObject_GetAttrStr(module, name); if (unlikely(!value) && PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Format(PyExc_ImportError, #if PY_MAJOR_VERSION < 3 "cannot import name %.230s", PyString_AS_STRING(name)); #else "cannot import name %S", name); #endif } return value; } /* HasAttr */ static CYTHON_INLINE int __Pyx_HasAttr(PyObject *o, PyObject *n) { PyObject *r; if (unlikely(!__Pyx_PyBaseString_Check(n))) { PyErr_SetString(PyExc_TypeError, "hasattr(): attribute name must be string"); return -1; } r = __Pyx_GetAttr(o, n); if (unlikely(!r)) { PyErr_Clear(); return 0; } else { Py_DECREF(r); return 1; } } /* PyObject_GenericGetAttrNoDict */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject *__Pyx_RaiseGenericGetAttributeError(PyTypeObject *tp, PyObject *attr_name) { PyErr_Format(PyExc_AttributeError, #if PY_MAJOR_VERSION >= 3 "'%.50s' object has no attribute '%U'", tp->tp_name, attr_name); #else "'%.50s' object has no attribute '%.400s'", tp->tp_name, PyString_AS_STRING(attr_name)); #endif return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name) { PyObject *descr; PyTypeObject *tp = Py_TYPE(obj); if (unlikely(!PyString_Check(attr_name))) { return PyObject_GenericGetAttr(obj, attr_name); } assert(!tp->tp_dictoffset); descr = _PyType_Lookup(tp, attr_name); if (unlikely(!descr)) { return __Pyx_RaiseGenericGetAttributeError(tp, attr_name); } Py_INCREF(descr); #if PY_MAJOR_VERSION < 3 if (likely(PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_HAVE_CLASS))) #endif { descrgetfunc f = Py_TYPE(descr)->tp_descr_get; if (unlikely(f)) { PyObject *res = f(descr, obj, (PyObject *)tp); Py_DECREF(descr); return res; } } return descr; } #endif /* PyObject_GenericGetAttr */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name) { if (unlikely(Py_TYPE(obj)->tp_dictoffset)) { return PyObject_GenericGetAttr(obj, attr_name); } return __Pyx_PyObject_GenericGetAttrNoDict(obj, attr_name); } #endif /* SetupReduce */ static int __Pyx_setup_reduce_is_named(PyObject* meth, PyObject* name) { int ret; PyObject *name_attr; name_attr = __Pyx_PyObject_GetAttrStr(meth, __pyx_n_s_name_2); if (likely(name_attr)) { ret = PyObject_RichCompareBool(name_attr, name, Py_EQ); } else { ret = -1; } if (unlikely(ret < 0)) { PyErr_Clear(); ret = 0; } Py_XDECREF(name_attr); return ret; } static int __Pyx_setup_reduce(PyObject* type_obj) { int ret = 0; PyObject *object_reduce = NULL; PyObject *object_reduce_ex = NULL; PyObject *reduce = NULL; PyObject *reduce_ex = NULL; PyObject *reduce_cython = NULL; PyObject *setstate = NULL; PyObject *setstate_cython = NULL; #if CYTHON_USE_PYTYPE_LOOKUP if (_PyType_Lookup((PyTypeObject*)type_obj, __pyx_n_s_getstate)) goto GOOD; #else if (PyObject_HasAttr(type_obj, __pyx_n_s_getstate)) goto GOOD; #endif #if CYTHON_USE_PYTYPE_LOOKUP object_reduce_ex = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #else object_reduce_ex = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #endif reduce_ex = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_ex); if (unlikely(!reduce_ex)) goto BAD; if (reduce_ex == object_reduce_ex) { #if CYTHON_USE_PYTYPE_LOOKUP object_reduce = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #else object_reduce = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #endif reduce = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce); if (unlikely(!reduce)) goto BAD; if (reduce == object_reduce || __Pyx_setup_reduce_is_named(reduce, __pyx_n_s_reduce_cython)) { reduce_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_cython); if (unlikely(!reduce_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce, reduce_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce_cython); if (unlikely(ret < 0)) goto BAD; setstate = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate); if (!setstate) PyErr_Clear(); if (!setstate || __Pyx_setup_reduce_is_named(setstate, __pyx_n_s_setstate_cython)) { setstate_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate_cython); if (unlikely(!setstate_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate, setstate_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate_cython); if (unlikely(ret < 0)) goto BAD; } PyType_Modified((PyTypeObject*)type_obj); } } goto GOOD; BAD: if (!PyErr_Occurred()) PyErr_Format(PyExc_RuntimeError, "Unable to initialize pickling for %s", ((PyTypeObject*)type_obj)->tp_name); ret = -1; GOOD: #if !CYTHON_USE_PYTYPE_LOOKUP Py_XDECREF(object_reduce); Py_XDECREF(object_reduce_ex); #endif Py_XDECREF(reduce); Py_XDECREF(reduce_ex); Py_XDECREF(reduce_cython); Py_XDECREF(setstate); Py_XDECREF(setstate_cython); return ret; } /* ClassMethod */ static PyObject* __Pyx_Method_ClassMethod(PyObject *method) { #if CYTHON_COMPILING_IN_PYPY && PYPY_VERSION_NUM <= 0x05080000 if (PyObject_TypeCheck(method, &PyWrapperDescr_Type)) { return PyClassMethod_New(method); } #else #if CYTHON_COMPILING_IN_PYSTON || CYTHON_COMPILING_IN_PYPY if (PyMethodDescr_Check(method)) #else static PyTypeObject *methoddescr_type = NULL; if (methoddescr_type == NULL) { PyObject *meth = PyObject_GetAttrString((PyObject*)&PyList_Type, "append"); if (!meth) return NULL; methoddescr_type = Py_TYPE(meth); Py_DECREF(meth); } if (__Pyx_TypeCheck(method, methoddescr_type)) #endif { PyMethodDescrObject *descr = (PyMethodDescrObject *)method; #if PY_VERSION_HEX < 0x03020000 PyTypeObject *d_type = descr->d_type; #else PyTypeObject *d_type = descr->d_common.d_type; #endif return PyDescr_NewClassMethod(d_type, descr->d_method); } #endif else if (PyMethod_Check(method)) { return PyClassMethod_New(PyMethod_GET_FUNCTION(method)); } else if (PyCFunction_Check(method)) { return PyClassMethod_New(method); } #ifdef __Pyx_CyFunction_USED else if (__Pyx_CyFunction_Check(method)) { return PyClassMethod_New(method); } #endif PyErr_SetString(PyExc_TypeError, "Class-level classmethod() can only be called on " "a method_descriptor or instance method."); return NULL; } /* GetNameInClass */ static PyObject *__Pyx_GetGlobalNameAfterAttributeLookup(PyObject *name) { PyObject *result; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign if (unlikely(!__Pyx_PyErr_ExceptionMatches(PyExc_AttributeError))) return NULL; __Pyx_PyErr_Clear(); __Pyx_GetModuleGlobalNameUncached(result, name); return result; } static PyObject *__Pyx__GetNameInClass(PyObject *nmspace, PyObject *name) { PyObject *result; result = __Pyx_PyObject_GetAttrStr(nmspace, name); if (!result) { result = __Pyx_GetGlobalNameAfterAttributeLookup(name); } return result; } /* CalculateMetaclass */ static PyObject *__Pyx_CalculateMetaclass(PyTypeObject *metaclass, PyObject *bases) { Py_ssize_t i, nbases = PyTuple_GET_SIZE(bases); for (i=0; i < nbases; i++) { PyTypeObject *tmptype; PyObject *tmp = PyTuple_GET_ITEM(bases, i); tmptype = Py_TYPE(tmp); #if PY_MAJOR_VERSION < 3 if (tmptype == &PyClass_Type) continue; #endif if (!metaclass) { metaclass = tmptype; continue; } if (PyType_IsSubtype(metaclass, tmptype)) continue; if (PyType_IsSubtype(tmptype, metaclass)) { metaclass = tmptype; continue; } PyErr_SetString(PyExc_TypeError, "metaclass conflict: " "the metaclass of a derived class " "must be a (non-strict) subclass " "of the metaclasses of all its bases"); return NULL; } if (!metaclass) { #if PY_MAJOR_VERSION < 3 metaclass = &PyClass_Type; #else metaclass = &PyType_Type; #endif } Py_INCREF((PyObject*) metaclass); return (PyObject*) metaclass; } /* FetchCommonType */ static PyTypeObject* __Pyx_FetchCommonType(PyTypeObject* type) { PyObject* fake_module; PyTypeObject* cached_type = NULL; fake_module = PyImport_AddModule((char*) "_cython_" CYTHON_ABI); if (!fake_module) return NULL; Py_INCREF(fake_module); cached_type = (PyTypeObject*) PyObject_GetAttrString(fake_module, type->tp_name); if (cached_type) { if (!PyType_Check((PyObject*)cached_type)) { PyErr_Format(PyExc_TypeError, "Shared Cython type %.200s is not a type object", type->tp_name); goto bad; } if (cached_type->tp_basicsize != type->tp_basicsize) { PyErr_Format(PyExc_TypeError, "Shared Cython type %.200s has the wrong size, try recompiling", type->tp_name); goto bad; } } else { if (!PyErr_ExceptionMatches(PyExc_AttributeError)) goto bad; PyErr_Clear(); if (PyType_Ready(type) < 0) goto bad; if (PyObject_SetAttrString(fake_module, type->tp_name, (PyObject*) type) < 0) goto bad; Py_INCREF(type); cached_type = type; } done: Py_DECREF(fake_module); return cached_type; bad: Py_XDECREF(cached_type); cached_type = NULL; goto done; } /* CythonFunction */ #include static PyObject * __Pyx_CyFunction_get_doc(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *closure) { if (unlikely(op->func_doc == NULL)) { if (op->func.m_ml->ml_doc) { #if PY_MAJOR_VERSION >= 3 op->func_doc = PyUnicode_FromString(op->func.m_ml->ml_doc); #else op->func_doc = PyString_FromString(op->func.m_ml->ml_doc); #endif if (unlikely(op->func_doc == NULL)) return NULL; } else { Py_INCREF(Py_None); return Py_None; } } Py_INCREF(op->func_doc); return op->func_doc; } static int __Pyx_CyFunction_set_doc(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp = op->func_doc; if (value == NULL) { value = Py_None; } Py_INCREF(value); op->func_doc = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_name(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { if (unlikely(op->func_name == NULL)) { #if PY_MAJOR_VERSION >= 3 op->func_name = PyUnicode_InternFromString(op->func.m_ml->ml_name); #else op->func_name = PyString_InternFromString(op->func.m_ml->ml_name); #endif if (unlikely(op->func_name == NULL)) return NULL; } Py_INCREF(op->func_name); return op->func_name; } static int __Pyx_CyFunction_set_name(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp; #if PY_MAJOR_VERSION >= 3 if (unlikely(value == NULL || !PyUnicode_Check(value))) #else if (unlikely(value == NULL || !PyString_Check(value))) #endif { PyErr_SetString(PyExc_TypeError, "__name__ must be set to a string object"); return -1; } tmp = op->func_name; Py_INCREF(value); op->func_name = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_qualname(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { Py_INCREF(op->func_qualname); return op->func_qualname; } static int __Pyx_CyFunction_set_qualname(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp; #if PY_MAJOR_VERSION >= 3 if (unlikely(value == NULL || !PyUnicode_Check(value))) #else if (unlikely(value == NULL || !PyString_Check(value))) #endif { PyErr_SetString(PyExc_TypeError, "__qualname__ must be set to a string object"); return -1; } tmp = op->func_qualname; Py_INCREF(value); op->func_qualname = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_self(__pyx_CyFunctionObject *m, CYTHON_UNUSED void *closure) { PyObject *self; self = m->func_closure; if (self == NULL) self = Py_None; Py_INCREF(self); return self; } static PyObject * __Pyx_CyFunction_get_dict(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { if (unlikely(op->func_dict == NULL)) { op->func_dict = PyDict_New(); if (unlikely(op->func_dict == NULL)) return NULL; } Py_INCREF(op->func_dict); return op->func_dict; } static int __Pyx_CyFunction_set_dict(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp; if (unlikely(value == NULL)) { PyErr_SetString(PyExc_TypeError, "function's dictionary may not be deleted"); return -1; } if (unlikely(!PyDict_Check(value))) { PyErr_SetString(PyExc_TypeError, "setting function's dictionary to a non-dict"); return -1; } tmp = op->func_dict; Py_INCREF(value); op->func_dict = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_globals(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { Py_INCREF(op->func_globals); return op->func_globals; } static PyObject * __Pyx_CyFunction_get_closure(CYTHON_UNUSED __pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { Py_INCREF(Py_None); return Py_None; } static PyObject * __Pyx_CyFunction_get_code(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = (op->func_code) ? op->func_code : Py_None; Py_INCREF(result); return result; } static int __Pyx_CyFunction_init_defaults(__pyx_CyFunctionObject *op) { int result = 0; PyObject *res = op->defaults_getter((PyObject *) op); if (unlikely(!res)) return -1; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS op->defaults_tuple = PyTuple_GET_ITEM(res, 0); Py_INCREF(op->defaults_tuple); op->defaults_kwdict = PyTuple_GET_ITEM(res, 1); Py_INCREF(op->defaults_kwdict); #else op->defaults_tuple = PySequence_ITEM(res, 0); if (unlikely(!op->defaults_tuple)) result = -1; else { op->defaults_kwdict = PySequence_ITEM(res, 1); if (unlikely(!op->defaults_kwdict)) result = -1; } #endif Py_DECREF(res); return result; } static int __Pyx_CyFunction_set_defaults(__pyx_CyFunctionObject *op, PyObject* value, CYTHON_UNUSED void *context) { PyObject* tmp; if (!value) { value = Py_None; } else if (value != Py_None && !PyTuple_Check(value)) { PyErr_SetString(PyExc_TypeError, "__defaults__ must be set to a tuple object"); return -1; } Py_INCREF(value); tmp = op->defaults_tuple; op->defaults_tuple = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_defaults(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = op->defaults_tuple; if (unlikely(!result)) { if (op->defaults_getter) { if (__Pyx_CyFunction_init_defaults(op) < 0) return NULL; result = op->defaults_tuple; } else { result = Py_None; } } Py_INCREF(result); return result; } static int __Pyx_CyFunction_set_kwdefaults(__pyx_CyFunctionObject *op, PyObject* value, CYTHON_UNUSED void *context) { PyObject* tmp; if (!value) { value = Py_None; } else if (value != Py_None && !PyDict_Check(value)) { PyErr_SetString(PyExc_TypeError, "__kwdefaults__ must be set to a dict object"); return -1; } Py_INCREF(value); tmp = op->defaults_kwdict; op->defaults_kwdict = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_kwdefaults(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = op->defaults_kwdict; if (unlikely(!result)) { if (op->defaults_getter) { if (__Pyx_CyFunction_init_defaults(op) < 0) return NULL; result = op->defaults_kwdict; } else { result = Py_None; } } Py_INCREF(result); return result; } static int __Pyx_CyFunction_set_annotations(__pyx_CyFunctionObject *op, PyObject* value, CYTHON_UNUSED void *context) { PyObject* tmp; if (!value || value == Py_None) { value = NULL; } else if (!PyDict_Check(value)) { PyErr_SetString(PyExc_TypeError, "__annotations__ must be set to a dict object"); return -1; } Py_XINCREF(value); tmp = op->func_annotations; op->func_annotations = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_annotations(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = op->func_annotations; if (unlikely(!result)) { result = PyDict_New(); if (unlikely(!result)) return NULL; op->func_annotations = result; } Py_INCREF(result); return result; } static PyGetSetDef __pyx_CyFunction_getsets[] = { {(char *) "func_doc", (getter)__Pyx_CyFunction_get_doc, (setter)__Pyx_CyFunction_set_doc, 0, 0}, {(char *) "__doc__", (getter)__Pyx_CyFunction_get_doc, (setter)__Pyx_CyFunction_set_doc, 0, 0}, {(char *) "func_name", (getter)__Pyx_CyFunction_get_name, (setter)__Pyx_CyFunction_set_name, 0, 0}, {(char *) "__name__", (getter)__Pyx_CyFunction_get_name, (setter)__Pyx_CyFunction_set_name, 0, 0}, {(char *) "__qualname__", (getter)__Pyx_CyFunction_get_qualname, (setter)__Pyx_CyFunction_set_qualname, 0, 0}, {(char *) "__self__", (getter)__Pyx_CyFunction_get_self, 0, 0, 0}, {(char *) "func_dict", (getter)__Pyx_CyFunction_get_dict, (setter)__Pyx_CyFunction_set_dict, 0, 0}, {(char *) "__dict__", (getter)__Pyx_CyFunction_get_dict, (setter)__Pyx_CyFunction_set_dict, 0, 0}, {(char *) "func_globals", (getter)__Pyx_CyFunction_get_globals, 0, 0, 0}, {(char *) "__globals__", (getter)__Pyx_CyFunction_get_globals, 0, 0, 0}, {(char *) "func_closure", (getter)__Pyx_CyFunction_get_closure, 0, 0, 0}, {(char *) "__closure__", (getter)__Pyx_CyFunction_get_closure, 0, 0, 0}, {(char *) "func_code", (getter)__Pyx_CyFunction_get_code, 0, 0, 0}, {(char *) "__code__", (getter)__Pyx_CyFunction_get_code, 0, 0, 0}, {(char *) "func_defaults", (getter)__Pyx_CyFunction_get_defaults, (setter)__Pyx_CyFunction_set_defaults, 0, 0}, {(char *) "__defaults__", (getter)__Pyx_CyFunction_get_defaults, (setter)__Pyx_CyFunction_set_defaults, 0, 0}, {(char *) "__kwdefaults__", (getter)__Pyx_CyFunction_get_kwdefaults, (setter)__Pyx_CyFunction_set_kwdefaults, 0, 0}, {(char *) "__annotations__", (getter)__Pyx_CyFunction_get_annotations, (setter)__Pyx_CyFunction_set_annotations, 0, 0}, {0, 0, 0, 0, 0} }; static PyMemberDef __pyx_CyFunction_members[] = { {(char *) "__module__", T_OBJECT, offsetof(PyCFunctionObject, m_module), PY_WRITE_RESTRICTED, 0}, {0, 0, 0, 0, 0} }; static PyObject * __Pyx_CyFunction_reduce(__pyx_CyFunctionObject *m, CYTHON_UNUSED PyObject *args) { #if PY_MAJOR_VERSION >= 3 return PyUnicode_FromString(m->func.m_ml->ml_name); #else return PyString_FromString(m->func.m_ml->ml_name); #endif } static PyMethodDef __pyx_CyFunction_methods[] = { {"__reduce__", (PyCFunction)__Pyx_CyFunction_reduce, METH_VARARGS, 0}, {0, 0, 0, 0} }; #if PY_VERSION_HEX < 0x030500A0 #define __Pyx_CyFunction_weakreflist(cyfunc) ((cyfunc)->func_weakreflist) #else #define __Pyx_CyFunction_weakreflist(cyfunc) ((cyfunc)->func.m_weakreflist) #endif static PyObject *__Pyx_CyFunction_New(PyTypeObject *type, PyMethodDef *ml, int flags, PyObject* qualname, PyObject *closure, PyObject *module, PyObject* globals, PyObject* code) { __pyx_CyFunctionObject *op = PyObject_GC_New(__pyx_CyFunctionObject, type); if (op == NULL) return NULL; op->flags = flags; __Pyx_CyFunction_weakreflist(op) = NULL; op->func.m_ml = ml; op->func.m_self = (PyObject *) op; Py_XINCREF(closure); op->func_closure = closure; Py_XINCREF(module); op->func.m_module = module; op->func_dict = NULL; op->func_name = NULL; Py_INCREF(qualname); op->func_qualname = qualname; op->func_doc = NULL; op->func_classobj = NULL; op->func_globals = globals; Py_INCREF(op->func_globals); Py_XINCREF(code); op->func_code = code; op->defaults_pyobjects = 0; op->defaults = NULL; op->defaults_tuple = NULL; op->defaults_kwdict = NULL; op->defaults_getter = NULL; op->func_annotations = NULL; PyObject_GC_Track(op); return (PyObject *) op; } static int __Pyx_CyFunction_clear(__pyx_CyFunctionObject *m) { Py_CLEAR(m->func_closure); Py_CLEAR(m->func.m_module); Py_CLEAR(m->func_dict); Py_CLEAR(m->func_name); Py_CLEAR(m->func_qualname); Py_CLEAR(m->func_doc); Py_CLEAR(m->func_globals); Py_CLEAR(m->func_code); Py_CLEAR(m->func_classobj); Py_CLEAR(m->defaults_tuple); Py_CLEAR(m->defaults_kwdict); Py_CLEAR(m->func_annotations); if (m->defaults) { PyObject **pydefaults = __Pyx_CyFunction_Defaults(PyObject *, m); int i; for (i = 0; i < m->defaults_pyobjects; i++) Py_XDECREF(pydefaults[i]); PyObject_Free(m->defaults); m->defaults = NULL; } return 0; } static void __Pyx__CyFunction_dealloc(__pyx_CyFunctionObject *m) { if (__Pyx_CyFunction_weakreflist(m) != NULL) PyObject_ClearWeakRefs((PyObject *) m); __Pyx_CyFunction_clear(m); PyObject_GC_Del(m); } static void __Pyx_CyFunction_dealloc(__pyx_CyFunctionObject *m) { PyObject_GC_UnTrack(m); __Pyx__CyFunction_dealloc(m); } static int __Pyx_CyFunction_traverse(__pyx_CyFunctionObject *m, visitproc visit, void *arg) { Py_VISIT(m->func_closure); Py_VISIT(m->func.m_module); Py_VISIT(m->func_dict); Py_VISIT(m->func_name); Py_VISIT(m->func_qualname); Py_VISIT(m->func_doc); Py_VISIT(m->func_globals); Py_VISIT(m->func_code); Py_VISIT(m->func_classobj); Py_VISIT(m->defaults_tuple); Py_VISIT(m->defaults_kwdict); if (m->defaults) { PyObject **pydefaults = __Pyx_CyFunction_Defaults(PyObject *, m); int i; for (i = 0; i < m->defaults_pyobjects; i++) Py_VISIT(pydefaults[i]); } return 0; } static PyObject *__Pyx_CyFunction_descr_get(PyObject *func, PyObject *obj, PyObject *type) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; if (m->flags & __Pyx_CYFUNCTION_STATICMETHOD) { Py_INCREF(func); return func; } if (m->flags & __Pyx_CYFUNCTION_CLASSMETHOD) { if (type == NULL) type = (PyObject *)(Py_TYPE(obj)); return __Pyx_PyMethod_New(func, type, (PyObject *)(Py_TYPE(type))); } if (obj == Py_None) obj = NULL; return __Pyx_PyMethod_New(func, obj, type); } static PyObject* __Pyx_CyFunction_repr(__pyx_CyFunctionObject *op) { #if PY_MAJOR_VERSION >= 3 return PyUnicode_FromFormat("", op->func_qualname, (void *)op); #else return PyString_FromFormat("", PyString_AsString(op->func_qualname), (void *)op); #endif } static PyObject * __Pyx_CyFunction_CallMethod(PyObject *func, PyObject *self, PyObject *arg, PyObject *kw) { PyCFunctionObject* f = (PyCFunctionObject*)func; PyCFunction meth = f->m_ml->ml_meth; Py_ssize_t size; switch (f->m_ml->ml_flags & (METH_VARARGS | METH_KEYWORDS | METH_NOARGS | METH_O)) { case METH_VARARGS: if (likely(kw == NULL || PyDict_Size(kw) == 0)) return (*meth)(self, arg); break; case METH_VARARGS | METH_KEYWORDS: return (*(PyCFunctionWithKeywords)(void*)meth)(self, arg, kw); case METH_NOARGS: if (likely(kw == NULL || PyDict_Size(kw) == 0)) { size = PyTuple_GET_SIZE(arg); if (likely(size == 0)) return (*meth)(self, NULL); PyErr_Format(PyExc_TypeError, "%.200s() takes no arguments (%" CYTHON_FORMAT_SSIZE_T "d given)", f->m_ml->ml_name, size); return NULL; } break; case METH_O: if (likely(kw == NULL || PyDict_Size(kw) == 0)) { size = PyTuple_GET_SIZE(arg); if (likely(size == 1)) { PyObject *result, *arg0; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS arg0 = PyTuple_GET_ITEM(arg, 0); #else arg0 = PySequence_ITEM(arg, 0); if (unlikely(!arg0)) return NULL; #endif result = (*meth)(self, arg0); #if !(CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS) Py_DECREF(arg0); #endif return result; } PyErr_Format(PyExc_TypeError, "%.200s() takes exactly one argument (%" CYTHON_FORMAT_SSIZE_T "d given)", f->m_ml->ml_name, size); return NULL; } break; default: PyErr_SetString(PyExc_SystemError, "Bad call flags in " "__Pyx_CyFunction_Call. METH_OLDARGS is no " "longer supported!"); return NULL; } PyErr_Format(PyExc_TypeError, "%.200s() takes no keyword arguments", f->m_ml->ml_name); return NULL; } static CYTHON_INLINE PyObject *__Pyx_CyFunction_Call(PyObject *func, PyObject *arg, PyObject *kw) { return __Pyx_CyFunction_CallMethod(func, ((PyCFunctionObject*)func)->m_self, arg, kw); } static PyObject *__Pyx_CyFunction_CallAsMethod(PyObject *func, PyObject *args, PyObject *kw) { PyObject *result; __pyx_CyFunctionObject *cyfunc = (__pyx_CyFunctionObject *) func; if ((cyfunc->flags & __Pyx_CYFUNCTION_CCLASS) && !(cyfunc->flags & __Pyx_CYFUNCTION_STATICMETHOD)) { Py_ssize_t argc; PyObject *new_args; PyObject *self; argc = PyTuple_GET_SIZE(args); new_args = PyTuple_GetSlice(args, 1, argc); if (unlikely(!new_args)) return NULL; self = PyTuple_GetItem(args, 0); if (unlikely(!self)) { Py_DECREF(new_args); return NULL; } result = __Pyx_CyFunction_CallMethod(func, self, new_args, kw); Py_DECREF(new_args); } else { result = __Pyx_CyFunction_Call(func, args, kw); } return result; } static PyTypeObject __pyx_CyFunctionType_type = { PyVarObject_HEAD_INIT(0, 0) "cython_function_or_method", sizeof(__pyx_CyFunctionObject), 0, (destructor) __Pyx_CyFunction_dealloc, 0, 0, 0, #if PY_MAJOR_VERSION < 3 0, #else 0, #endif (reprfunc) __Pyx_CyFunction_repr, 0, 0, 0, 0, __Pyx_CyFunction_CallAsMethod, 0, 0, 0, 0, Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, 0, (traverseproc) __Pyx_CyFunction_traverse, (inquiry) __Pyx_CyFunction_clear, 0, #if PY_VERSION_HEX < 0x030500A0 offsetof(__pyx_CyFunctionObject, func_weakreflist), #else offsetof(PyCFunctionObject, m_weakreflist), #endif 0, 0, __pyx_CyFunction_methods, __pyx_CyFunction_members, __pyx_CyFunction_getsets, 0, 0, __Pyx_CyFunction_descr_get, 0, offsetof(__pyx_CyFunctionObject, func_dict), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, #if PY_VERSION_HEX >= 0x030400a1 0, #endif }; static int __pyx_CyFunction_init(void) { __pyx_CyFunctionType = __Pyx_FetchCommonType(&__pyx_CyFunctionType_type); if (unlikely(__pyx_CyFunctionType == NULL)) { return -1; } return 0; } static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->defaults = PyObject_Malloc(size); if (unlikely(!m->defaults)) return PyErr_NoMemory(); memset(m->defaults, 0, size); m->defaults_pyobjects = pyobjects; return m->defaults; } static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->defaults_tuple = tuple; Py_INCREF(tuple); } static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->defaults_kwdict = dict; Py_INCREF(dict); } static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->func_annotations = dict; Py_INCREF(dict); } /* Py3ClassCreate */ static PyObject *__Pyx_Py3MetaclassPrepare(PyObject *metaclass, PyObject *bases, PyObject *name, PyObject *qualname, PyObject *mkw, PyObject *modname, PyObject *doc) { PyObject *ns; if (metaclass) { PyObject *prep = __Pyx_PyObject_GetAttrStr(metaclass, __pyx_n_s_prepare); if (prep) { PyObject *pargs = PyTuple_Pack(2, name, bases); if (unlikely(!pargs)) { Py_DECREF(prep); return NULL; } ns = PyObject_Call(prep, pargs, mkw); Py_DECREF(prep); Py_DECREF(pargs); } else { if (unlikely(!PyErr_ExceptionMatches(PyExc_AttributeError))) return NULL; PyErr_Clear(); ns = PyDict_New(); } } else { ns = PyDict_New(); } if (unlikely(!ns)) return NULL; if (unlikely(PyObject_SetItem(ns, __pyx_n_s_module, modname) < 0)) goto bad; if (unlikely(PyObject_SetItem(ns, __pyx_n_s_qualname, qualname) < 0)) goto bad; if (unlikely(doc && PyObject_SetItem(ns, __pyx_n_s_doc, doc) < 0)) goto bad; return ns; bad: Py_DECREF(ns); return NULL; } static PyObject *__Pyx_Py3ClassCreate(PyObject *metaclass, PyObject *name, PyObject *bases, PyObject *dict, PyObject *mkw, int calculate_metaclass, int allow_py2_metaclass) { PyObject *result, *margs; PyObject *owned_metaclass = NULL; if (allow_py2_metaclass) { owned_metaclass = PyObject_GetItem(dict, __pyx_n_s_metaclass); if (owned_metaclass) { metaclass = owned_metaclass; } else if (likely(PyErr_ExceptionMatches(PyExc_KeyError))) { PyErr_Clear(); } else { return NULL; } } if (calculate_metaclass && (!metaclass || PyType_Check(metaclass))) { metaclass = __Pyx_CalculateMetaclass((PyTypeObject*) metaclass, bases); Py_XDECREF(owned_metaclass); if (unlikely(!metaclass)) return NULL; owned_metaclass = metaclass; } margs = PyTuple_Pack(3, name, bases, dict); if (unlikely(!margs)) { result = NULL; } else { result = PyObject_Call(metaclass, margs, mkw); Py_DECREF(margs); } Py_XDECREF(owned_metaclass); return result; } /* CyFunctionClassCell */ static int __Pyx_CyFunction_InitClassCell(PyObject *cyfunctions, PyObject *classobj) { Py_ssize_t i, count = PyList_GET_SIZE(cyfunctions); for (i = 0; i < count; i++) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS PyList_GET_ITEM(cyfunctions, i); #else PySequence_ITEM(cyfunctions, i); if (unlikely(!m)) return -1; #endif Py_INCREF(classobj); m->func_classobj = classobj; #if !(CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS) Py_DECREF((PyObject*)m); #endif } return 0; } /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(int) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(int) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(int) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(int), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* CheckBinaryVersion */ static int __Pyx_check_binary_version(void) { char ctversion[4], rtversion[4]; PyOS_snprintf(ctversion, 4, "%d.%d", PY_MAJOR_VERSION, PY_MINOR_VERSION); PyOS_snprintf(rtversion, 4, "%s", Py_GetVersion()); if (ctversion[0] != rtversion[0] || ctversion[2] != rtversion[2]) { char message[200]; PyOS_snprintf(message, sizeof(message), "compiletime version %s of module '%.100s' " "does not match runtime version %s", ctversion, __Pyx_MODULE_NAME, rtversion); return PyErr_WarnEx(NULL, message, 1); } return 0; } /* InitStrings */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t) { while (t->p) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/compress.pyx0000644000175000017500000001410613431513133017216 0ustar useruser00000000000000import zlib try: import lzma except ImportError: lzma = None from .helpers import Buffer, DecompressionError API_VERSION = '1.0_01' cdef extern from "lz4.h": int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize) nogil int LZ4_decompress_safe(const char* source, char* dest, int inputSize, int maxOutputSize) nogil int LZ4_compressBound(int inputSize) nogil buffer = Buffer(bytearray, size=0) cdef class CompressorBase: """ base class for all (de)compression classes, also handles compression format auto detection and adding/stripping the ID header (which enable auto detection). """ ID = b'\xFF\xFF' # reserved and not used # overwrite with a unique 2-bytes bytestring in child classes name = 'baseclass' @classmethod def detect(cls, data): return data.startswith(cls.ID) def __init__(self, **kwargs): pass def compress(self, data): # add ID bytes return self.ID + data def decompress(self, data): # strip ID bytes return data[2:] class CNONE(CompressorBase): """ none - no compression, just pass through data """ ID = b'\x00\x00' name = 'none' def compress(self, data): return super().compress(data) def decompress(self, data): data = super().decompress(data) if not isinstance(data, bytes): data = bytes(data) return data class LZ4(CompressorBase): """ raw LZ4 compression / decompression (liblz4). Features: - lz4 is super fast - wrapper releases CPython's GIL to support multithreaded code - uses safe lz4 methods that never go beyond the end of the output buffer """ ID = b'\x01\x00' name = 'lz4' def __init__(self, **kwargs): pass def compress(self, idata): if not isinstance(idata, bytes): idata = bytes(idata) # code below does not work with memoryview cdef int isize = len(idata) cdef int osize cdef char *source = idata cdef char *dest osize = LZ4_compressBound(isize) buf = buffer.get(osize) dest = buf with nogil: osize = LZ4_compress_limitedOutput(source, dest, isize, osize) if not osize: raise Exception('lz4 compress failed') return super().compress(dest[:osize]) def decompress(self, idata): if not isinstance(idata, bytes): idata = bytes(idata) # code below does not work with memoryview idata = super().decompress(idata) cdef int isize = len(idata) cdef int osize cdef int rsize cdef char *source = idata cdef char *dest # a bit more than 8MB is enough for the usual data sizes yielded by the chunker. # allocate more if isize * 3 is already bigger, to avoid having to resize often. osize = max(int(1.1 * 2**23), isize * 3) while True: buf = buffer.get(osize) dest = buf with nogil: rsize = LZ4_decompress_safe(source, dest, isize, osize) if rsize >= 0: break if osize > 2 ** 30: # this is insane, get out of here raise DecompressionError('lz4 decompress failed') # likely the buffer was too small, get a bigger one: osize = int(1.5 * osize) return dest[:rsize] class LZMA(CompressorBase): """ lzma compression / decompression """ ID = b'\x02\x00' name = 'lzma' def __init__(self, level=6, **kwargs): super().__init__(**kwargs) self.level = level if lzma is None: raise ValueError('No lzma support found.') def compress(self, data): # we do not need integrity checks in lzma, we do that already data = lzma.compress(data, preset=self.level, check=lzma.CHECK_NONE) return super().compress(data) def decompress(self, data): data = super().decompress(data) try: return lzma.decompress(data) except lzma.LZMAError as e: raise DecompressionError(str(e)) from None class ZLIB(CompressorBase): """ zlib compression / decompression (python stdlib) """ ID = b'\x08\x00' # not used here, see detect() # avoid all 0x.8.. IDs elsewhere! name = 'zlib' @classmethod def detect(cls, data): # matches misc. patterns 0x.8.. used by zlib cmf, flg = data[:2] is_deflate = cmf & 0x0f == 8 check_ok = (cmf * 256 + flg) % 31 == 0 return check_ok and is_deflate def __init__(self, level=6, **kwargs): super().__init__(**kwargs) self.level = level def compress(self, data): # note: for compatibility no super call, do not add ID bytes return zlib.compress(data, self.level) def decompress(self, data): # note: for compatibility no super call, do not strip ID bytes try: return zlib.decompress(data) except zlib.error as e: raise DecompressionError(str(e)) from None COMPRESSOR_TABLE = { CNONE.name: CNONE, LZ4.name: LZ4, ZLIB.name: ZLIB, LZMA.name: LZMA, } COMPRESSOR_LIST = [LZ4, CNONE, ZLIB, LZMA, ] # check fast stuff first def get_compressor(name, **kwargs): cls = COMPRESSOR_TABLE[name] return cls(**kwargs) class Compressor: """ compresses using a compressor with given name and parameters decompresses everything we can handle (autodetect) """ def __init__(self, name='null', **kwargs): self.params = kwargs self.compressor = get_compressor(name, **self.params) def compress(self, data): return self.compressor.compress(data) def decompress(self, data): hdr = bytes(data[:2]) # detect() does not work with memoryview for cls in COMPRESSOR_LIST: if cls.detect(hdr): return cls(**self.params).decompress(data) else: raise ValueError('No decompressor for this data found: %r.', data[:2]) borgbackup-1.0.13/borg/crypto.c0000644000175000017500000131743113431514213016315 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__crypto #define __PYX_HAVE_API__borg__crypto /* Early includes */ #include #include #include "openssl/rand.h" #include "openssl/evp.h" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "borg/crypto.pyx", "stringsource", }; /*--- Type declarations ---*/ struct __pyx_obj_4borg_6crypto_AES; /* "borg/crypto.pyx":87 * * * cdef class AES: # <<<<<<<<<<<<<< * """A thin wrapper around the OpenSSL EVP cipher API * """ */ struct __pyx_obj_4borg_6crypto_AES { PyObject_HEAD EVP_CIPHER_CTX *ctx; int is_encrypt; unsigned char iv_orig[16]; PY_LONG_LONG blocks; }; /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* GetModuleGlobalName.proto */ #if CYTHON_USE_DICT_VERSIONS #define __Pyx_GetModuleGlobalName(var, name) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } #define __Pyx_GetModuleGlobalNameUncached(var, name) {\ PY_UINT64_T __pyx_dict_version;\ PyObject *__pyx_dict_cached_value;\ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value); #else #define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name) #define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name) static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name); #endif /* PyFunctionFastCall.proto */ #if CYTHON_FAST_PYCALL #define __Pyx_PyFunction_FastCall(func, args, nargs)\ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL) #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #else #define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs) #endif #define __Pyx_BUILD_ASSERT_EXPR(cond)\ (sizeof(char [1 - 2*!(cond)]) - 1) #ifndef Py_MEMBER_SIZE #define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) #endif static size_t __pyx_pyframe_localsplus_offset = 0; #include "frameobject.h" #define __Pxy_PyFrame_Initialize_Offsets()\ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\ (void)(__pyx_pyframe_localsplus_offset = PyFrame_Type.tp_basicsize - Py_MEMBER_SIZE(PyFrameObject, f_localsplus))) #define __Pyx_PyFrame_GetLocalsplus(frame)\ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset)) #endif /* PyCFunctionFastCall.proto */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs); #else #define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL) #endif /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* GetItemInt.proto */ #define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\ __Pyx_GetItemInt_Generic(o, to_py_func(i)))) #define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); #define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j); static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, int wraparound, int boundscheck); /* PyObjectCall2Args.proto */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2); /* PyObjectCallMethO.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg); #endif /* PyObjectCallOneArg.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg); /* RaiseTooManyValuesToUnpack.proto */ static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected); /* RaiseNeedMoreValuesToUnpack.proto */ static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index); /* IterFinish.proto */ static CYTHON_INLINE int __Pyx_IterFinish(void); /* UnpackItemEndCheck.proto */ static int __Pyx_IternextUnpackEndCheck(PyObject *retval, Py_ssize_t expected); /* None.proto */ static CYTHON_INLINE long __Pyx_div_long(long, long); /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* RaiseException.proto */ static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause); /* GetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_GetException(type, value, tb) __Pyx__GetException(__pyx_tstate, type, value, tb) static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb); #endif /* SwapException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSwap(type, value, tb) __Pyx__ExceptionSwap(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb); #endif /* GetTopmostException.proto */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate); #endif /* SaveResetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSave(type, value, tb) __Pyx__ExceptionSave(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #define __Pyx_ExceptionReset(type, value, tb) __Pyx__ExceptionReset(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); #else #define __Pyx_ExceptionSave(type, value, tb) PyErr_GetExcInfo(type, value, tb) #define __Pyx_ExceptionReset(type, value, tb) PyErr_SetExcInfo(type, value, tb) #endif /* PyObjectCallNoArg.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallNoArg(PyObject *func); #else #define __Pyx_PyObject_CallNoArg(func) __Pyx_PyObject_Call(func, __pyx_empty_tuple, NULL) #endif /* PyIntBinop.proto */ #if !CYTHON_COMPILING_IN_PYPY static PyObject* __Pyx_PyInt_AddObjC(PyObject *op1, PyObject *op2, long intval, int inplace); #else #define __Pyx_PyInt_AddObjC(op1, op2, intval, inplace)\ (inplace ? PyNumber_InPlaceAdd(op1, op2) : PyNumber_Add(op1, op2)) #endif /* SliceObject.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice( PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** py_start, PyObject** py_stop, PyObject** py_slice, int has_cstart, int has_cstop, int wraparound); /* PyObject_GenericGetAttrNoDict.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttrNoDict PyObject_GenericGetAttr #endif /* PyObject_GenericGetAttr.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttr PyObject_GenericGetAttr #endif /* SetupReduce.proto */ static int __Pyx_setup_reduce(PyObject* type_obj); /* Import.proto */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level); /* ImportFrom.proto */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name); /* FetchCommonType.proto */ static PyTypeObject* __Pyx_FetchCommonType(PyTypeObject* type); /* CythonFunction.proto */ #define __Pyx_CyFunction_USED 1 #define __Pyx_CYFUNCTION_STATICMETHOD 0x01 #define __Pyx_CYFUNCTION_CLASSMETHOD 0x02 #define __Pyx_CYFUNCTION_CCLASS 0x04 #define __Pyx_CyFunction_GetClosure(f)\ (((__pyx_CyFunctionObject *) (f))->func_closure) #define __Pyx_CyFunction_GetClassObj(f)\ (((__pyx_CyFunctionObject *) (f))->func_classobj) #define __Pyx_CyFunction_Defaults(type, f)\ ((type *)(((__pyx_CyFunctionObject *) (f))->defaults)) #define __Pyx_CyFunction_SetDefaultsGetter(f, g)\ ((__pyx_CyFunctionObject *) (f))->defaults_getter = (g) typedef struct { PyCFunctionObject func; #if PY_VERSION_HEX < 0x030500A0 PyObject *func_weakreflist; #endif PyObject *func_dict; PyObject *func_name; PyObject *func_qualname; PyObject *func_doc; PyObject *func_globals; PyObject *func_code; PyObject *func_closure; PyObject *func_classobj; void *defaults; int defaults_pyobjects; int flags; PyObject *defaults_tuple; PyObject *defaults_kwdict; PyObject *(*defaults_getter)(PyObject *); PyObject *func_annotations; } __pyx_CyFunctionObject; static PyTypeObject *__pyx_CyFunctionType = 0; #define __Pyx_CyFunction_Check(obj) (__Pyx_TypeCheck(obj, __pyx_CyFunctionType)) #define __Pyx_CyFunction_NewEx(ml, flags, qualname, self, module, globals, code)\ __Pyx_CyFunction_New(__pyx_CyFunctionType, ml, flags, qualname, self, module, globals, code) static PyObject *__Pyx_CyFunction_New(PyTypeObject *, PyMethodDef *ml, int flags, PyObject* qualname, PyObject *self, PyObject *module, PyObject *globals, PyObject* code); static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *m, size_t size, int pyobjects); static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *m, PyObject *tuple); static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *m, PyObject *dict); static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *m, PyObject *dict); static int __pyx_CyFunction_init(void); /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_PY_LONG_LONG(PY_LONG_LONG value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE unsigned char __Pyx_PyInt_As_unsigned_char(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE PY_LONG_LONG __Pyx_PyInt_As_PY_LONG_LONG(PyObject *); /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); /* Module declarations from 'libc.string' */ /* Module declarations from 'libc.stdlib' */ /* Module declarations from 'borg.crypto' */ static PyTypeObject *__pyx_ptype_4borg_6crypto_AES = 0; #define __Pyx_MODULE_NAME "borg.crypto" extern int __pyx_module_is_main_borg__crypto; int __pyx_module_is_main_borg__crypto = 0; /* Implementation of 'borg.crypto' */ static PyObject *__pyx_builtin_range; static PyObject *__pyx_builtin_MemoryError; static PyObject *__pyx_builtin_TypeError; static const char __pyx_k_I[] = ">I"; static const char __pyx_k_Q[] = ">Q"; static const char __pyx_k_b[] = "b"; static const char __pyx_k_h[] = "h"; static const char __pyx_k_i[] = "i"; static const char __pyx_k_l[] = "l"; static const char __pyx_k_n[] = "n"; static const char __pyx_k_x[] = "x"; static const char __pyx_k_QQ[] = ">QQ"; static const char __pyx_k_iv[] = "iv"; static const char __pyx_k_AES[] = "AES"; static const char __pyx_k__10[] = ""; static const char __pyx_k_ikm[] = "ikm"; static const char __pyx_k_int[] = "_int"; static const char __pyx_k_key[] = "key"; static const char __pyx_k_msg[] = "msg"; static const char __pyx_k_prk[] = "prk"; static const char __pyx_k_t_n[] = "t_n"; static const char __pyx_k_HMAC[] = "HMAC"; static const char __pyx_k_ceil[] = "ceil"; static const char __pyx_k_hmac[] = "hmac"; static const char __pyx_k_info[] = "info"; static const char __pyx_k_long[] = "_long"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_math[] = "math"; static const char __pyx_k_name[] = "__name__"; static const char __pyx_k_pack[] = "pack"; static const char __pyx_k_salt[] = "salt"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_2long[] = "_2long"; static const char __pyx_k_range[] = "range"; static const char __pyx_k_reset[] = "reset"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_Struct[] = "Struct"; static const char __pyx_k_amount[] = "amount"; static const char __pyx_k_digest[] = "digest"; static const char __pyx_k_import[] = "__import__"; static const char __pyx_k_lambda[] = ""; static const char __pyx_k_length[] = "length"; static const char __pyx_k_little[] = "little"; static const char __pyx_k_offset[] = "offset"; static const char __pyx_k_output[] = "output"; static const char __pyx_k_reduce[] = "__reduce__"; static const char __pyx_k_sha512[] = "sha512"; static const char __pyx_k_struct[] = "struct"; static const char __pyx_k_hashlib[] = "hashlib"; static const char __pyx_k_getstate[] = "__getstate__"; static const char __pyx_k_setstate[] = "__setstate__"; static const char __pyx_k_to_bytes[] = "to_bytes"; static const char __pyx_k_TypeError[] = "TypeError"; static const char __pyx_k_reduce_ex[] = "__reduce_ex__"; static const char __pyx_k_is_encrypt[] = "is_encrypt"; static const char __pyx_k_max_uint64[] = "max_uint64"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_MemoryError[] = "MemoryError"; static const char __pyx_k_borg_crypto[] = "borg.crypto"; static const char __pyx_k_unpack_from[] = "unpack_from"; static const char __pyx_k_bytes_to_int[] = "bytes_to_int"; static const char __pyx_k_increment_iv[] = "increment_iv"; static const char __pyx_k_bytes_to_long[] = "bytes_to_long"; static const char __pyx_k_digest_length[] = "digest_length"; static const char __pyx_k_long_to_bytes[] = "long_to_bytes"; static const char __pyx_k_output_length[] = "output_length"; static const char __pyx_k_reduce_cython[] = "__reduce_cython__"; static const char __pyx_k_bytes16_to_int[] = "bytes16_to_int"; static const char __pyx_k_int_to_bytes16[] = "int_to_bytes16"; static const char __pyx_k_num_aes_blocks[] = "num_aes_blocks"; static const char __pyx_k_borg_crypto_pyx[] = "borg/crypto.pyx"; static const char __pyx_k_setstate_cython[] = "__setstate_cython__"; static const char __pyx_k_hkdf_hmac_sha512[] = "hkdf_hmac_sha512"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_EVP_DecryptFinal_failed[] = "EVP_DecryptFinal failed"; static const char __pyx_k_EVP_EncryptFinal_failed[] = "EVP_EncryptFinal failed"; static const char __pyx_k_EVP_DecryptUpdate_failed[] = "EVP_DecryptUpdate failed"; static const char __pyx_k_EVP_EncryptUpdate_failed[] = "EVP_EncryptUpdate failed"; static const char __pyx_k_EVP_DecryptInit_ex_failed[] = "EVP_DecryptInit_ex failed"; static const char __pyx_k_EVP_EncryptInit_ex_failed[] = "EVP_EncryptInit_ex failed"; static const char __pyx_k_A_thin_OpenSSL_wrapper_This_coul[] = "A thin OpenSSL wrapper\n\nThis could be replaced by PyCrypto maybe?\n"; static const char __pyx_k_no_default___reduce___due_to_non[] = "no default __reduce__ due to non-trivial __cinit__"; static const char __pyx_k_output_length_must_be_255_64_byt[] = "output_length must be <= 255 * 64 bytes"; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_2long; static PyObject *__pyx_n_s_AES; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_kp_s_EVP_DecryptFinal_failed; static PyObject *__pyx_kp_s_EVP_DecryptInit_ex_failed; static PyObject *__pyx_kp_s_EVP_DecryptUpdate_failed; static PyObject *__pyx_kp_s_EVP_EncryptFinal_failed; static PyObject *__pyx_kp_s_EVP_EncryptInit_ex_failed; static PyObject *__pyx_kp_s_EVP_EncryptUpdate_failed; static PyObject *__pyx_n_s_HMAC; static PyObject *__pyx_kp_s_I; static PyObject *__pyx_n_s_MemoryError; static PyObject *__pyx_kp_s_Q; static PyObject *__pyx_kp_s_QQ; static PyObject *__pyx_n_s_Struct; static PyObject *__pyx_n_s_TypeError; static PyObject *__pyx_kp_b__10; static PyObject *__pyx_n_s_amount; static PyObject *__pyx_n_s_b; static PyObject *__pyx_n_s_borg_crypto; static PyObject *__pyx_kp_s_borg_crypto_pyx; static PyObject *__pyx_n_s_bytes16_to_int; static PyObject *__pyx_n_s_bytes_to_int; static PyObject *__pyx_n_s_bytes_to_long; static PyObject *__pyx_n_s_ceil; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_digest; static PyObject *__pyx_n_s_digest_length; static PyObject *__pyx_n_s_getstate; static PyObject *__pyx_n_s_h; static PyObject *__pyx_n_s_hashlib; static PyObject *__pyx_n_s_hkdf_hmac_sha512; static PyObject *__pyx_n_s_hmac; static PyObject *__pyx_n_s_i; static PyObject *__pyx_n_s_ikm; static PyObject *__pyx_n_s_import; static PyObject *__pyx_n_s_increment_iv; static PyObject *__pyx_n_s_info; static PyObject *__pyx_n_s_int; static PyObject *__pyx_n_s_int_to_bytes16; static PyObject *__pyx_n_s_is_encrypt; static PyObject *__pyx_n_s_iv; static PyObject *__pyx_n_s_key; static PyObject *__pyx_n_s_l; static PyObject *__pyx_n_s_lambda; static PyObject *__pyx_n_s_length; static PyObject *__pyx_n_s_little; static PyObject *__pyx_n_s_long; static PyObject *__pyx_n_s_long_to_bytes; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_math; static PyObject *__pyx_n_s_max_uint64; static PyObject *__pyx_n_s_msg; static PyObject *__pyx_n_s_n; static PyObject *__pyx_n_s_name; static PyObject *__pyx_kp_s_no_default___reduce___due_to_non; static PyObject *__pyx_n_s_num_aes_blocks; static PyObject *__pyx_n_s_offset; static PyObject *__pyx_n_s_output; static PyObject *__pyx_n_s_output_length; static PyObject *__pyx_kp_s_output_length_must_be_255_64_byt; static PyObject *__pyx_n_s_pack; static PyObject *__pyx_n_s_prk; static PyObject *__pyx_n_s_range; static PyObject *__pyx_n_s_reduce; static PyObject *__pyx_n_s_reduce_cython; static PyObject *__pyx_n_s_reduce_ex; static PyObject *__pyx_n_s_reset; static PyObject *__pyx_n_s_salt; static PyObject *__pyx_n_s_setstate; static PyObject *__pyx_n_s_setstate_cython; static PyObject *__pyx_n_s_sha512; static PyObject *__pyx_n_s_struct; static PyObject *__pyx_n_s_t_n; static PyObject *__pyx_n_s_test; static PyObject *__pyx_n_s_to_bytes; static PyObject *__pyx_n_s_unpack_from; static PyObject *__pyx_n_s_x; static PyObject *__pyx_lambda_funcdef_4borg_6crypto_lambda(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x, PyObject *__pyx_v_offset); /* proto */ static PyObject *__pyx_lambda_funcdef_4borg_6crypto_lambda1(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x, PyObject *__pyx_v_offset); /* proto */ static PyObject *__pyx_lambda_funcdef_4borg_6crypto_lambda2(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_bytes16_to_int(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_b, PyObject *__pyx_v_offset); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_2int_to_bytes16(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_i); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_4increment_iv(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_iv, PyObject *__pyx_v_amount); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_6num_aes_blocks(CYTHON_UNUSED PyObject *__pyx_self, int __pyx_v_length); /* proto */ static int __pyx_pf_4borg_6crypto_3AES___cinit__(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_is_encrypt, PyObject *__pyx_v_key, PyObject *__pyx_v_iv); /* proto */ static void __pyx_pf_4borg_6crypto_3AES_2__dealloc__(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_3AES_4reset(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_iv); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_3AES_2iv___get__(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_3AES_6encrypt(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_3AES_8decrypt(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_data); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_3AES_10__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_3AES_12__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_pf_4borg_6crypto_8hkdf_hmac_sha512(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_ikm, PyObject *__pyx_v_salt, PyObject *__pyx_v_info, PyObject *__pyx_v_output_length); /* proto */ static PyObject *__pyx_tp_new_4borg_6crypto_AES(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_int_0; static PyObject *__pyx_int_1; static PyObject *__pyx_int_64; static PyObject *__pyx_int_255; static PyObject *__pyx_int_18446744073709551615; static PyObject *__pyx_tuple_; static PyObject *__pyx_tuple__2; static PyObject *__pyx_tuple__3; static PyObject *__pyx_tuple__4; static PyObject *__pyx_tuple__5; static PyObject *__pyx_tuple__6; static PyObject *__pyx_tuple__7; static PyObject *__pyx_tuple__8; static PyObject *__pyx_tuple__9; static PyObject *__pyx_tuple__11; static PyObject *__pyx_tuple__12; static PyObject *__pyx_tuple__13; static PyObject *__pyx_tuple__14; static PyObject *__pyx_tuple__15; static PyObject *__pyx_tuple__16; static PyObject *__pyx_tuple__17; static PyObject *__pyx_tuple__19; static PyObject *__pyx_tuple__21; static PyObject *__pyx_tuple__23; static PyObject *__pyx_tuple__25; static PyObject *__pyx_codeobj__18; static PyObject *__pyx_codeobj__20; static PyObject *__pyx_codeobj__22; static PyObject *__pyx_codeobj__24; static PyObject *__pyx_codeobj__26; /* Late includes */ /* "borg/crypto.pyx":48 * _2long = struct.Struct('>QQ') * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] # <<<<<<<<<<<<<< * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] * long_to_bytes = lambda x: _long.pack(x) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_10lambda(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_6crypto_10lambda = {"lambda", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_6crypto_10lambda, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_6crypto_10lambda(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_x = 0; PyObject *__pyx_v_offset = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("lambda (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_x,&__pyx_n_s_offset,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)((PyObject *)__pyx_int_0)); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_x)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_offset); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "lambda") < 0)) __PYX_ERR(0, 48, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_x = values[0]; __pyx_v_offset = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("lambda", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 48, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.lambda", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_lambda_funcdef_4borg_6crypto_lambda(__pyx_self, __pyx_v_x, __pyx_v_offset); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_lambda_funcdef_4borg_6crypto_lambda(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x, PyObject *__pyx_v_offset) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("lambda", 0); __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_int); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_unpack_from); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; __pyx_t_4 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_4 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_x, __pyx_v_offset}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_x, __pyx_v_offset}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif { __pyx_t_5 = PyTuple_New(2+__pyx_t_4); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); if (__pyx_t_2) { __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_2); __pyx_t_2 = NULL; } __Pyx_INCREF(__pyx_v_x); __Pyx_GIVEREF(__pyx_v_x); PyTuple_SET_ITEM(__pyx_t_5, 0+__pyx_t_4, __pyx_v_x); __Pyx_INCREF(__pyx_v_offset); __Pyx_GIVEREF(__pyx_v_offset); PyTuple_SET_ITEM(__pyx_t_5, 1+__pyx_t_4, __pyx_v_offset); __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_5, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_GetItemInt(__pyx_t_1, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.crypto.lambda", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":49 * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] # <<<<<<<<<<<<<< * long_to_bytes = lambda x: _long.pack(x) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_11lambda1(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_6crypto_11lambda1 = {"lambda1", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_6crypto_11lambda1, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_6crypto_11lambda1(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_x = 0; PyObject *__pyx_v_offset = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("lambda1 (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_x,&__pyx_n_s_offset,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)((PyObject *)__pyx_int_0)); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_x)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_offset); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "lambda1") < 0)) __PYX_ERR(0, 49, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_x = values[0]; __pyx_v_offset = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("lambda1", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 49, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.lambda1", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_lambda_funcdef_4borg_6crypto_lambda1(__pyx_self, __pyx_v_x, __pyx_v_offset); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_lambda_funcdef_4borg_6crypto_lambda1(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x, PyObject *__pyx_v_offset) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("lambda1", 0); __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_long); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_unpack_from); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; __pyx_t_4 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_4 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_x, __pyx_v_offset}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_x, __pyx_v_offset}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif { __pyx_t_5 = PyTuple_New(2+__pyx_t_4); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); if (__pyx_t_2) { __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_2); __pyx_t_2 = NULL; } __Pyx_INCREF(__pyx_v_x); __Pyx_GIVEREF(__pyx_v_x); PyTuple_SET_ITEM(__pyx_t_5, 0+__pyx_t_4, __pyx_v_x); __Pyx_INCREF(__pyx_v_offset); __Pyx_GIVEREF(__pyx_v_offset); PyTuple_SET_ITEM(__pyx_t_5, 1+__pyx_t_4, __pyx_v_offset); __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_5, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_GetItemInt(__pyx_t_1, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.crypto.lambda1", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":50 * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] * long_to_bytes = lambda x: _long.pack(x) # <<<<<<<<<<<<<< * * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_12lambda2(PyObject *__pyx_self, PyObject *__pyx_v_x); /*proto*/ static PyMethodDef __pyx_mdef_4borg_6crypto_12lambda2 = {"lambda2", (PyCFunction)__pyx_pw_4borg_6crypto_12lambda2, METH_O, 0}; static PyObject *__pyx_pw_4borg_6crypto_12lambda2(PyObject *__pyx_self, PyObject *__pyx_v_x) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("lambda2 (wrapper)", 0); __pyx_r = __pyx_lambda_funcdef_4borg_6crypto_lambda2(__pyx_self, ((PyObject *)__pyx_v_x)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_lambda_funcdef_4borg_6crypto_lambda2(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_x) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; __Pyx_RefNannySetupContext("lambda2", 0); __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_long); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_pack); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_x) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_x); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.crypto.lambda2", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":53 * * * def bytes16_to_int(b, offset=0): # <<<<<<<<<<<<<< * h, l = _2long.unpack_from(b, offset) * return (h << 64) + l */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_1bytes16_to_int(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_6crypto_1bytes16_to_int = {"bytes16_to_int", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_6crypto_1bytes16_to_int, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_6crypto_1bytes16_to_int(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_b = 0; PyObject *__pyx_v_offset = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("bytes16_to_int (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_b,&__pyx_n_s_offset,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)__pyx_int_0); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_b)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_offset); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "bytes16_to_int") < 0)) __PYX_ERR(0, 53, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_b = values[0]; __pyx_v_offset = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("bytes16_to_int", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 53, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.bytes16_to_int", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_6crypto_bytes16_to_int(__pyx_self, __pyx_v_b, __pyx_v_offset); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_bytes16_to_int(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_b, PyObject *__pyx_v_offset) { PyObject *__pyx_v_h = NULL; PyObject *__pyx_v_l = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; PyObject *(*__pyx_t_6)(PyObject *); __Pyx_RefNannySetupContext("bytes16_to_int", 0); /* "borg/crypto.pyx":54 * * def bytes16_to_int(b, offset=0): * h, l = _2long.unpack_from(b, offset) # <<<<<<<<<<<<<< * return (h << 64) + l * */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_2long); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_unpack_from); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; __pyx_t_4 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_4 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_b, __pyx_v_offset}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_b, __pyx_v_offset}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif { __pyx_t_5 = PyTuple_New(2+__pyx_t_4); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); if (__pyx_t_2) { __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_2); __pyx_t_2 = NULL; } __Pyx_INCREF(__pyx_v_b); __Pyx_GIVEREF(__pyx_v_b); PyTuple_SET_ITEM(__pyx_t_5, 0+__pyx_t_4, __pyx_v_b); __Pyx_INCREF(__pyx_v_offset); __Pyx_GIVEREF(__pyx_v_offset); PyTuple_SET_ITEM(__pyx_t_5, 1+__pyx_t_4, __pyx_v_offset); __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_5, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if ((likely(PyTuple_CheckExact(__pyx_t_1))) || (PyList_CheckExact(__pyx_t_1))) { PyObject* sequence = __pyx_t_1; Py_ssize_t size = __Pyx_PySequence_SIZE(sequence); if (unlikely(size != 2)) { if (size > 2) __Pyx_RaiseTooManyValuesError(2); else if (size >= 0) __Pyx_RaiseNeedMoreValuesError(size); __PYX_ERR(0, 54, __pyx_L1_error) } #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS if (likely(PyTuple_CheckExact(sequence))) { __pyx_t_3 = PyTuple_GET_ITEM(sequence, 0); __pyx_t_5 = PyTuple_GET_ITEM(sequence, 1); } else { __pyx_t_3 = PyList_GET_ITEM(sequence, 0); __pyx_t_5 = PyList_GET_ITEM(sequence, 1); } __Pyx_INCREF(__pyx_t_3); __Pyx_INCREF(__pyx_t_5); #else __pyx_t_3 = PySequence_ITEM(sequence, 0); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_5 = PySequence_ITEM(sequence, 1); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); #endif __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; } else { Py_ssize_t index = -1; __pyx_t_2 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_6 = Py_TYPE(__pyx_t_2)->tp_iternext; index = 0; __pyx_t_3 = __pyx_t_6(__pyx_t_2); if (unlikely(!__pyx_t_3)) goto __pyx_L3_unpacking_failed; __Pyx_GOTREF(__pyx_t_3); index = 1; __pyx_t_5 = __pyx_t_6(__pyx_t_2); if (unlikely(!__pyx_t_5)) goto __pyx_L3_unpacking_failed; __Pyx_GOTREF(__pyx_t_5); if (__Pyx_IternextUnpackEndCheck(__pyx_t_6(__pyx_t_2), 2) < 0) __PYX_ERR(0, 54, __pyx_L1_error) __pyx_t_6 = NULL; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; goto __pyx_L4_unpacking_done; __pyx_L3_unpacking_failed:; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_6 = NULL; if (__Pyx_IterFinish() == 0) __Pyx_RaiseNeedMoreValuesError(index); __PYX_ERR(0, 54, __pyx_L1_error) __pyx_L4_unpacking_done:; } __pyx_v_h = __pyx_t_3; __pyx_t_3 = 0; __pyx_v_l = __pyx_t_5; __pyx_t_5 = 0; /* "borg/crypto.pyx":55 * def bytes16_to_int(b, offset=0): * h, l = _2long.unpack_from(b, offset) * return (h << 64) + l # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = PyNumber_Lshift(__pyx_v_h, __pyx_int_64); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 55, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_5 = PyNumber_Add(__pyx_t_1, __pyx_v_l); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 55, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_5; __pyx_t_5 = 0; goto __pyx_L0; /* "borg/crypto.pyx":53 * * * def bytes16_to_int(b, offset=0): # <<<<<<<<<<<<<< * h, l = _2long.unpack_from(b, offset) * return (h << 64) + l */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.crypto.bytes16_to_int", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_h); __Pyx_XDECREF(__pyx_v_l); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":58 * * * def int_to_bytes16(i): # <<<<<<<<<<<<<< * max_uint64 = 0xffffffffffffffff * l = i & max_uint64 */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3int_to_bytes16(PyObject *__pyx_self, PyObject *__pyx_v_i); /*proto*/ static PyMethodDef __pyx_mdef_4borg_6crypto_3int_to_bytes16 = {"int_to_bytes16", (PyCFunction)__pyx_pw_4borg_6crypto_3int_to_bytes16, METH_O, 0}; static PyObject *__pyx_pw_4borg_6crypto_3int_to_bytes16(PyObject *__pyx_self, PyObject *__pyx_v_i) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("int_to_bytes16 (wrapper)", 0); __pyx_r = __pyx_pf_4borg_6crypto_2int_to_bytes16(__pyx_self, ((PyObject *)__pyx_v_i)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_2int_to_bytes16(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_i) { PyObject *__pyx_v_max_uint64 = NULL; PyObject *__pyx_v_l = NULL; PyObject *__pyx_v_h = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("int_to_bytes16", 0); /* "borg/crypto.pyx":59 * * def int_to_bytes16(i): * max_uint64 = 0xffffffffffffffff # <<<<<<<<<<<<<< * l = i & max_uint64 * h = (i >> 64) & max_uint64 */ __Pyx_INCREF(__pyx_int_18446744073709551615); __pyx_v_max_uint64 = __pyx_int_18446744073709551615; /* "borg/crypto.pyx":60 * def int_to_bytes16(i): * max_uint64 = 0xffffffffffffffff * l = i & max_uint64 # <<<<<<<<<<<<<< * h = (i >> 64) & max_uint64 * return _2long.pack(h, l) */ __pyx_t_1 = PyNumber_And(__pyx_v_i, __pyx_v_max_uint64); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 60, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_l = __pyx_t_1; __pyx_t_1 = 0; /* "borg/crypto.pyx":61 * max_uint64 = 0xffffffffffffffff * l = i & max_uint64 * h = (i >> 64) & max_uint64 # <<<<<<<<<<<<<< * return _2long.pack(h, l) * */ __pyx_t_1 = PyNumber_Rshift(__pyx_v_i, __pyx_int_64); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 61, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyNumber_And(__pyx_t_1, __pyx_v_max_uint64); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 61, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_h = __pyx_t_2; __pyx_t_2 = 0; /* "borg/crypto.pyx":62 * l = i & max_uint64 * h = (i >> 64) & max_uint64 * return _2long.pack(h, l) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_2long); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 62, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_pack); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 62, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = NULL; __pyx_t_4 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_1 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_1)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_1); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_4 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_1, __pyx_v_h, __pyx_v_l}; __pyx_t_2 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 62, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_GOTREF(__pyx_t_2); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_1, __pyx_v_h, __pyx_v_l}; __pyx_t_2 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_4, 2+__pyx_t_4); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 62, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_GOTREF(__pyx_t_2); } else #endif { __pyx_t_5 = PyTuple_New(2+__pyx_t_4); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 62, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); if (__pyx_t_1) { __Pyx_GIVEREF(__pyx_t_1); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_1); __pyx_t_1 = NULL; } __Pyx_INCREF(__pyx_v_h); __Pyx_GIVEREF(__pyx_v_h); PyTuple_SET_ITEM(__pyx_t_5, 0+__pyx_t_4, __pyx_v_h); __Pyx_INCREF(__pyx_v_l); __Pyx_GIVEREF(__pyx_v_l); PyTuple_SET_ITEM(__pyx_t_5, 1+__pyx_t_4, __pyx_v_l); __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_5, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 62, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_r = __pyx_t_2; __pyx_t_2 = 0; goto __pyx_L0; /* "borg/crypto.pyx":58 * * * def int_to_bytes16(i): # <<<<<<<<<<<<<< * max_uint64 = 0xffffffffffffffff * l = i & max_uint64 */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.crypto.int_to_bytes16", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_max_uint64); __Pyx_XDECREF(__pyx_v_l); __Pyx_XDECREF(__pyx_v_h); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":65 * * * def increment_iv(iv, amount=1): # <<<<<<<<<<<<<< * """ * Increment the IV by the given amount (default 1). */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_5increment_iv(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_6crypto_4increment_iv[] = "\n Increment the IV by the given amount (default 1).\n\n :param iv: input IV, 16 bytes (128 bit)\n :param amount: increment value\n :return: input_IV + amount, 16 bytes (128 bit)\n "; static PyMethodDef __pyx_mdef_4borg_6crypto_5increment_iv = {"increment_iv", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_6crypto_5increment_iv, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_6crypto_4increment_iv}; static PyObject *__pyx_pw_4borg_6crypto_5increment_iv(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_iv = 0; PyObject *__pyx_v_amount = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("increment_iv (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_iv,&__pyx_n_s_amount,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)__pyx_int_1); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_iv)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_amount); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "increment_iv") < 0)) __PYX_ERR(0, 65, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_iv = values[0]; __pyx_v_amount = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("increment_iv", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 65, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.increment_iv", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_6crypto_4increment_iv(__pyx_self, __pyx_v_iv, __pyx_v_amount); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_4increment_iv(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_iv, PyObject *__pyx_v_amount) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("increment_iv", 0); __Pyx_INCREF(__pyx_v_iv); /* "borg/crypto.pyx":73 * :return: input_IV + amount, 16 bytes (128 bit) * """ * assert len(iv) == 16 # <<<<<<<<<<<<<< * iv = bytes16_to_int(iv) * iv += amount */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_iv); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 73, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == 16) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 73, __pyx_L1_error) } } #endif /* "borg/crypto.pyx":74 * """ * assert len(iv) == 16 * iv = bytes16_to_int(iv) # <<<<<<<<<<<<<< * iv += amount * iv = int_to_bytes16(iv) */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_bytes16_to_int); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 74, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_4, __pyx_v_iv) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_iv); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 74, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF_SET(__pyx_v_iv, __pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":75 * assert len(iv) == 16 * iv = bytes16_to_int(iv) * iv += amount # <<<<<<<<<<<<<< * iv = int_to_bytes16(iv) * return iv */ __pyx_t_2 = PyNumber_InPlaceAdd(__pyx_v_iv, __pyx_v_amount); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 75, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF_SET(__pyx_v_iv, __pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":76 * iv = bytes16_to_int(iv) * iv += amount * iv = int_to_bytes16(iv) # <<<<<<<<<<<<<< * return iv * */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_int_to_bytes16); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 76, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_4, __pyx_v_iv) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_iv); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 76, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF_SET(__pyx_v_iv, __pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":77 * iv += amount * iv = int_to_bytes16(iv) * return iv # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(__pyx_v_iv); __pyx_r = __pyx_v_iv; goto __pyx_L0; /* "borg/crypto.pyx":65 * * * def increment_iv(iv, amount=1): # <<<<<<<<<<<<<< * """ * Increment the IV by the given amount (default 1). */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.crypto.increment_iv", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_iv); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":80 * * * def num_aes_blocks(int length): # <<<<<<<<<<<<<< * """Return the number of AES blocks required to encrypt/decrypt *length* bytes of data. * Note: this is only correct for modes without padding, like AES-CTR. */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_7num_aes_blocks(PyObject *__pyx_self, PyObject *__pyx_arg_length); /*proto*/ static char __pyx_doc_4borg_6crypto_6num_aes_blocks[] = "Return the number of AES blocks required to encrypt/decrypt *length* bytes of data.\n Note: this is only correct for modes without padding, like AES-CTR.\n "; static PyMethodDef __pyx_mdef_4borg_6crypto_7num_aes_blocks = {"num_aes_blocks", (PyCFunction)__pyx_pw_4borg_6crypto_7num_aes_blocks, METH_O, __pyx_doc_4borg_6crypto_6num_aes_blocks}; static PyObject *__pyx_pw_4borg_6crypto_7num_aes_blocks(PyObject *__pyx_self, PyObject *__pyx_arg_length) { int __pyx_v_length; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("num_aes_blocks (wrapper)", 0); assert(__pyx_arg_length); { __pyx_v_length = __Pyx_PyInt_As_int(__pyx_arg_length); if (unlikely((__pyx_v_length == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 80, __pyx_L3_error) } goto __pyx_L4_argument_unpacking_done; __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.num_aes_blocks", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_6crypto_6num_aes_blocks(__pyx_self, ((int)__pyx_v_length)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_6num_aes_blocks(CYTHON_UNUSED PyObject *__pyx_self, int __pyx_v_length) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("num_aes_blocks", 0); /* "borg/crypto.pyx":84 * Note: this is only correct for modes without padding, like AES-CTR. * """ * return (length + 15) // 16 # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = __Pyx_PyInt_From_long(__Pyx_div_long((__pyx_v_length + 15), 16)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 84, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/crypto.pyx":80 * * * def num_aes_blocks(int length): # <<<<<<<<<<<<<< * """Return the number of AES blocks required to encrypt/decrypt *length* bytes of data. * Note: this is only correct for modes without padding, like AES-CTR. */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.crypto.num_aes_blocks", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":95 * cdef long long blocks * * def __cinit__(self, is_encrypt, key, iv=None): # <<<<<<<<<<<<<< * self.ctx = EVP_CIPHER_CTX_new() * self.is_encrypt = is_encrypt */ /* Python wrapper */ static int __pyx_pw_4borg_6crypto_3AES_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static int __pyx_pw_4borg_6crypto_3AES_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_is_encrypt = 0; PyObject *__pyx_v_key = 0; PyObject *__pyx_v_iv = 0; int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_is_encrypt,&__pyx_n_s_key,&__pyx_n_s_iv,0}; PyObject* values[3] = {0,0,0}; values[2] = ((PyObject *)Py_None); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_is_encrypt)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 2, 3, 1); __PYX_ERR(0, 95, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_iv); if (value) { values[2] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(0, 95, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_is_encrypt = values[0]; __pyx_v_key = values[1]; __pyx_v_iv = values[2]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 2, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 95, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.AES.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return -1; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_6crypto_3AES___cinit__(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self), __pyx_v_is_encrypt, __pyx_v_key, __pyx_v_iv); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_6crypto_3AES___cinit__(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_is_encrypt, PyObject *__pyx_v_key, PyObject *__pyx_v_iv) { EVP_CIPHER const *__pyx_v_cipher_mode; int __pyx_r; __Pyx_RefNannyDeclarations int __pyx_t_1; int __pyx_t_2; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; __Pyx_RefNannySetupContext("__cinit__", 0); /* "borg/crypto.pyx":96 * * def __cinit__(self, is_encrypt, key, iv=None): * self.ctx = EVP_CIPHER_CTX_new() # <<<<<<<<<<<<<< * self.is_encrypt = is_encrypt * # Set cipher type and mode */ __pyx_v_self->ctx = EVP_CIPHER_CTX_new(); /* "borg/crypto.pyx":97 * def __cinit__(self, is_encrypt, key, iv=None): * self.ctx = EVP_CIPHER_CTX_new() * self.is_encrypt = is_encrypt # <<<<<<<<<<<<<< * # Set cipher type and mode * cipher_mode = EVP_aes_256_ctr() */ __pyx_t_1 = __Pyx_PyInt_As_int(__pyx_v_is_encrypt); if (unlikely((__pyx_t_1 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 97, __pyx_L1_error) __pyx_v_self->is_encrypt = __pyx_t_1; /* "borg/crypto.pyx":99 * self.is_encrypt = is_encrypt * # Set cipher type and mode * cipher_mode = EVP_aes_256_ctr() # <<<<<<<<<<<<<< * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): */ __pyx_v_cipher_mode = EVP_aes_256_ctr(); /* "borg/crypto.pyx":100 * # Set cipher type and mode * cipher_mode = EVP_aes_256_ctr() * if self.is_encrypt: # <<<<<<<<<<<<<< * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_EncryptInit_ex failed') */ __pyx_t_2 = (__pyx_v_self->is_encrypt != 0); if (__pyx_t_2) { /* "borg/crypto.pyx":101 * cipher_mode = EVP_aes_256_ctr() * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt */ __pyx_t_2 = ((!(EVP_EncryptInit_ex(__pyx_v_self->ctx, __pyx_v_cipher_mode, NULL, NULL, NULL) != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":102 * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_EncryptInit_ex failed') # <<<<<<<<<<<<<< * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): */ __pyx_t_3 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple_, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 102, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_Raise(__pyx_t_3, 0, 0, 0); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __PYX_ERR(0, 102, __pyx_L1_error) /* "borg/crypto.pyx":101 * cipher_mode = EVP_aes_256_ctr() * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt */ } /* "borg/crypto.pyx":100 * # Set cipher type and mode * cipher_mode = EVP_aes_256_ctr() * if self.is_encrypt: # <<<<<<<<<<<<<< * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_EncryptInit_ex failed') */ goto __pyx_L3; } /* "borg/crypto.pyx":104 * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): # <<<<<<<<<<<<<< * raise Exception('EVP_DecryptInit_ex failed') * self.reset(key, iv) */ /*else*/ { __pyx_t_2 = ((!(EVP_DecryptInit_ex(__pyx_v_self->ctx, __pyx_v_cipher_mode, NULL, NULL, NULL) != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":105 * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_DecryptInit_ex failed') # <<<<<<<<<<<<<< * self.reset(key, iv) * */ __pyx_t_3 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_Raise(__pyx_t_3, 0, 0, 0); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __PYX_ERR(0, 105, __pyx_L1_error) /* "borg/crypto.pyx":104 * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): # <<<<<<<<<<<<<< * raise Exception('EVP_DecryptInit_ex failed') * self.reset(key, iv) */ } } __pyx_L3:; /* "borg/crypto.pyx":106 * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_DecryptInit_ex failed') * self.reset(key, iv) # <<<<<<<<<<<<<< * * def __dealloc__(self): */ __pyx_t_4 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_reset); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 106, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = NULL; __pyx_t_1 = 0; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_4))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_4); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_4); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_4, function); __pyx_t_1 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_4)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_v_key, __pyx_v_iv}; __pyx_t_3 = __Pyx_PyFunction_FastCall(__pyx_t_4, __pyx_temp+1-__pyx_t_1, 2+__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 106, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_3); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_4)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_v_key, __pyx_v_iv}; __pyx_t_3 = __Pyx_PyCFunction_FastCall(__pyx_t_4, __pyx_temp+1-__pyx_t_1, 2+__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 106, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_3); } else #endif { __pyx_t_6 = PyTuple_New(2+__pyx_t_1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 106, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (__pyx_t_5) { __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_5); __pyx_t_5 = NULL; } __Pyx_INCREF(__pyx_v_key); __Pyx_GIVEREF(__pyx_v_key); PyTuple_SET_ITEM(__pyx_t_6, 0+__pyx_t_1, __pyx_v_key); __Pyx_INCREF(__pyx_v_iv); __Pyx_GIVEREF(__pyx_v_iv); PyTuple_SET_ITEM(__pyx_t_6, 1+__pyx_t_1, __pyx_v_iv); __pyx_t_3 = __Pyx_PyObject_Call(__pyx_t_4, __pyx_t_6, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 106, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/crypto.pyx":95 * cdef long long blocks * * def __cinit__(self, is_encrypt, key, iv=None): # <<<<<<<<<<<<<< * self.ctx = EVP_CIPHER_CTX_new() * self.is_encrypt = is_encrypt */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.crypto.AES.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":108 * self.reset(key, iv) * * def __dealloc__(self): # <<<<<<<<<<<<<< * EVP_CIPHER_CTX_free(self.ctx) * */ /* Python wrapper */ static void __pyx_pw_4borg_6crypto_3AES_3__dealloc__(PyObject *__pyx_v_self); /*proto*/ static void __pyx_pw_4borg_6crypto_3AES_3__dealloc__(PyObject *__pyx_v_self) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0); __pyx_pf_4borg_6crypto_3AES_2__dealloc__(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); } static void __pyx_pf_4borg_6crypto_3AES_2__dealloc__(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__dealloc__", 0); /* "borg/crypto.pyx":109 * * def __dealloc__(self): * EVP_CIPHER_CTX_free(self.ctx) # <<<<<<<<<<<<<< * * def reset(self, key=None, iv=None): */ EVP_CIPHER_CTX_free(__pyx_v_self->ctx); /* "borg/crypto.pyx":108 * self.reset(key, iv) * * def __dealloc__(self): # <<<<<<<<<<<<<< * EVP_CIPHER_CTX_free(self.ctx) * */ /* function exit code */ __Pyx_RefNannyFinishContext(); } /* "borg/crypto.pyx":111 * EVP_CIPHER_CTX_free(self.ctx) * * def reset(self, key=None, iv=None): # <<<<<<<<<<<<<< * cdef const unsigned char *key2 = NULL * cdef const unsigned char *iv2 = NULL */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3AES_5reset(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_6crypto_3AES_5reset(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key = 0; PyObject *__pyx_v_iv = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("reset (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key,&__pyx_n_s_iv,0}; PyObject* values[2] = {0,0}; values[0] = ((PyObject *)Py_None); values[1] = ((PyObject *)Py_None); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key); if (value) { values[0] = value; kw_args--; } } CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_iv); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "reset") < 0)) __PYX_ERR(0, 111, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_key = values[0]; __pyx_v_iv = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("reset", 0, 0, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 111, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.AES.reset", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_6crypto_3AES_4reset(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self), __pyx_v_key, __pyx_v_iv); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_3AES_4reset(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_iv) { unsigned char const *__pyx_v_key2; unsigned char const *__pyx_v_iv2; long __pyx_v_i; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; unsigned char const *__pyx_t_2; unsigned char const *__pyx_t_3; int __pyx_t_4; int __pyx_t_5; Py_ssize_t __pyx_t_6; long __pyx_t_7; PyObject *__pyx_t_8 = NULL; unsigned char __pyx_t_9; __Pyx_RefNannySetupContext("reset", 0); /* "borg/crypto.pyx":112 * * def reset(self, key=None, iv=None): * cdef const unsigned char *key2 = NULL # <<<<<<<<<<<<<< * cdef const unsigned char *iv2 = NULL * if key: */ __pyx_v_key2 = NULL; /* "borg/crypto.pyx":113 * def reset(self, key=None, iv=None): * cdef const unsigned char *key2 = NULL * cdef const unsigned char *iv2 = NULL # <<<<<<<<<<<<<< * if key: * key2 = key */ __pyx_v_iv2 = NULL; /* "borg/crypto.pyx":114 * cdef const unsigned char *key2 = NULL * cdef const unsigned char *iv2 = NULL * if key: # <<<<<<<<<<<<<< * key2 = key * if iv: */ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_v_key); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(0, 114, __pyx_L1_error) if (__pyx_t_1) { /* "borg/crypto.pyx":115 * cdef const unsigned char *iv2 = NULL * if key: * key2 = key # <<<<<<<<<<<<<< * if iv: * iv2 = iv */ __pyx_t_2 = __Pyx_PyObject_AsUString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 115, __pyx_L1_error) __pyx_v_key2 = __pyx_t_2; /* "borg/crypto.pyx":114 * cdef const unsigned char *key2 = NULL * cdef const unsigned char *iv2 = NULL * if key: # <<<<<<<<<<<<<< * key2 = key * if iv: */ } /* "borg/crypto.pyx":116 * if key: * key2 = key * if iv: # <<<<<<<<<<<<<< * iv2 = iv * assert isinstance(iv, bytes) and len(iv) == 16 */ __pyx_t_1 = __Pyx_PyObject_IsTrue(__pyx_v_iv); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(0, 116, __pyx_L1_error) if (__pyx_t_1) { /* "borg/crypto.pyx":117 * key2 = key * if iv: * iv2 = iv # <<<<<<<<<<<<<< * assert isinstance(iv, bytes) and len(iv) == 16 * for i in range(16): */ __pyx_t_3 = __Pyx_PyObject_AsUString(__pyx_v_iv); if (unlikely((!__pyx_t_3) && PyErr_Occurred())) __PYX_ERR(0, 117, __pyx_L1_error) __pyx_v_iv2 = __pyx_t_3; /* "borg/crypto.pyx":118 * if iv: * iv2 = iv * assert isinstance(iv, bytes) and len(iv) == 16 # <<<<<<<<<<<<<< * for i in range(16): * self.iv_orig[i] = iv[i] */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_4 = PyBytes_Check(__pyx_v_iv); __pyx_t_5 = (__pyx_t_4 != 0); if (__pyx_t_5) { } else { __pyx_t_1 = __pyx_t_5; goto __pyx_L5_bool_binop_done; } __pyx_t_6 = PyObject_Length(__pyx_v_iv); if (unlikely(__pyx_t_6 == ((Py_ssize_t)-1))) __PYX_ERR(0, 118, __pyx_L1_error) __pyx_t_5 = ((__pyx_t_6 == 16) != 0); __pyx_t_1 = __pyx_t_5; __pyx_L5_bool_binop_done:; if (unlikely(!__pyx_t_1)) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 118, __pyx_L1_error) } } #endif /* "borg/crypto.pyx":119 * iv2 = iv * assert isinstance(iv, bytes) and len(iv) == 16 * for i in range(16): # <<<<<<<<<<<<<< * self.iv_orig[i] = iv[i] * self.blocks = 0 # number of AES blocks encrypted starting with iv_orig */ for (__pyx_t_7 = 0; __pyx_t_7 < 16; __pyx_t_7+=1) { __pyx_v_i = __pyx_t_7; /* "borg/crypto.pyx":120 * assert isinstance(iv, bytes) and len(iv) == 16 * for i in range(16): * self.iv_orig[i] = iv[i] # <<<<<<<<<<<<<< * self.blocks = 0 # number of AES blocks encrypted starting with iv_orig * # Initialise key and IV */ __pyx_t_8 = __Pyx_GetItemInt(__pyx_v_iv, __pyx_v_i, long, 1, __Pyx_PyInt_From_long, 0, 1, 1); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 120, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_9 = __Pyx_PyInt_As_unsigned_char(__pyx_t_8); if (unlikely((__pyx_t_9 == (unsigned char)-1) && PyErr_Occurred())) __PYX_ERR(0, 120, __pyx_L1_error) __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; (__pyx_v_self->iv_orig[__pyx_v_i]) = __pyx_t_9; } /* "borg/crypto.pyx":121 * for i in range(16): * self.iv_orig[i] = iv[i] * self.blocks = 0 # number of AES blocks encrypted starting with iv_orig # <<<<<<<<<<<<<< * # Initialise key and IV * if self.is_encrypt: */ __pyx_v_self->blocks = 0; /* "borg/crypto.pyx":116 * if key: * key2 = key * if iv: # <<<<<<<<<<<<<< * iv2 = iv * assert isinstance(iv, bytes) and len(iv) == 16 */ } /* "borg/crypto.pyx":123 * self.blocks = 0 # number of AES blocks encrypted starting with iv_orig * # Initialise key and IV * if self.is_encrypt: # <<<<<<<<<<<<<< * if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): * raise Exception('EVP_EncryptInit_ex failed') */ __pyx_t_1 = (__pyx_v_self->is_encrypt != 0); if (__pyx_t_1) { /* "borg/crypto.pyx":124 * # Initialise key and IV * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt */ __pyx_t_1 = ((!(EVP_EncryptInit_ex(__pyx_v_self->ctx, NULL, NULL, __pyx_v_key2, __pyx_v_iv2) != 0)) != 0); if (unlikely(__pyx_t_1)) { /* "borg/crypto.pyx":125 * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): * raise Exception('EVP_EncryptInit_ex failed') # <<<<<<<<<<<<<< * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): */ __pyx_t_8 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple_, NULL); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 125, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_Raise(__pyx_t_8, 0, 0, 0); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __PYX_ERR(0, 125, __pyx_L1_error) /* "borg/crypto.pyx":124 * # Initialise key and IV * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt */ } /* "borg/crypto.pyx":123 * self.blocks = 0 # number of AES blocks encrypted starting with iv_orig * # Initialise key and IV * if self.is_encrypt: # <<<<<<<<<<<<<< * if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): * raise Exception('EVP_EncryptInit_ex failed') */ goto __pyx_L9; } /* "borg/crypto.pyx":127 * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): # <<<<<<<<<<<<<< * raise Exception('EVP_DecryptInit_ex failed') * */ /*else*/ { __pyx_t_1 = ((!(EVP_DecryptInit_ex(__pyx_v_self->ctx, NULL, NULL, __pyx_v_key2, __pyx_v_iv2) != 0)) != 0); if (unlikely(__pyx_t_1)) { /* "borg/crypto.pyx":128 * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): * raise Exception('EVP_DecryptInit_ex failed') # <<<<<<<<<<<<<< * * @property */ __pyx_t_8 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_Raise(__pyx_t_8, 0, 0, 0); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; __PYX_ERR(0, 128, __pyx_L1_error) /* "borg/crypto.pyx":127 * raise Exception('EVP_EncryptInit_ex failed') * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): # <<<<<<<<<<<<<< * raise Exception('EVP_DecryptInit_ex failed') * */ } } __pyx_L9:; /* "borg/crypto.pyx":111 * EVP_CIPHER_CTX_free(self.ctx) * * def reset(self, key=None, iv=None): # <<<<<<<<<<<<<< * cdef const unsigned char *key2 = NULL * cdef const unsigned char *iv2 = NULL */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.crypto.AES.reset", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":131 * * @property * def iv(self): # <<<<<<<<<<<<<< * return increment_iv(self.iv_orig[:16], self.blocks) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3AES_2iv_1__get__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_6crypto_3AES_2iv_1__get__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__get__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_6crypto_3AES_2iv___get__(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_3AES_2iv___get__(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; int __pyx_t_6; PyObject *__pyx_t_7 = NULL; __Pyx_RefNannySetupContext("__get__", 0); /* "borg/crypto.pyx":132 * @property * def iv(self): * return increment_iv(self.iv_orig[:16], self.blocks) # <<<<<<<<<<<<<< * * def encrypt(self, data): */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_increment_iv); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyBytes_FromStringAndSize(((const char*)__pyx_v_self->iv_orig) + 0, 16 - 0); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyInt_From_PY_LONG_LONG(__pyx_v_self->blocks); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = NULL; __pyx_t_6 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); __pyx_t_6 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_2)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_t_3, __pyx_t_4}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_2, __pyx_temp+1-__pyx_t_6, 2+__pyx_t_6); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_2)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_t_3, __pyx_t_4}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_2, __pyx_temp+1-__pyx_t_6, 2+__pyx_t_6); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; } else #endif { __pyx_t_7 = PyTuple_New(2+__pyx_t_6); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); if (__pyx_t_5) { __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_7, 0, __pyx_t_5); __pyx_t_5 = NULL; } __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_7, 0+__pyx_t_6, __pyx_t_3); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_7, 1+__pyx_t_6, __pyx_t_4); __pyx_t_3 = 0; __pyx_t_4 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_t_7, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; } __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_r = __pyx_t_1; __pyx_t_1 = 0; goto __pyx_L0; /* "borg/crypto.pyx":131 * * @property * def iv(self): # <<<<<<<<<<<<<< * return increment_iv(self.iv_orig[:16], self.blocks) * */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_7); __Pyx_AddTraceback("borg.crypto.AES.iv.__get__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":134 * return increment_iv(self.iv_orig[:16], self.blocks) * * def encrypt(self, data): # <<<<<<<<<<<<<< * cdef int inl = len(data) * cdef int ctl = 0 */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3AES_7encrypt(PyObject *__pyx_v_self, PyObject *__pyx_v_data); /*proto*/ static PyObject *__pyx_pw_4borg_6crypto_3AES_7encrypt(PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("encrypt (wrapper)", 0); __pyx_r = __pyx_pf_4borg_6crypto_3AES_6encrypt(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self), ((PyObject *)__pyx_v_data)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_3AES_6encrypt(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_data) { int __pyx_v_inl; int __pyx_v_ctl; int __pyx_v_outl; unsigned char *__pyx_v_out; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; int __pyx_t_2; unsigned char const *__pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; PY_LONG_LONG __pyx_t_9; int __pyx_t_10; int __pyx_t_11; char const *__pyx_t_12; PyObject *__pyx_t_13 = NULL; PyObject *__pyx_t_14 = NULL; PyObject *__pyx_t_15 = NULL; PyObject *__pyx_t_16 = NULL; PyObject *__pyx_t_17 = NULL; PyObject *__pyx_t_18 = NULL; __Pyx_RefNannySetupContext("encrypt", 0); /* "borg/crypto.pyx":135 * * def encrypt(self, data): * cdef int inl = len(data) # <<<<<<<<<<<<<< * cdef int ctl = 0 * cdef int outl = 0 */ __pyx_t_1 = PyObject_Length(__pyx_v_data); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 135, __pyx_L1_error) __pyx_v_inl = __pyx_t_1; /* "borg/crypto.pyx":136 * def encrypt(self, data): * cdef int inl = len(data) * cdef int ctl = 0 # <<<<<<<<<<<<<< * cdef int outl = 0 * # note: modes that use padding, need up to one extra AES block (16b) */ __pyx_v_ctl = 0; /* "borg/crypto.pyx":137 * cdef int inl = len(data) * cdef int ctl = 0 * cdef int outl = 0 # <<<<<<<<<<<<<< * # note: modes that use padding, need up to one extra AES block (16b) * cdef unsigned char *out = malloc(inl+16) */ __pyx_v_outl = 0; /* "borg/crypto.pyx":139 * cdef int outl = 0 * # note: modes that use padding, need up to one extra AES block (16b) * cdef unsigned char *out = malloc(inl+16) # <<<<<<<<<<<<<< * if not out: * raise MemoryError */ __pyx_v_out = ((unsigned char *)malloc((__pyx_v_inl + 16))); /* "borg/crypto.pyx":140 * # note: modes that use padding, need up to one extra AES block (16b) * cdef unsigned char *out = malloc(inl+16) * if not out: # <<<<<<<<<<<<<< * raise MemoryError * try: */ __pyx_t_2 = ((!(__pyx_v_out != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":141 * cdef unsigned char *out = malloc(inl+16) * if not out: * raise MemoryError # <<<<<<<<<<<<<< * try: * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): */ PyErr_NoMemory(); __PYX_ERR(0, 141, __pyx_L1_error) /* "borg/crypto.pyx":140 * # note: modes that use padding, need up to one extra AES block (16b) * cdef unsigned char *out = malloc(inl+16) * if not out: # <<<<<<<<<<<<<< * raise MemoryError * try: */ } /* "borg/crypto.pyx":142 * if not out: * raise MemoryError * try: # <<<<<<<<<<<<<< * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_EncryptUpdate failed') */ /*try:*/ { /* "borg/crypto.pyx":143 * raise MemoryError * try: * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptUpdate failed') * ctl = outl */ __pyx_t_3 = __Pyx_PyObject_AsUString(__pyx_v_data); if (unlikely((!__pyx_t_3) && PyErr_Occurred())) __PYX_ERR(0, 143, __pyx_L5_error) __pyx_t_2 = ((!(EVP_EncryptUpdate(__pyx_v_self->ctx, __pyx_v_out, (&__pyx_v_outl), __pyx_t_3, __pyx_v_inl) != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":144 * try: * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_EncryptUpdate failed') # <<<<<<<<<<<<<< * ctl = outl * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): */ __pyx_t_4 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__3, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 144, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 144, __pyx_L5_error) /* "borg/crypto.pyx":143 * raise MemoryError * try: * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptUpdate failed') * ctl = outl */ } /* "borg/crypto.pyx":145 * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_EncryptUpdate failed') * ctl = outl # <<<<<<<<<<<<<< * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): * raise Exception('EVP_EncryptFinal failed') */ __pyx_v_ctl = __pyx_v_outl; /* "borg/crypto.pyx":146 * raise Exception('EVP_EncryptUpdate failed') * ctl = outl * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptFinal failed') * ctl += outl */ __pyx_t_2 = ((!(EVP_EncryptFinal_ex(__pyx_v_self->ctx, (__pyx_v_out + __pyx_v_ctl), (&__pyx_v_outl)) != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":147 * ctl = outl * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): * raise Exception('EVP_EncryptFinal failed') # <<<<<<<<<<<<<< * ctl += outl * self.blocks += num_aes_blocks(ctl) */ __pyx_t_4 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__4, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 147, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 147, __pyx_L5_error) /* "borg/crypto.pyx":146 * raise Exception('EVP_EncryptUpdate failed') * ctl = outl * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): # <<<<<<<<<<<<<< * raise Exception('EVP_EncryptFinal failed') * ctl += outl */ } /* "borg/crypto.pyx":148 * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): * raise Exception('EVP_EncryptFinal failed') * ctl += outl # <<<<<<<<<<<<<< * self.blocks += num_aes_blocks(ctl) * return out[:ctl] */ __pyx_v_ctl = (__pyx_v_ctl + __pyx_v_outl); /* "borg/crypto.pyx":149 * raise Exception('EVP_EncryptFinal failed') * ctl += outl * self.blocks += num_aes_blocks(ctl) # <<<<<<<<<<<<<< * return out[:ctl] * finally: */ __pyx_t_4 = __Pyx_PyInt_From_PY_LONG_LONG(__pyx_v_self->blocks); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 149, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_GetModuleGlobalName(__pyx_t_6, __pyx_n_s_num_aes_blocks); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 149, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = __Pyx_PyInt_From_int(__pyx_v_ctl); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 149, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_6))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); } } __pyx_t_5 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_6, __pyx_t_8, __pyx_t_7) : __Pyx_PyObject_CallOneArg(__pyx_t_6, __pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 149, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_t_6 = PyNumber_InPlaceAdd(__pyx_t_4, __pyx_t_5); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 149, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_9 = __Pyx_PyInt_As_PY_LONG_LONG(__pyx_t_6); if (unlikely((__pyx_t_9 == (PY_LONG_LONG)-1) && PyErr_Occurred())) __PYX_ERR(0, 149, __pyx_L5_error) __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_v_self->blocks = __pyx_t_9; /* "borg/crypto.pyx":150 * ctl += outl * self.blocks += num_aes_blocks(ctl) * return out[:ctl] # <<<<<<<<<<<<<< * finally: * free(out) */ __Pyx_XDECREF(__pyx_r); __pyx_t_6 = __Pyx_PyBytes_FromStringAndSize(((const char*)__pyx_v_out) + 0, __pyx_v_ctl - 0); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 150, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_6); __pyx_r = __pyx_t_6; __pyx_t_6 = 0; goto __pyx_L4_return; } /* "borg/crypto.pyx":152 * return out[:ctl] * finally: * free(out) # <<<<<<<<<<<<<< * * def decrypt(self, data): */ /*finally:*/ { __pyx_L5_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_t_17 = 0; __pyx_t_18 = 0; __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_16, &__pyx_t_17, &__pyx_t_18); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_13, &__pyx_t_14, &__pyx_t_15) < 0)) __Pyx_ErrFetch(&__pyx_t_13, &__pyx_t_14, &__pyx_t_15); __Pyx_XGOTREF(__pyx_t_13); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_15); __Pyx_XGOTREF(__pyx_t_16); __Pyx_XGOTREF(__pyx_t_17); __Pyx_XGOTREF(__pyx_t_18); __pyx_t_10 = __pyx_lineno; __pyx_t_11 = __pyx_clineno; __pyx_t_12 = __pyx_filename; { free(__pyx_v_out); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_16); __Pyx_XGIVEREF(__pyx_t_17); __Pyx_XGIVEREF(__pyx_t_18); __Pyx_ExceptionReset(__pyx_t_16, __pyx_t_17, __pyx_t_18); } __Pyx_XGIVEREF(__pyx_t_13); __Pyx_XGIVEREF(__pyx_t_14); __Pyx_XGIVEREF(__pyx_t_15); __Pyx_ErrRestore(__pyx_t_13, __pyx_t_14, __pyx_t_15); __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_t_17 = 0; __pyx_t_18 = 0; __pyx_lineno = __pyx_t_10; __pyx_clineno = __pyx_t_11; __pyx_filename = __pyx_t_12; goto __pyx_L1_error; } __pyx_L4_return: { __pyx_t_18 = __pyx_r; __pyx_r = 0; free(__pyx_v_out); __pyx_r = __pyx_t_18; __pyx_t_18 = 0; goto __pyx_L0; } } /* "borg/crypto.pyx":134 * return increment_iv(self.iv_orig[:16], self.blocks) * * def encrypt(self, data): # <<<<<<<<<<<<<< * cdef int inl = len(data) * cdef int ctl = 0 */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.crypto.AES.encrypt", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":154 * free(out) * * def decrypt(self, data): # <<<<<<<<<<<<<< * cdef int inl = len(data) * cdef int ptl = 0 */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3AES_9decrypt(PyObject *__pyx_v_self, PyObject *__pyx_v_data); /*proto*/ static PyObject *__pyx_pw_4borg_6crypto_3AES_9decrypt(PyObject *__pyx_v_self, PyObject *__pyx_v_data) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decrypt (wrapper)", 0); __pyx_r = __pyx_pf_4borg_6crypto_3AES_8decrypt(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self), ((PyObject *)__pyx_v_data)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_3AES_8decrypt(struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, PyObject *__pyx_v_data) { int __pyx_v_inl; int __pyx_v_ptl; int __pyx_v_outl; unsigned char *__pyx_v_out; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; int __pyx_t_2; unsigned char const *__pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; PY_LONG_LONG __pyx_t_9; int __pyx_t_10; int __pyx_t_11; char const *__pyx_t_12; PyObject *__pyx_t_13 = NULL; PyObject *__pyx_t_14 = NULL; PyObject *__pyx_t_15 = NULL; PyObject *__pyx_t_16 = NULL; PyObject *__pyx_t_17 = NULL; PyObject *__pyx_t_18 = NULL; __Pyx_RefNannySetupContext("decrypt", 0); /* "borg/crypto.pyx":155 * * def decrypt(self, data): * cdef int inl = len(data) # <<<<<<<<<<<<<< * cdef int ptl = 0 * cdef int outl = 0 */ __pyx_t_1 = PyObject_Length(__pyx_v_data); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 155, __pyx_L1_error) __pyx_v_inl = __pyx_t_1; /* "borg/crypto.pyx":156 * def decrypt(self, data): * cdef int inl = len(data) * cdef int ptl = 0 # <<<<<<<<<<<<<< * cdef int outl = 0 * # note: modes that use padding, need up to one extra AES block (16b). */ __pyx_v_ptl = 0; /* "borg/crypto.pyx":157 * cdef int inl = len(data) * cdef int ptl = 0 * cdef int outl = 0 # <<<<<<<<<<<<<< * # note: modes that use padding, need up to one extra AES block (16b). * # This is what the openssl docs say. I am not sure this is correct, */ __pyx_v_outl = 0; /* "borg/crypto.pyx":161 * # This is what the openssl docs say. I am not sure this is correct, * # but OTOH it will not cause any harm if our buffer is a little bigger. * cdef unsigned char *out = malloc(inl+16) # <<<<<<<<<<<<<< * if not out: * raise MemoryError */ __pyx_v_out = ((unsigned char *)malloc((__pyx_v_inl + 16))); /* "borg/crypto.pyx":162 * # but OTOH it will not cause any harm if our buffer is a little bigger. * cdef unsigned char *out = malloc(inl+16) * if not out: # <<<<<<<<<<<<<< * raise MemoryError * try: */ __pyx_t_2 = ((!(__pyx_v_out != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":163 * cdef unsigned char *out = malloc(inl+16) * if not out: * raise MemoryError # <<<<<<<<<<<<<< * try: * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): */ PyErr_NoMemory(); __PYX_ERR(0, 163, __pyx_L1_error) /* "borg/crypto.pyx":162 * # but OTOH it will not cause any harm if our buffer is a little bigger. * cdef unsigned char *out = malloc(inl+16) * if not out: # <<<<<<<<<<<<<< * raise MemoryError * try: */ } /* "borg/crypto.pyx":164 * if not out: * raise MemoryError * try: # <<<<<<<<<<<<<< * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_DecryptUpdate failed') */ /*try:*/ { /* "borg/crypto.pyx":165 * raise MemoryError * try: * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): # <<<<<<<<<<<<<< * raise Exception('EVP_DecryptUpdate failed') * ptl = outl */ __pyx_t_3 = __Pyx_PyObject_AsUString(__pyx_v_data); if (unlikely((!__pyx_t_3) && PyErr_Occurred())) __PYX_ERR(0, 165, __pyx_L5_error) __pyx_t_2 = ((!(EVP_DecryptUpdate(__pyx_v_self->ctx, __pyx_v_out, (&__pyx_v_outl), __pyx_t_3, __pyx_v_inl) != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":166 * try: * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_DecryptUpdate failed') # <<<<<<<<<<<<<< * ptl = outl * if EVP_DecryptFinal_ex(self.ctx, out+ptl, &outl) <= 0: */ __pyx_t_4 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__5, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 166, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 166, __pyx_L5_error) /* "borg/crypto.pyx":165 * raise MemoryError * try: * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): # <<<<<<<<<<<<<< * raise Exception('EVP_DecryptUpdate failed') * ptl = outl */ } /* "borg/crypto.pyx":167 * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_DecryptUpdate failed') * ptl = outl # <<<<<<<<<<<<<< * if EVP_DecryptFinal_ex(self.ctx, out+ptl, &outl) <= 0: * # this error check is very important for modes with padding or */ __pyx_v_ptl = __pyx_v_outl; /* "borg/crypto.pyx":168 * raise Exception('EVP_DecryptUpdate failed') * ptl = outl * if EVP_DecryptFinal_ex(self.ctx, out+ptl, &outl) <= 0: # <<<<<<<<<<<<<< * # this error check is very important for modes with padding or * # authentication. for them, a failure here means corrupted data. */ __pyx_t_2 = ((EVP_DecryptFinal_ex(__pyx_v_self->ctx, (__pyx_v_out + __pyx_v_ptl), (&__pyx_v_outl)) <= 0) != 0); if (unlikely(__pyx_t_2)) { /* "borg/crypto.pyx":172 * # authentication. for them, a failure here means corrupted data. * # CTR mode does not use padding nor authentication. * raise Exception('EVP_DecryptFinal failed') # <<<<<<<<<<<<<< * ptl += outl * self.blocks += num_aes_blocks(inl) */ __pyx_t_4 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__6, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 172, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 172, __pyx_L5_error) /* "borg/crypto.pyx":168 * raise Exception('EVP_DecryptUpdate failed') * ptl = outl * if EVP_DecryptFinal_ex(self.ctx, out+ptl, &outl) <= 0: # <<<<<<<<<<<<<< * # this error check is very important for modes with padding or * # authentication. for them, a failure here means corrupted data. */ } /* "borg/crypto.pyx":173 * # CTR mode does not use padding nor authentication. * raise Exception('EVP_DecryptFinal failed') * ptl += outl # <<<<<<<<<<<<<< * self.blocks += num_aes_blocks(inl) * return out[:ptl] */ __pyx_v_ptl = (__pyx_v_ptl + __pyx_v_outl); /* "borg/crypto.pyx":174 * raise Exception('EVP_DecryptFinal failed') * ptl += outl * self.blocks += num_aes_blocks(inl) # <<<<<<<<<<<<<< * return out[:ptl] * finally: */ __pyx_t_4 = __Pyx_PyInt_From_PY_LONG_LONG(__pyx_v_self->blocks); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 174, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_GetModuleGlobalName(__pyx_t_6, __pyx_n_s_num_aes_blocks); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 174, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = __Pyx_PyInt_From_int(__pyx_v_inl); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 174, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_6))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); } } __pyx_t_5 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_6, __pyx_t_8, __pyx_t_7) : __Pyx_PyObject_CallOneArg(__pyx_t_6, __pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 174, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_t_6 = PyNumber_InPlaceAdd(__pyx_t_4, __pyx_t_5); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 174, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_9 = __Pyx_PyInt_As_PY_LONG_LONG(__pyx_t_6); if (unlikely((__pyx_t_9 == (PY_LONG_LONG)-1) && PyErr_Occurred())) __PYX_ERR(0, 174, __pyx_L5_error) __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_v_self->blocks = __pyx_t_9; /* "borg/crypto.pyx":175 * ptl += outl * self.blocks += num_aes_blocks(inl) * return out[:ptl] # <<<<<<<<<<<<<< * finally: * free(out) */ __Pyx_XDECREF(__pyx_r); __pyx_t_6 = __Pyx_PyBytes_FromStringAndSize(((const char*)__pyx_v_out) + 0, __pyx_v_ptl - 0); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 175, __pyx_L5_error) __Pyx_GOTREF(__pyx_t_6); __pyx_r = __pyx_t_6; __pyx_t_6 = 0; goto __pyx_L4_return; } /* "borg/crypto.pyx":177 * return out[:ptl] * finally: * free(out) # <<<<<<<<<<<<<< * * */ /*finally:*/ { __pyx_L5_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_t_17 = 0; __pyx_t_18 = 0; __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_16, &__pyx_t_17, &__pyx_t_18); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_13, &__pyx_t_14, &__pyx_t_15) < 0)) __Pyx_ErrFetch(&__pyx_t_13, &__pyx_t_14, &__pyx_t_15); __Pyx_XGOTREF(__pyx_t_13); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_15); __Pyx_XGOTREF(__pyx_t_16); __Pyx_XGOTREF(__pyx_t_17); __Pyx_XGOTREF(__pyx_t_18); __pyx_t_10 = __pyx_lineno; __pyx_t_11 = __pyx_clineno; __pyx_t_12 = __pyx_filename; { free(__pyx_v_out); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_16); __Pyx_XGIVEREF(__pyx_t_17); __Pyx_XGIVEREF(__pyx_t_18); __Pyx_ExceptionReset(__pyx_t_16, __pyx_t_17, __pyx_t_18); } __Pyx_XGIVEREF(__pyx_t_13); __Pyx_XGIVEREF(__pyx_t_14); __Pyx_XGIVEREF(__pyx_t_15); __Pyx_ErrRestore(__pyx_t_13, __pyx_t_14, __pyx_t_15); __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_t_17 = 0; __pyx_t_18 = 0; __pyx_lineno = __pyx_t_10; __pyx_clineno = __pyx_t_11; __pyx_filename = __pyx_t_12; goto __pyx_L1_error; } __pyx_L4_return: { __pyx_t_18 = __pyx_r; __pyx_r = 0; free(__pyx_v_out); __pyx_r = __pyx_t_18; __pyx_t_18 = 0; goto __pyx_L0; } } /* "borg/crypto.pyx":154 * free(out) * * def decrypt(self, data): # <<<<<<<<<<<<<< * cdef int inl = len(data) * cdef int ptl = 0 */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.crypto.AES.decrypt", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3AES_11__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_6crypto_3AES_11__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_6crypto_3AES_10__reduce_cython__(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_3AES_10__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__7, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.crypto.AES.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_3AES_13__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_6crypto_3AES_13__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_6crypto_3AES_12__setstate_cython__(((struct __pyx_obj_4borg_6crypto_AES *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_3AES_12__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_6crypto_AES *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__8, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.crypto.AES.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/crypto.pyx":180 * * * def hkdf_hmac_sha512(ikm, salt, info, output_length): # <<<<<<<<<<<<<< * """ * Compute HKDF-HMAC-SHA512 with input key material *ikm*, *salt* and *info* to produce *output_length* bytes. */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_6crypto_9hkdf_hmac_sha512(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_6crypto_8hkdf_hmac_sha512[] = "\n Compute HKDF-HMAC-SHA512 with input key material *ikm*, *salt* and *info* to produce *output_length* bytes.\n\n This is the \"HMAC-based Extract-and-Expand Key Derivation Function (HKDF)\" (RFC 5869)\n instantiated with HMAC-SHA512.\n\n *output_length* must not be greater than 64 * 255 bytes.\n "; static PyMethodDef __pyx_mdef_4borg_6crypto_9hkdf_hmac_sha512 = {"hkdf_hmac_sha512", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_6crypto_9hkdf_hmac_sha512, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_6crypto_8hkdf_hmac_sha512}; static PyObject *__pyx_pw_4borg_6crypto_9hkdf_hmac_sha512(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_ikm = 0; PyObject *__pyx_v_salt = 0; PyObject *__pyx_v_info = 0; PyObject *__pyx_v_output_length = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("hkdf_hmac_sha512 (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_ikm,&__pyx_n_s_salt,&__pyx_n_s_info,&__pyx_n_s_output_length,0}; PyObject* values[4] = {0,0,0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_ikm)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_salt)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("hkdf_hmac_sha512", 1, 4, 4, 1); __PYX_ERR(0, 180, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_info)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("hkdf_hmac_sha512", 1, 4, 4, 2); __PYX_ERR(0, 180, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (likely((values[3] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_output_length)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("hkdf_hmac_sha512", 1, 4, 4, 3); __PYX_ERR(0, 180, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "hkdf_hmac_sha512") < 0)) __PYX_ERR(0, 180, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 4) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[3] = PyTuple_GET_ITEM(__pyx_args, 3); } __pyx_v_ikm = values[0]; __pyx_v_salt = values[1]; __pyx_v_info = values[2]; __pyx_v_output_length = values[3]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("hkdf_hmac_sha512", 1, 4, 4, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 180, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.crypto.hkdf_hmac_sha512", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_6crypto_8hkdf_hmac_sha512(__pyx_self, __pyx_v_ikm, __pyx_v_salt, __pyx_v_info, __pyx_v_output_length); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_6crypto_8hkdf_hmac_sha512(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_ikm, PyObject *__pyx_v_salt, PyObject *__pyx_v_info, PyObject *__pyx_v_output_length) { PyObject *__pyx_v_digest_length = NULL; PyObject *__pyx_v_prk = NULL; PyObject *__pyx_v_n = NULL; PyObject *__pyx_v_t_n = NULL; PyObject *__pyx_v_output = NULL; PyObject *__pyx_v_i = NULL; PyObject *__pyx_v_msg = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; int __pyx_t_3; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; int __pyx_t_8; PyObject *__pyx_t_9 = NULL; Py_ssize_t __pyx_t_10; PyObject *(*__pyx_t_11)(PyObject *); PyObject *__pyx_t_12 = NULL; __Pyx_RefNannySetupContext("hkdf_hmac_sha512", 0); __Pyx_INCREF(__pyx_v_salt); /* "borg/crypto.pyx":189 * *output_length* must not be greater than 64 * 255 bytes. * """ * digest_length = 64 # <<<<<<<<<<<<<< * assert output_length <= (255 * digest_length), 'output_length must be <= 255 * 64 bytes' * # Step 1. HKDF-Extract (ikm, salt) -> prk */ __Pyx_INCREF(__pyx_int_64); __pyx_v_digest_length = __pyx_int_64; /* "borg/crypto.pyx":190 * """ * digest_length = 64 * assert output_length <= (255 * digest_length), 'output_length must be <= 255 * 64 bytes' # <<<<<<<<<<<<<< * # Step 1. HKDF-Extract (ikm, salt) -> prk * if salt is None: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyNumber_Multiply(__pyx_int_255, __pyx_v_digest_length); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 190, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyObject_RichCompare(__pyx_v_output_length, __pyx_t_1, Py_LE); __Pyx_XGOTREF(__pyx_t_2); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 190, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_3 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_3 < 0)) __PYX_ERR(0, 190, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_output_length_must_be_255_64_byt); __PYX_ERR(0, 190, __pyx_L1_error) } } #endif /* "borg/crypto.pyx":192 * assert output_length <= (255 * digest_length), 'output_length must be <= 255 * 64 bytes' * # Step 1. HKDF-Extract (ikm, salt) -> prk * if salt is None: # <<<<<<<<<<<<<< * salt = bytes(64) * prk = hmac.HMAC(salt, ikm, hashlib.sha512).digest() */ __pyx_t_3 = (__pyx_v_salt == Py_None); __pyx_t_4 = (__pyx_t_3 != 0); if (__pyx_t_4) { /* "borg/crypto.pyx":193 * # Step 1. HKDF-Extract (ikm, salt) -> prk * if salt is None: * salt = bytes(64) # <<<<<<<<<<<<<< * prk = hmac.HMAC(salt, ikm, hashlib.sha512).digest() * */ __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)(&PyBytes_Type)), __pyx_tuple__9, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 193, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF_SET(__pyx_v_salt, __pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":192 * assert output_length <= (255 * digest_length), 'output_length must be <= 255 * 64 bytes' * # Step 1. HKDF-Extract (ikm, salt) -> prk * if salt is None: # <<<<<<<<<<<<<< * salt = bytes(64) * prk = hmac.HMAC(salt, ikm, hashlib.sha512).digest() */ } /* "borg/crypto.pyx":194 * if salt is None: * salt = bytes(64) * prk = hmac.HMAC(salt, ikm, hashlib.sha512).digest() # <<<<<<<<<<<<<< * * # Step 2. HKDF-Expand (prk, info, output_length) -> output key */ __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_hmac); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_t_5, __pyx_n_s_HMAC); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_hashlib); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_5, __pyx_n_s_sha512); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = NULL; __pyx_t_8 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_6))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); __pyx_t_8 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_6)) { PyObject *__pyx_temp[4] = {__pyx_t_5, __pyx_v_salt, __pyx_v_ikm, __pyx_t_7}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_6, __pyx_temp+1-__pyx_t_8, 3+__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_6)) { PyObject *__pyx_temp[4] = {__pyx_t_5, __pyx_v_salt, __pyx_v_ikm, __pyx_t_7}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_6, __pyx_temp+1-__pyx_t_8, 3+__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; } else #endif { __pyx_t_9 = PyTuple_New(3+__pyx_t_8); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); if (__pyx_t_5) { __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_9, 0, __pyx_t_5); __pyx_t_5 = NULL; } __Pyx_INCREF(__pyx_v_salt); __Pyx_GIVEREF(__pyx_v_salt); PyTuple_SET_ITEM(__pyx_t_9, 0+__pyx_t_8, __pyx_v_salt); __Pyx_INCREF(__pyx_v_ikm); __Pyx_GIVEREF(__pyx_v_ikm); PyTuple_SET_ITEM(__pyx_t_9, 1+__pyx_t_8, __pyx_v_ikm); __Pyx_GIVEREF(__pyx_t_7); PyTuple_SET_ITEM(__pyx_t_9, 2+__pyx_t_8, __pyx_t_7); __pyx_t_7 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_6, __pyx_t_9, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; } __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_t_6 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_digest); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_6))) { __pyx_t_1 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_1)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_1); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); } } __pyx_t_2 = (__pyx_t_1) ? __Pyx_PyObject_CallOneArg(__pyx_t_6, __pyx_t_1) : __Pyx_PyObject_CallNoArg(__pyx_t_6); __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 194, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_v_prk = __pyx_t_2; __pyx_t_2 = 0; /* "borg/crypto.pyx":197 * * # Step 2. HKDF-Expand (prk, info, output_length) -> output key * n = ceil(output_length / digest_length) # <<<<<<<<<<<<<< * t_n = b'' * output = b'' */ __Pyx_GetModuleGlobalName(__pyx_t_6, __pyx_n_s_ceil); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_1 = __Pyx_PyNumber_Divide(__pyx_v_output_length, __pyx_v_digest_length); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_9 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_6))) { __pyx_t_9 = PyMethod_GET_SELF(__pyx_t_6); if (likely(__pyx_t_9)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_6); __Pyx_INCREF(__pyx_t_9); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_6, function); } } __pyx_t_2 = (__pyx_t_9) ? __Pyx_PyObject_Call2Args(__pyx_t_6, __pyx_t_9, __pyx_t_1) : __Pyx_PyObject_CallOneArg(__pyx_t_6, __pyx_t_1); __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 197, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __pyx_v_n = __pyx_t_2; __pyx_t_2 = 0; /* "borg/crypto.pyx":198 * # Step 2. HKDF-Expand (prk, info, output_length) -> output key * n = ceil(output_length / digest_length) * t_n = b'' # <<<<<<<<<<<<<< * output = b'' * for i in range(n): */ __Pyx_INCREF(__pyx_kp_b__10); __pyx_v_t_n = __pyx_kp_b__10; /* "borg/crypto.pyx":199 * n = ceil(output_length / digest_length) * t_n = b'' * output = b'' # <<<<<<<<<<<<<< * for i in range(n): * msg = t_n + info + (i + 1).to_bytes(1, 'little') */ __Pyx_INCREF(__pyx_kp_b__10); __pyx_v_output = __pyx_kp_b__10; /* "borg/crypto.pyx":200 * t_n = b'' * output = b'' * for i in range(n): # <<<<<<<<<<<<<< * msg = t_n + info + (i + 1).to_bytes(1, 'little') * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() */ __pyx_t_2 = __Pyx_PyObject_CallOneArg(__pyx_builtin_range, __pyx_v_n); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 200, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (likely(PyList_CheckExact(__pyx_t_2)) || PyTuple_CheckExact(__pyx_t_2)) { __pyx_t_6 = __pyx_t_2; __Pyx_INCREF(__pyx_t_6); __pyx_t_10 = 0; __pyx_t_11 = NULL; } else { __pyx_t_10 = -1; __pyx_t_6 = PyObject_GetIter(__pyx_t_2); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 200, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_11 = Py_TYPE(__pyx_t_6)->tp_iternext; if (unlikely(!__pyx_t_11)) __PYX_ERR(0, 200, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; for (;;) { if (likely(!__pyx_t_11)) { if (likely(PyList_CheckExact(__pyx_t_6))) { if (__pyx_t_10 >= PyList_GET_SIZE(__pyx_t_6)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_2 = PyList_GET_ITEM(__pyx_t_6, __pyx_t_10); __Pyx_INCREF(__pyx_t_2); __pyx_t_10++; if (unlikely(0 < 0)) __PYX_ERR(0, 200, __pyx_L1_error) #else __pyx_t_2 = PySequence_ITEM(__pyx_t_6, __pyx_t_10); __pyx_t_10++; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 200, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); #endif } else { if (__pyx_t_10 >= PyTuple_GET_SIZE(__pyx_t_6)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_2 = PyTuple_GET_ITEM(__pyx_t_6, __pyx_t_10); __Pyx_INCREF(__pyx_t_2); __pyx_t_10++; if (unlikely(0 < 0)) __PYX_ERR(0, 200, __pyx_L1_error) #else __pyx_t_2 = PySequence_ITEM(__pyx_t_6, __pyx_t_10); __pyx_t_10++; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 200, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); #endif } } else { __pyx_t_2 = __pyx_t_11(__pyx_t_6); if (unlikely(!__pyx_t_2)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 200, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_2); } __Pyx_XDECREF_SET(__pyx_v_i, __pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":201 * output = b'' * for i in range(n): * msg = t_n + info + (i + 1).to_bytes(1, 'little') # <<<<<<<<<<<<<< * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() * output += t_n */ __pyx_t_2 = PyNumber_Add(__pyx_v_t_n, __pyx_v_info); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_1 = __Pyx_PyInt_AddObjC(__pyx_v_i, __pyx_int_1, 1, 0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_9 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_to_bytes); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_9, __pyx_tuple__11, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; __pyx_t_9 = PyNumber_Add(__pyx_t_2, __pyx_t_1); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF_SET(__pyx_v_msg, __pyx_t_9); __pyx_t_9 = 0; /* "borg/crypto.pyx":202 * for i in range(n): * msg = t_n + info + (i + 1).to_bytes(1, 'little') * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() # <<<<<<<<<<<<<< * output += t_n * return output[:output_length] */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_hmac); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_HMAC); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_hashlib); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_sha512); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; __pyx_t_8 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_7))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); __pyx_t_8 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_7)) { PyObject *__pyx_temp[4] = {__pyx_t_2, __pyx_v_prk, __pyx_v_msg, __pyx_t_5}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_7, __pyx_temp+1-__pyx_t_8, 3+__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_7)) { PyObject *__pyx_temp[4] = {__pyx_t_2, __pyx_v_prk, __pyx_v_msg, __pyx_t_5}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_7, __pyx_temp+1-__pyx_t_8, 3+__pyx_t_8); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } else #endif { __pyx_t_12 = PyTuple_New(3+__pyx_t_8); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); if (__pyx_t_2) { __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_12, 0, __pyx_t_2); __pyx_t_2 = NULL; } __Pyx_INCREF(__pyx_v_prk); __Pyx_GIVEREF(__pyx_v_prk); PyTuple_SET_ITEM(__pyx_t_12, 0+__pyx_t_8, __pyx_v_prk); __Pyx_INCREF(__pyx_v_msg); __Pyx_GIVEREF(__pyx_v_msg); PyTuple_SET_ITEM(__pyx_t_12, 1+__pyx_t_8, __pyx_v_msg); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_12, 2+__pyx_t_8, __pyx_t_5); __pyx_t_5 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_7, __pyx_t_12, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; } __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_7 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_digest); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_7))) { __pyx_t_1 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_1)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_1); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); } } __pyx_t_9 = (__pyx_t_1) ? __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_1) : __Pyx_PyObject_CallNoArg(__pyx_t_7); __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 202, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF_SET(__pyx_v_t_n, __pyx_t_9); __pyx_t_9 = 0; /* "borg/crypto.pyx":203 * msg = t_n + info + (i + 1).to_bytes(1, 'little') * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() * output += t_n # <<<<<<<<<<<<<< * return output[:output_length] */ __pyx_t_9 = PyNumber_InPlaceAdd(__pyx_v_output, __pyx_v_t_n); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 203, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_DECREF_SET(__pyx_v_output, __pyx_t_9); __pyx_t_9 = 0; /* "borg/crypto.pyx":200 * t_n = b'' * output = b'' * for i in range(n): # <<<<<<<<<<<<<< * msg = t_n + info + (i + 1).to_bytes(1, 'little') * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() */ } __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; /* "borg/crypto.pyx":204 * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() * output += t_n * return output[:output_length] # <<<<<<<<<<<<<< */ __Pyx_XDECREF(__pyx_r); __pyx_t_6 = __Pyx_PyObject_GetSlice(__pyx_v_output, 0, 0, NULL, &__pyx_v_output_length, NULL, 0, 0, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 204, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_r = __pyx_t_6; __pyx_t_6 = 0; goto __pyx_L0; /* "borg/crypto.pyx":180 * * * def hkdf_hmac_sha512(ikm, salt, info, output_length): # <<<<<<<<<<<<<< * """ * Compute HKDF-HMAC-SHA512 with input key material *ikm*, *salt* and *info* to produce *output_length* bytes. */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_9); __Pyx_XDECREF(__pyx_t_12); __Pyx_AddTraceback("borg.crypto.hkdf_hmac_sha512", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_digest_length); __Pyx_XDECREF(__pyx_v_prk); __Pyx_XDECREF(__pyx_v_n); __Pyx_XDECREF(__pyx_v_t_n); __Pyx_XDECREF(__pyx_v_output); __Pyx_XDECREF(__pyx_v_i); __Pyx_XDECREF(__pyx_v_msg); __Pyx_XDECREF(__pyx_v_salt); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_tp_new_4borg_6crypto_AES(PyTypeObject *t, PyObject *a, PyObject *k) { PyObject *o; if (likely((t->tp_flags & Py_TPFLAGS_IS_ABSTRACT) == 0)) { o = (*t->tp_alloc)(t, 0); } else { o = (PyObject *) PyBaseObject_Type.tp_new(t, __pyx_empty_tuple, 0); } if (unlikely(!o)) return 0; if (unlikely(__pyx_pw_4borg_6crypto_3AES_1__cinit__(o, a, k) < 0)) goto bad; return o; bad: Py_DECREF(o); o = 0; return NULL; } static void __pyx_tp_dealloc_4borg_6crypto_AES(PyObject *o) { #if CYTHON_USE_TP_FINALIZE if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && (!PyType_IS_GC(Py_TYPE(o)) || !_PyGC_FINALIZED(o))) { if (PyObject_CallFinalizerFromDealloc(o)) return; } #endif { PyObject *etype, *eval, *etb; PyErr_Fetch(&etype, &eval, &etb); ++Py_REFCNT(o); __pyx_pw_4borg_6crypto_3AES_3__dealloc__(o); --Py_REFCNT(o); PyErr_Restore(etype, eval, etb); } (*Py_TYPE(o)->tp_free)(o); } static PyObject *__pyx_getprop_4borg_6crypto_3AES_iv(PyObject *o, CYTHON_UNUSED void *x) { return __pyx_pw_4borg_6crypto_3AES_2iv_1__get__(o); } static PyMethodDef __pyx_methods_4borg_6crypto_AES[] = { {"reset", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_6crypto_3AES_5reset, METH_VARARGS|METH_KEYWORDS, 0}, {"encrypt", (PyCFunction)__pyx_pw_4borg_6crypto_3AES_7encrypt, METH_O, 0}, {"decrypt", (PyCFunction)__pyx_pw_4borg_6crypto_3AES_9decrypt, METH_O, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_6crypto_3AES_11__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_6crypto_3AES_13__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static struct PyGetSetDef __pyx_getsets_4borg_6crypto_AES[] = { {(char *)"iv", __pyx_getprop_4borg_6crypto_3AES_iv, 0, (char *)0, 0}, {0, 0, 0, 0, 0} }; static PyTypeObject __pyx_type_4borg_6crypto_AES = { PyVarObject_HEAD_INIT(0, 0) "borg.crypto.AES", /*tp_name*/ sizeof(struct __pyx_obj_4borg_6crypto_AES), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_6crypto_AES, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE, /*tp_flags*/ "A thin wrapper around the OpenSSL EVP cipher API\n ", /*tp_doc*/ 0, /*tp_traverse*/ 0, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ 0, /*tp_iter*/ 0, /*tp_iternext*/ __pyx_methods_4borg_6crypto_AES, /*tp_methods*/ 0, /*tp_members*/ __pyx_getsets_4borg_6crypto_AES, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_6crypto_AES, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_crypto(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_crypto}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "crypto", __pyx_k_A_thin_OpenSSL_wrapper_This_coul, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_2long, __pyx_k_2long, sizeof(__pyx_k_2long), 0, 0, 1, 1}, {&__pyx_n_s_AES, __pyx_k_AES, sizeof(__pyx_k_AES), 0, 0, 1, 1}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_kp_s_EVP_DecryptFinal_failed, __pyx_k_EVP_DecryptFinal_failed, sizeof(__pyx_k_EVP_DecryptFinal_failed), 0, 0, 1, 0}, {&__pyx_kp_s_EVP_DecryptInit_ex_failed, __pyx_k_EVP_DecryptInit_ex_failed, sizeof(__pyx_k_EVP_DecryptInit_ex_failed), 0, 0, 1, 0}, {&__pyx_kp_s_EVP_DecryptUpdate_failed, __pyx_k_EVP_DecryptUpdate_failed, sizeof(__pyx_k_EVP_DecryptUpdate_failed), 0, 0, 1, 0}, {&__pyx_kp_s_EVP_EncryptFinal_failed, __pyx_k_EVP_EncryptFinal_failed, sizeof(__pyx_k_EVP_EncryptFinal_failed), 0, 0, 1, 0}, {&__pyx_kp_s_EVP_EncryptInit_ex_failed, __pyx_k_EVP_EncryptInit_ex_failed, sizeof(__pyx_k_EVP_EncryptInit_ex_failed), 0, 0, 1, 0}, {&__pyx_kp_s_EVP_EncryptUpdate_failed, __pyx_k_EVP_EncryptUpdate_failed, sizeof(__pyx_k_EVP_EncryptUpdate_failed), 0, 0, 1, 0}, {&__pyx_n_s_HMAC, __pyx_k_HMAC, sizeof(__pyx_k_HMAC), 0, 0, 1, 1}, {&__pyx_kp_s_I, __pyx_k_I, sizeof(__pyx_k_I), 0, 0, 1, 0}, {&__pyx_n_s_MemoryError, __pyx_k_MemoryError, sizeof(__pyx_k_MemoryError), 0, 0, 1, 1}, {&__pyx_kp_s_Q, __pyx_k_Q, sizeof(__pyx_k_Q), 0, 0, 1, 0}, {&__pyx_kp_s_QQ, __pyx_k_QQ, sizeof(__pyx_k_QQ), 0, 0, 1, 0}, {&__pyx_n_s_Struct, __pyx_k_Struct, sizeof(__pyx_k_Struct), 0, 0, 1, 1}, {&__pyx_n_s_TypeError, __pyx_k_TypeError, sizeof(__pyx_k_TypeError), 0, 0, 1, 1}, {&__pyx_kp_b__10, __pyx_k__10, sizeof(__pyx_k__10), 0, 0, 0, 0}, {&__pyx_n_s_amount, __pyx_k_amount, sizeof(__pyx_k_amount), 0, 0, 1, 1}, {&__pyx_n_s_b, __pyx_k_b, sizeof(__pyx_k_b), 0, 0, 1, 1}, {&__pyx_n_s_borg_crypto, __pyx_k_borg_crypto, sizeof(__pyx_k_borg_crypto), 0, 0, 1, 1}, {&__pyx_kp_s_borg_crypto_pyx, __pyx_k_borg_crypto_pyx, sizeof(__pyx_k_borg_crypto_pyx), 0, 0, 1, 0}, {&__pyx_n_s_bytes16_to_int, __pyx_k_bytes16_to_int, sizeof(__pyx_k_bytes16_to_int), 0, 0, 1, 1}, {&__pyx_n_s_bytes_to_int, __pyx_k_bytes_to_int, sizeof(__pyx_k_bytes_to_int), 0, 0, 1, 1}, {&__pyx_n_s_bytes_to_long, __pyx_k_bytes_to_long, sizeof(__pyx_k_bytes_to_long), 0, 0, 1, 1}, {&__pyx_n_s_ceil, __pyx_k_ceil, sizeof(__pyx_k_ceil), 0, 0, 1, 1}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_digest, __pyx_k_digest, sizeof(__pyx_k_digest), 0, 0, 1, 1}, {&__pyx_n_s_digest_length, __pyx_k_digest_length, sizeof(__pyx_k_digest_length), 0, 0, 1, 1}, {&__pyx_n_s_getstate, __pyx_k_getstate, sizeof(__pyx_k_getstate), 0, 0, 1, 1}, {&__pyx_n_s_h, __pyx_k_h, sizeof(__pyx_k_h), 0, 0, 1, 1}, {&__pyx_n_s_hashlib, __pyx_k_hashlib, sizeof(__pyx_k_hashlib), 0, 0, 1, 1}, {&__pyx_n_s_hkdf_hmac_sha512, __pyx_k_hkdf_hmac_sha512, sizeof(__pyx_k_hkdf_hmac_sha512), 0, 0, 1, 1}, {&__pyx_n_s_hmac, __pyx_k_hmac, sizeof(__pyx_k_hmac), 0, 0, 1, 1}, {&__pyx_n_s_i, __pyx_k_i, sizeof(__pyx_k_i), 0, 0, 1, 1}, {&__pyx_n_s_ikm, __pyx_k_ikm, sizeof(__pyx_k_ikm), 0, 0, 1, 1}, {&__pyx_n_s_import, __pyx_k_import, sizeof(__pyx_k_import), 0, 0, 1, 1}, {&__pyx_n_s_increment_iv, __pyx_k_increment_iv, sizeof(__pyx_k_increment_iv), 0, 0, 1, 1}, {&__pyx_n_s_info, __pyx_k_info, sizeof(__pyx_k_info), 0, 0, 1, 1}, {&__pyx_n_s_int, __pyx_k_int, sizeof(__pyx_k_int), 0, 0, 1, 1}, {&__pyx_n_s_int_to_bytes16, __pyx_k_int_to_bytes16, sizeof(__pyx_k_int_to_bytes16), 0, 0, 1, 1}, {&__pyx_n_s_is_encrypt, __pyx_k_is_encrypt, sizeof(__pyx_k_is_encrypt), 0, 0, 1, 1}, {&__pyx_n_s_iv, __pyx_k_iv, sizeof(__pyx_k_iv), 0, 0, 1, 1}, {&__pyx_n_s_key, __pyx_k_key, sizeof(__pyx_k_key), 0, 0, 1, 1}, {&__pyx_n_s_l, __pyx_k_l, sizeof(__pyx_k_l), 0, 0, 1, 1}, {&__pyx_n_s_lambda, __pyx_k_lambda, sizeof(__pyx_k_lambda), 0, 0, 1, 1}, {&__pyx_n_s_length, __pyx_k_length, sizeof(__pyx_k_length), 0, 0, 1, 1}, {&__pyx_n_s_little, __pyx_k_little, sizeof(__pyx_k_little), 0, 0, 1, 1}, {&__pyx_n_s_long, __pyx_k_long, sizeof(__pyx_k_long), 0, 0, 1, 1}, {&__pyx_n_s_long_to_bytes, __pyx_k_long_to_bytes, sizeof(__pyx_k_long_to_bytes), 0, 0, 1, 1}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_math, __pyx_k_math, sizeof(__pyx_k_math), 0, 0, 1, 1}, {&__pyx_n_s_max_uint64, __pyx_k_max_uint64, sizeof(__pyx_k_max_uint64), 0, 0, 1, 1}, {&__pyx_n_s_msg, __pyx_k_msg, sizeof(__pyx_k_msg), 0, 0, 1, 1}, {&__pyx_n_s_n, __pyx_k_n, sizeof(__pyx_k_n), 0, 0, 1, 1}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_kp_s_no_default___reduce___due_to_non, __pyx_k_no_default___reduce___due_to_non, sizeof(__pyx_k_no_default___reduce___due_to_non), 0, 0, 1, 0}, {&__pyx_n_s_num_aes_blocks, __pyx_k_num_aes_blocks, sizeof(__pyx_k_num_aes_blocks), 0, 0, 1, 1}, {&__pyx_n_s_offset, __pyx_k_offset, sizeof(__pyx_k_offset), 0, 0, 1, 1}, {&__pyx_n_s_output, __pyx_k_output, sizeof(__pyx_k_output), 0, 0, 1, 1}, {&__pyx_n_s_output_length, __pyx_k_output_length, sizeof(__pyx_k_output_length), 0, 0, 1, 1}, {&__pyx_kp_s_output_length_must_be_255_64_byt, __pyx_k_output_length_must_be_255_64_byt, sizeof(__pyx_k_output_length_must_be_255_64_byt), 0, 0, 1, 0}, {&__pyx_n_s_pack, __pyx_k_pack, sizeof(__pyx_k_pack), 0, 0, 1, 1}, {&__pyx_n_s_prk, __pyx_k_prk, sizeof(__pyx_k_prk), 0, 0, 1, 1}, {&__pyx_n_s_range, __pyx_k_range, sizeof(__pyx_k_range), 0, 0, 1, 1}, {&__pyx_n_s_reduce, __pyx_k_reduce, sizeof(__pyx_k_reduce), 0, 0, 1, 1}, {&__pyx_n_s_reduce_cython, __pyx_k_reduce_cython, sizeof(__pyx_k_reduce_cython), 0, 0, 1, 1}, {&__pyx_n_s_reduce_ex, __pyx_k_reduce_ex, sizeof(__pyx_k_reduce_ex), 0, 0, 1, 1}, {&__pyx_n_s_reset, __pyx_k_reset, sizeof(__pyx_k_reset), 0, 0, 1, 1}, {&__pyx_n_s_salt, __pyx_k_salt, sizeof(__pyx_k_salt), 0, 0, 1, 1}, {&__pyx_n_s_setstate, __pyx_k_setstate, sizeof(__pyx_k_setstate), 0, 0, 1, 1}, {&__pyx_n_s_setstate_cython, __pyx_k_setstate_cython, sizeof(__pyx_k_setstate_cython), 0, 0, 1, 1}, {&__pyx_n_s_sha512, __pyx_k_sha512, sizeof(__pyx_k_sha512), 0, 0, 1, 1}, {&__pyx_n_s_struct, __pyx_k_struct, sizeof(__pyx_k_struct), 0, 0, 1, 1}, {&__pyx_n_s_t_n, __pyx_k_t_n, sizeof(__pyx_k_t_n), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {&__pyx_n_s_to_bytes, __pyx_k_to_bytes, sizeof(__pyx_k_to_bytes), 0, 0, 1, 1}, {&__pyx_n_s_unpack_from, __pyx_k_unpack_from, sizeof(__pyx_k_unpack_from), 0, 0, 1, 1}, {&__pyx_n_s_x, __pyx_k_x, sizeof(__pyx_k_x), 0, 0, 1, 1}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { __pyx_builtin_range = __Pyx_GetBuiltinName(__pyx_n_s_range); if (!__pyx_builtin_range) __PYX_ERR(0, 119, __pyx_L1_error) __pyx_builtin_MemoryError = __Pyx_GetBuiltinName(__pyx_n_s_MemoryError); if (!__pyx_builtin_MemoryError) __PYX_ERR(0, 141, __pyx_L1_error) __pyx_builtin_TypeError = __Pyx_GetBuiltinName(__pyx_n_s_TypeError); if (!__pyx_builtin_TypeError) __PYX_ERR(1, 2, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "borg/crypto.pyx":102 * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_EncryptInit_ex failed') # <<<<<<<<<<<<<< * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): */ __pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_s_EVP_EncryptInit_ex_failed); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 102, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple_); __Pyx_GIVEREF(__pyx_tuple_); /* "borg/crypto.pyx":105 * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): * raise Exception('EVP_DecryptInit_ex failed') # <<<<<<<<<<<<<< * self.reset(key, iv) * */ __pyx_tuple__2 = PyTuple_Pack(1, __pyx_kp_s_EVP_DecryptInit_ex_failed); if (unlikely(!__pyx_tuple__2)) __PYX_ERR(0, 105, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__2); __Pyx_GIVEREF(__pyx_tuple__2); /* "borg/crypto.pyx":125 * if self.is_encrypt: * if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): * raise Exception('EVP_EncryptInit_ex failed') # <<<<<<<<<<<<<< * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): */ __pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_s_EVP_EncryptInit_ex_failed); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 125, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple_); __Pyx_GIVEREF(__pyx_tuple_); /* "borg/crypto.pyx":128 * else: # decrypt * if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): * raise Exception('EVP_DecryptInit_ex failed') # <<<<<<<<<<<<<< * * @property */ __pyx_tuple__2 = PyTuple_Pack(1, __pyx_kp_s_EVP_DecryptInit_ex_failed); if (unlikely(!__pyx_tuple__2)) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__2); __Pyx_GIVEREF(__pyx_tuple__2); /* "borg/crypto.pyx":144 * try: * if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_EncryptUpdate failed') # <<<<<<<<<<<<<< * ctl = outl * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): */ __pyx_tuple__3 = PyTuple_Pack(1, __pyx_kp_s_EVP_EncryptUpdate_failed); if (unlikely(!__pyx_tuple__3)) __PYX_ERR(0, 144, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__3); __Pyx_GIVEREF(__pyx_tuple__3); /* "borg/crypto.pyx":147 * ctl = outl * if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): * raise Exception('EVP_EncryptFinal failed') # <<<<<<<<<<<<<< * ctl += outl * self.blocks += num_aes_blocks(ctl) */ __pyx_tuple__4 = PyTuple_Pack(1, __pyx_kp_s_EVP_EncryptFinal_failed); if (unlikely(!__pyx_tuple__4)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__4); __Pyx_GIVEREF(__pyx_tuple__4); /* "borg/crypto.pyx":166 * try: * if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): * raise Exception('EVP_DecryptUpdate failed') # <<<<<<<<<<<<<< * ptl = outl * if EVP_DecryptFinal_ex(self.ctx, out+ptl, &outl) <= 0: */ __pyx_tuple__5 = PyTuple_Pack(1, __pyx_kp_s_EVP_DecryptUpdate_failed); if (unlikely(!__pyx_tuple__5)) __PYX_ERR(0, 166, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__5); __Pyx_GIVEREF(__pyx_tuple__5); /* "borg/crypto.pyx":172 * # authentication. for them, a failure here means corrupted data. * # CTR mode does not use padding nor authentication. * raise Exception('EVP_DecryptFinal failed') # <<<<<<<<<<<<<< * ptl += outl * self.blocks += num_aes_blocks(inl) */ __pyx_tuple__6 = PyTuple_Pack(1, __pyx_kp_s_EVP_DecryptFinal_failed); if (unlikely(!__pyx_tuple__6)) __PYX_ERR(0, 172, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__6); __Pyx_GIVEREF(__pyx_tuple__6); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple__7 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__7)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__7); __Pyx_GIVEREF(__pyx_tuple__7); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__8 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__8); __Pyx_GIVEREF(__pyx_tuple__8); /* "borg/crypto.pyx":193 * # Step 1. HKDF-Extract (ikm, salt) -> prk * if salt is None: * salt = bytes(64) # <<<<<<<<<<<<<< * prk = hmac.HMAC(salt, ikm, hashlib.sha512).digest() * */ __pyx_tuple__9 = PyTuple_Pack(1, __pyx_int_64); if (unlikely(!__pyx_tuple__9)) __PYX_ERR(0, 193, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__9); __Pyx_GIVEREF(__pyx_tuple__9); /* "borg/crypto.pyx":201 * output = b'' * for i in range(n): * msg = t_n + info + (i + 1).to_bytes(1, 'little') # <<<<<<<<<<<<<< * t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() * output += t_n */ __pyx_tuple__11 = PyTuple_Pack(2, __pyx_int_1, __pyx_n_s_little); if (unlikely(!__pyx_tuple__11)) __PYX_ERR(0, 201, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__11); __Pyx_GIVEREF(__pyx_tuple__11); /* "borg/crypto.pyx":44 * import struct * * _int = struct.Struct('>I') # <<<<<<<<<<<<<< * _long = struct.Struct('>Q') * _2long = struct.Struct('>QQ') */ __pyx_tuple__12 = PyTuple_Pack(1, __pyx_kp_s_I); if (unlikely(!__pyx_tuple__12)) __PYX_ERR(0, 44, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__12); __Pyx_GIVEREF(__pyx_tuple__12); /* "borg/crypto.pyx":45 * * _int = struct.Struct('>I') * _long = struct.Struct('>Q') # <<<<<<<<<<<<<< * _2long = struct.Struct('>QQ') * */ __pyx_tuple__13 = PyTuple_Pack(1, __pyx_kp_s_Q); if (unlikely(!__pyx_tuple__13)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__13); __Pyx_GIVEREF(__pyx_tuple__13); /* "borg/crypto.pyx":46 * _int = struct.Struct('>I') * _long = struct.Struct('>Q') * _2long = struct.Struct('>QQ') # <<<<<<<<<<<<<< * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] */ __pyx_tuple__14 = PyTuple_Pack(1, __pyx_kp_s_QQ); if (unlikely(!__pyx_tuple__14)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__14); __Pyx_GIVEREF(__pyx_tuple__14); /* "borg/crypto.pyx":48 * _2long = struct.Struct('>QQ') * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] # <<<<<<<<<<<<<< * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] * long_to_bytes = lambda x: _long.pack(x) */ __pyx_tuple__15 = PyTuple_Pack(1, ((PyObject *)__pyx_int_0)); if (unlikely(!__pyx_tuple__15)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__15); __Pyx_GIVEREF(__pyx_tuple__15); /* "borg/crypto.pyx":49 * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] # <<<<<<<<<<<<<< * long_to_bytes = lambda x: _long.pack(x) * */ __pyx_tuple__16 = PyTuple_Pack(1, ((PyObject *)__pyx_int_0)); if (unlikely(!__pyx_tuple__16)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__16); __Pyx_GIVEREF(__pyx_tuple__16); /* "borg/crypto.pyx":53 * * * def bytes16_to_int(b, offset=0): # <<<<<<<<<<<<<< * h, l = _2long.unpack_from(b, offset) * return (h << 64) + l */ __pyx_tuple__17 = PyTuple_Pack(4, __pyx_n_s_b, __pyx_n_s_offset, __pyx_n_s_h, __pyx_n_s_l); if (unlikely(!__pyx_tuple__17)) __PYX_ERR(0, 53, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__17); __Pyx_GIVEREF(__pyx_tuple__17); __pyx_codeobj__18 = (PyObject*)__Pyx_PyCode_New(2, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__17, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_crypto_pyx, __pyx_n_s_bytes16_to_int, 53, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__18)) __PYX_ERR(0, 53, __pyx_L1_error) /* "borg/crypto.pyx":58 * * * def int_to_bytes16(i): # <<<<<<<<<<<<<< * max_uint64 = 0xffffffffffffffff * l = i & max_uint64 */ __pyx_tuple__19 = PyTuple_Pack(4, __pyx_n_s_i, __pyx_n_s_max_uint64, __pyx_n_s_l, __pyx_n_s_h); if (unlikely(!__pyx_tuple__19)) __PYX_ERR(0, 58, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__19); __Pyx_GIVEREF(__pyx_tuple__19); __pyx_codeobj__20 = (PyObject*)__Pyx_PyCode_New(1, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__19, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_crypto_pyx, __pyx_n_s_int_to_bytes16, 58, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__20)) __PYX_ERR(0, 58, __pyx_L1_error) /* "borg/crypto.pyx":65 * * * def increment_iv(iv, amount=1): # <<<<<<<<<<<<<< * """ * Increment the IV by the given amount (default 1). */ __pyx_tuple__21 = PyTuple_Pack(2, __pyx_n_s_iv, __pyx_n_s_amount); if (unlikely(!__pyx_tuple__21)) __PYX_ERR(0, 65, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__21); __Pyx_GIVEREF(__pyx_tuple__21); __pyx_codeobj__22 = (PyObject*)__Pyx_PyCode_New(2, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__21, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_crypto_pyx, __pyx_n_s_increment_iv, 65, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__22)) __PYX_ERR(0, 65, __pyx_L1_error) /* "borg/crypto.pyx":80 * * * def num_aes_blocks(int length): # <<<<<<<<<<<<<< * """Return the number of AES blocks required to encrypt/decrypt *length* bytes of data. * Note: this is only correct for modes without padding, like AES-CTR. */ __pyx_tuple__23 = PyTuple_Pack(2, __pyx_n_s_length, __pyx_n_s_length); if (unlikely(!__pyx_tuple__23)) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__23); __Pyx_GIVEREF(__pyx_tuple__23); __pyx_codeobj__24 = (PyObject*)__Pyx_PyCode_New(1, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__23, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_crypto_pyx, __pyx_n_s_num_aes_blocks, 80, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__24)) __PYX_ERR(0, 80, __pyx_L1_error) /* "borg/crypto.pyx":180 * * * def hkdf_hmac_sha512(ikm, salt, info, output_length): # <<<<<<<<<<<<<< * """ * Compute HKDF-HMAC-SHA512 with input key material *ikm*, *salt* and *info* to produce *output_length* bytes. */ __pyx_tuple__25 = PyTuple_Pack(11, __pyx_n_s_ikm, __pyx_n_s_salt, __pyx_n_s_info, __pyx_n_s_output_length, __pyx_n_s_digest_length, __pyx_n_s_prk, __pyx_n_s_n, __pyx_n_s_t_n, __pyx_n_s_output, __pyx_n_s_i, __pyx_n_s_msg); if (unlikely(!__pyx_tuple__25)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__25); __Pyx_GIVEREF(__pyx_tuple__25); __pyx_codeobj__26 = (PyObject*)__Pyx_PyCode_New(4, 0, 11, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__25, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_crypto_pyx, __pyx_n_s_hkdf_hmac_sha512, 180, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__26)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(0, 1, __pyx_L1_error); __pyx_int_0 = PyInt_FromLong(0); if (unlikely(!__pyx_int_0)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_1 = PyInt_FromLong(1); if (unlikely(!__pyx_int_1)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_64 = PyInt_FromLong(64); if (unlikely(!__pyx_int_64)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_255 = PyInt_FromLong(255); if (unlikely(!__pyx_int_255)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_18446744073709551615 = PyInt_FromString((char *)"18446744073709551615", 0, 0); if (unlikely(!__pyx_int_18446744073709551615)) __PYX_ERR(0, 1, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ if (PyType_Ready(&__pyx_type_4borg_6crypto_AES) < 0) __PYX_ERR(0, 87, __pyx_L1_error) __pyx_type_4borg_6crypto_AES.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_6crypto_AES.tp_dictoffset && __pyx_type_4borg_6crypto_AES.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_6crypto_AES.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (PyObject_SetAttr(__pyx_m, __pyx_n_s_AES, (PyObject *)&__pyx_type_4borg_6crypto_AES) < 0) __PYX_ERR(0, 87, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_6crypto_AES) < 0) __PYX_ERR(0, 87, __pyx_L1_error) __pyx_ptype_4borg_6crypto_AES = &__pyx_type_4borg_6crypto_AES; __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC initcrypto(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC initcrypto(void) #else __Pyx_PyMODINIT_FUNC PyInit_crypto(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_crypto(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_crypto(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'crypto' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_crypto(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("crypto", __pyx_methods, __pyx_k_A_thin_OpenSSL_wrapper_This_coul, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(0, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(0, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(0, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(0, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__crypto) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name, __pyx_n_s_main) < 0) __PYX_ERR(0, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(0, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.crypto")) { if (unlikely(PyDict_SetItemString(modules, "borg.crypto", __pyx_m) < 0)) __PYX_ERR(0, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); if (unlikely(__Pyx_modinit_type_init_code() != 0)) goto __pyx_L1_error; (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /* "borg/crypto.pyx":5 * This could be replaced by PyCrypto maybe? * """ * import hashlib # <<<<<<<<<<<<<< * import hmac * from math import ceil */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_hashlib, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_hashlib, __pyx_t_1) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/crypto.pyx":6 * """ * import hashlib * import hmac # <<<<<<<<<<<<<< * from math import ceil * */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_hmac, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 6, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_hmac, __pyx_t_1) < 0) __PYX_ERR(0, 6, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/crypto.pyx":7 * import hashlib * import hmac * from math import ceil # <<<<<<<<<<<<<< * * from libc.stdlib cimport malloc, free */ __pyx_t_1 = PyList_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_n_s_ceil); __Pyx_GIVEREF(__pyx_n_s_ceil); PyList_SET_ITEM(__pyx_t_1, 0, __pyx_n_s_ceil); __pyx_t_2 = __Pyx_Import(__pyx_n_s_math, __pyx_t_1, -1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_ceil); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_ceil, __pyx_t_1) < 0) __PYX_ERR(0, 7, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":11 * from libc.stdlib cimport malloc, free * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * cdef extern from "openssl/rand.h": */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(0, 11, __pyx_L1_error) /* "borg/crypto.pyx":42 * * * import struct # <<<<<<<<<<<<<< * * _int = struct.Struct('>I') */ __pyx_t_2 = __Pyx_Import(__pyx_n_s_struct, 0, -1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_struct, __pyx_t_2) < 0) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":44 * import struct * * _int = struct.Struct('>I') # <<<<<<<<<<<<<< * _long = struct.Struct('>Q') * _2long = struct.Struct('>QQ') */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_struct); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 44, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_Struct); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 44, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_tuple__12, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 44, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_int, __pyx_t_2) < 0) __PYX_ERR(0, 44, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":45 * * _int = struct.Struct('>I') * _long = struct.Struct('>Q') # <<<<<<<<<<<<<< * _2long = struct.Struct('>QQ') * */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_struct); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_Struct); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_tuple__13, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_long, __pyx_t_2) < 0) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":46 * _int = struct.Struct('>I') * _long = struct.Struct('>Q') * _2long = struct.Struct('>QQ') # <<<<<<<<<<<<<< * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_struct); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_Struct); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_tuple__14, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_2long, __pyx_t_2) < 0) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":48 * _2long = struct.Struct('>QQ') * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] # <<<<<<<<<<<<<< * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] * long_to_bytes = lambda x: _long.pack(x) */ __pyx_t_2 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_6crypto_10lambda, 0, __pyx_n_s_lambda, NULL, __pyx_n_s_borg_crypto, __pyx_d, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_2, __pyx_tuple__15); if (PyDict_SetItem(__pyx_d, __pyx_n_s_bytes_to_int, __pyx_t_2) < 0) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":49 * * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] # <<<<<<<<<<<<<< * long_to_bytes = lambda x: _long.pack(x) * */ __pyx_t_2 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_6crypto_11lambda1, 0, __pyx_n_s_lambda, NULL, __pyx_n_s_borg_crypto, __pyx_d, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_CyFunction_SetDefaultsTuple(__pyx_t_2, __pyx_tuple__16); if (PyDict_SetItem(__pyx_d, __pyx_n_s_bytes_to_long, __pyx_t_2) < 0) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":50 * bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] * bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] * long_to_bytes = lambda x: _long.pack(x) # <<<<<<<<<<<<<< * * */ __pyx_t_2 = __Pyx_CyFunction_NewEx(&__pyx_mdef_4borg_6crypto_12lambda2, 0, __pyx_n_s_lambda, NULL, __pyx_n_s_borg_crypto, __pyx_d, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_long_to_bytes, __pyx_t_2) < 0) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":53 * * * def bytes16_to_int(b, offset=0): # <<<<<<<<<<<<<< * h, l = _2long.unpack_from(b, offset) * return (h << 64) + l */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_6crypto_1bytes16_to_int, NULL, __pyx_n_s_borg_crypto); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 53, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_bytes16_to_int, __pyx_t_2) < 0) __PYX_ERR(0, 53, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":58 * * * def int_to_bytes16(i): # <<<<<<<<<<<<<< * max_uint64 = 0xffffffffffffffff * l = i & max_uint64 */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_6crypto_3int_to_bytes16, NULL, __pyx_n_s_borg_crypto); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 58, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_int_to_bytes16, __pyx_t_2) < 0) __PYX_ERR(0, 58, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":65 * * * def increment_iv(iv, amount=1): # <<<<<<<<<<<<<< * """ * Increment the IV by the given amount (default 1). */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_6crypto_5increment_iv, NULL, __pyx_n_s_borg_crypto); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 65, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_increment_iv, __pyx_t_2) < 0) __PYX_ERR(0, 65, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":80 * * * def num_aes_blocks(int length): # <<<<<<<<<<<<<< * """Return the number of AES blocks required to encrypt/decrypt *length* bytes of data. * Note: this is only correct for modes without padding, like AES-CTR. */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_6crypto_7num_aes_blocks, NULL, __pyx_n_s_borg_crypto); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_num_aes_blocks, __pyx_t_2) < 0) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":180 * * * def hkdf_hmac_sha512(ikm, salt, info, output_length): # <<<<<<<<<<<<<< * """ * Compute HKDF-HMAC-SHA512 with input key material *ikm*, *salt* and *info* to produce *output_length* bytes. */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_6crypto_9hkdf_hmac_sha512, NULL, __pyx_n_s_borg_crypto); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_hkdf_hmac_sha512, __pyx_t_2) < 0) __PYX_ERR(0, 180, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/crypto.pyx":1 * """A thin OpenSSL wrapper # <<<<<<<<<<<<<< * * This could be replaced by PyCrypto maybe? */ __pyx_t_2 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_2) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.crypto", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.crypto"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* GetModuleGlobalName */ #if CYTHON_USE_DICT_VERSIONS static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value) #else static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name) #endif { PyObject *result; #if !CYTHON_AVOID_BORROWED_REFS #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 result = _PyDict_GetItem_KnownHash(__pyx_d, name, ((PyASCIIObject *) name)->hash); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } else if (unlikely(PyErr_Occurred())) { return NULL; } #else result = PyDict_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } #endif #else result = PyObject_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } PyErr_Clear(); #endif return __Pyx_GetBuiltinName(name); } /* PyFunctionFastCall */ #if CYTHON_FAST_PYCALL static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject **fastlocals; Py_ssize_t i; PyObject *result; assert(globals != NULL); /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) { return NULL; } fastlocals = __Pyx_PyFrame_GetLocalsplus(f); for (i = 0; i < na; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } result = PyEval_EvalFrameEx(f,0); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; return result; } #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *closure; #if PY_MAJOR_VERSION >= 3 PyObject *kwdefs; #endif PyObject *kwtuple, **k; PyObject **d; Py_ssize_t nd; Py_ssize_t nk; PyObject *result; assert(kwargs == NULL || PyDict_Check(kwargs)); nk = kwargs ? PyDict_Size(kwargs) : 0; if (Py_EnterRecursiveCall((char*)" while calling a Python object")) { return NULL; } if ( #if PY_MAJOR_VERSION >= 3 co->co_kwonlyargcount == 0 && #endif likely(kwargs == NULL || nk == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { result = __Pyx_PyFunction_FastCallNoKw(co, args, nargs, globals); goto done; } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); result =__Pyx_PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), globals); goto done; } } if (kwargs != NULL) { Py_ssize_t pos, i; kwtuple = PyTuple_New(2 * nk); if (kwtuple == NULL) { result = NULL; goto done; } k = &PyTuple_GET_ITEM(kwtuple, 0); pos = i = 0; while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { Py_INCREF(k[i]); Py_INCREF(k[i+1]); i += 2; } nk = i / 2; } else { kwtuple = NULL; k = NULL; } closure = PyFunction_GET_CLOSURE(func); #if PY_MAJOR_VERSION >= 3 kwdefs = PyFunction_GET_KW_DEFAULTS(func); #endif if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } else { d = NULL; nd = 0; } #if PY_MAJOR_VERSION >= 3 result = PyEval_EvalCodeEx((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, kwdefs, closure); #else result = PyEval_EvalCodeEx(co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, closure); #endif Py_XDECREF(kwtuple); done: Py_LeaveRecursiveCall(); return result; } #endif #endif /* PyCFunctionFastCall */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, Py_ssize_t nargs) { PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); int flags = PyCFunction_GET_FLAGS(func); assert(PyCFunction_Check(func)); assert(METH_FASTCALL == (flags & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))); assert(nargs >= 0); assert(nargs == 0 || args != NULL); /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); if ((PY_VERSION_HEX < 0x030700A0) || unlikely(flags & METH_KEYWORDS)) { return (*((__Pyx_PyCFunctionFastWithKeywords)(void*)meth)) (self, args, nargs, NULL); } else { return (*((__Pyx_PyCFunctionFast)(void*)meth)) (self, args, nargs); } } #endif /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* GetItemInt */ static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) { PyObject *r; if (!j) return NULL; r = PyObject_GetItem(o, j); Py_DECREF(j); return r; } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyList_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyList_GET_SIZE(o)))) { PyObject *r = PyList_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyTuple_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyList_GET_SIZE(o); if ((!boundscheck) || (likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o))))) { PyObject *r = PyList_GET_ITEM(o, n); Py_INCREF(r); return r; } } else if (PyTuple_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyTuple_GET_SIZE(o); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, n); Py_INCREF(r); return r; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return NULL; PyErr_Clear(); } } return m->sq_item(o, i); } } #else if (is_list || PySequence_Check(o)) { return PySequence_GetItem(o, i); } #endif return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); } /* PyObjectCall2Args */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2) { PyObject *args, *result = NULL; #if CYTHON_FAST_PYCALL if (PyFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyFunction_FastCall(function, args, 2); } #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyCFunction_FastCall(function, args, 2); } #endif args = PyTuple_New(2); if (unlikely(!args)) goto done; Py_INCREF(arg1); PyTuple_SET_ITEM(args, 0, arg1); Py_INCREF(arg2); PyTuple_SET_ITEM(args, 1, arg2); Py_INCREF(function); result = __Pyx_PyObject_Call(function, args, NULL); Py_DECREF(args); Py_DECREF(function); done: return result; } /* PyObjectCallMethO */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg) { PyObject *self, *result; PyCFunction cfunc; cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = cfunc(self, arg); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCallOneArg */ #if CYTHON_COMPILING_IN_CPYTHON static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_New(1); if (unlikely(!args)) return NULL; Py_INCREF(arg); PyTuple_SET_ITEM(args, 0, arg); result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, &arg, 1); } #endif if (likely(PyCFunction_Check(func))) { if (likely(PyCFunction_GET_FLAGS(func) & METH_O)) { return __Pyx_PyObject_CallMethO(func, arg); #if CYTHON_FAST_PYCCALL } else if (PyCFunction_GET_FLAGS(func) & METH_FASTCALL) { return __Pyx_PyCFunction_FastCall(func, &arg, 1); #endif } } return __Pyx__PyObject_CallOneArg(func, arg); } #else static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_Pack(1, arg); if (unlikely(!args)) return NULL; result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } #endif /* RaiseTooManyValuesToUnpack */ static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected) { PyErr_Format(PyExc_ValueError, "too many values to unpack (expected %" CYTHON_FORMAT_SSIZE_T "d)", expected); } /* RaiseNeedMoreValuesToUnpack */ static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index) { PyErr_Format(PyExc_ValueError, "need more than %" CYTHON_FORMAT_SSIZE_T "d value%.1s to unpack", index, (index == 1) ? "" : "s"); } /* IterFinish */ static CYTHON_INLINE int __Pyx_IterFinish(void) { #if CYTHON_FAST_THREAD_STATE PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* exc_type = tstate->curexc_type; if (unlikely(exc_type)) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) { PyObject *exc_value, *exc_tb; exc_value = tstate->curexc_value; exc_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; Py_DECREF(exc_type); Py_XDECREF(exc_value); Py_XDECREF(exc_tb); return 0; } else { return -1; } } return 0; #else if (unlikely(PyErr_Occurred())) { if (likely(PyErr_ExceptionMatches(PyExc_StopIteration))) { PyErr_Clear(); return 0; } else { return -1; } } return 0; #endif } /* UnpackItemEndCheck */ static int __Pyx_IternextUnpackEndCheck(PyObject *retval, Py_ssize_t expected) { if (unlikely(retval)) { Py_DECREF(retval); __Pyx_RaiseTooManyValuesError(expected); return -1; } else { return __Pyx_IterFinish(); } return 0; } /* None */ static CYTHON_INLINE long __Pyx_div_long(long a, long b) { long q = a / b; long r = a - q*b; q -= ((r != 0) & ((r ^ b) < 0)); return q; } /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* RaiseException */ #if PY_MAJOR_VERSION < 3 static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, CYTHON_UNUSED PyObject *cause) { __Pyx_PyThreadState_declare Py_XINCREF(type); if (!value || value == Py_None) value = NULL; else Py_INCREF(value); if (!tb || tb == Py_None) tb = NULL; else { Py_INCREF(tb); if (!PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto raise_error; } } if (PyType_Check(type)) { #if CYTHON_COMPILING_IN_PYPY if (!value) { Py_INCREF(Py_None); value = Py_None; } #endif PyErr_NormalizeException(&type, &value, &tb); } else { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto raise_error; } value = type; type = (PyObject*) Py_TYPE(type); Py_INCREF(type); if (!PyType_IsSubtype((PyTypeObject *)type, (PyTypeObject *)PyExc_BaseException)) { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto raise_error; } } __Pyx_PyThreadState_assign __Pyx_ErrRestore(type, value, tb); return; raise_error: Py_XDECREF(value); Py_XDECREF(type); Py_XDECREF(tb); return; } #else static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause) { PyObject* owned_instance = NULL; if (tb == Py_None) { tb = 0; } else if (tb && !PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto bad; } if (value == Py_None) value = 0; if (PyExceptionInstance_Check(type)) { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto bad; } value = type; type = (PyObject*) Py_TYPE(value); } else if (PyExceptionClass_Check(type)) { PyObject *instance_class = NULL; if (value && PyExceptionInstance_Check(value)) { instance_class = (PyObject*) Py_TYPE(value); if (instance_class != type) { int is_subclass = PyObject_IsSubclass(instance_class, type); if (!is_subclass) { instance_class = NULL; } else if (unlikely(is_subclass == -1)) { goto bad; } else { type = instance_class; } } } if (!instance_class) { PyObject *args; if (!value) args = PyTuple_New(0); else if (PyTuple_Check(value)) { Py_INCREF(value); args = value; } else args = PyTuple_Pack(1, value); if (!args) goto bad; owned_instance = PyObject_Call(type, args, NULL); Py_DECREF(args); if (!owned_instance) goto bad; value = owned_instance; if (!PyExceptionInstance_Check(value)) { PyErr_Format(PyExc_TypeError, "calling %R should have returned an instance of " "BaseException, not %R", type, Py_TYPE(value)); goto bad; } } } else { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto bad; } if (cause) { PyObject *fixed_cause; if (cause == Py_None) { fixed_cause = NULL; } else if (PyExceptionClass_Check(cause)) { fixed_cause = PyObject_CallObject(cause, NULL); if (fixed_cause == NULL) goto bad; } else if (PyExceptionInstance_Check(cause)) { fixed_cause = cause; Py_INCREF(fixed_cause); } else { PyErr_SetString(PyExc_TypeError, "exception causes must derive from " "BaseException"); goto bad; } PyException_SetCause(value, fixed_cause); } PyErr_SetObject(type, value); if (tb) { #if CYTHON_COMPILING_IN_PYPY PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_Fetch(&tmp_type, &tmp_value, &tmp_tb); Py_INCREF(tb); PyErr_Restore(tmp_type, tmp_value, tb); Py_XDECREF(tmp_tb); #else PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* tmp_tb = tstate->curexc_traceback; if (tb != tmp_tb) { Py_INCREF(tb); tstate->curexc_traceback = tb; Py_XDECREF(tmp_tb); } #endif } bad: Py_XDECREF(owned_instance); return; } #endif /* GetException */ #if CYTHON_FAST_THREAD_STATE static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb) #endif { PyObject *local_type, *local_value, *local_tb; #if CYTHON_FAST_THREAD_STATE PyObject *tmp_type, *tmp_value, *tmp_tb; local_type = tstate->curexc_type; local_value = tstate->curexc_value; local_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; #else PyErr_Fetch(&local_type, &local_value, &local_tb); #endif PyErr_NormalizeException(&local_type, &local_value, &local_tb); #if CYTHON_FAST_THREAD_STATE if (unlikely(tstate->curexc_type)) #else if (unlikely(PyErr_Occurred())) #endif goto bad; #if PY_MAJOR_VERSION >= 3 if (local_tb) { if (unlikely(PyException_SetTraceback(local_value, local_tb) < 0)) goto bad; } #endif Py_XINCREF(local_tb); Py_XINCREF(local_type); Py_XINCREF(local_value); *type = local_type; *value = local_value; *tb = local_tb; #if CYTHON_FAST_THREAD_STATE #if CYTHON_USE_EXC_INFO_STACK { _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = local_type; exc_info->exc_value = local_value; exc_info->exc_traceback = local_tb; } #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = local_type; tstate->exc_value = local_value; tstate->exc_traceback = local_tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); #else PyErr_SetExcInfo(local_type, local_value, local_tb); #endif return 0; bad: *type = 0; *value = 0; *tb = 0; Py_XDECREF(local_type); Py_XDECREF(local_value); Py_XDECREF(local_tb); return -1; } /* SwapException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = *type; exc_info->exc_value = *value; exc_info->exc_traceback = *tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = *type; tstate->exc_value = *value; tstate->exc_traceback = *tb; #endif *type = tmp_type; *value = tmp_value; *tb = tmp_tb; } #else static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_GetExcInfo(&tmp_type, &tmp_value, &tmp_tb); PyErr_SetExcInfo(*type, *value, *tb); *type = tmp_type; *value = tmp_value; *tb = tmp_tb; } #endif /* GetTopmostException */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate) { _PyErr_StackItem *exc_info = tstate->exc_info; while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && exc_info->previous_item != NULL) { exc_info = exc_info->previous_item; } return exc_info; } #endif /* SaveResetException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = __Pyx_PyErr_GetTopmostException(tstate); *type = exc_info->exc_type; *value = exc_info->exc_value; *tb = exc_info->exc_traceback; #else *type = tstate->exc_type; *value = tstate->exc_value; *tb = tstate->exc_traceback; #endif Py_XINCREF(*type); Py_XINCREF(*value); Py_XINCREF(*tb); } static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = type; exc_info->exc_value = value; exc_info->exc_traceback = tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = type; tstate->exc_value = value; tstate->exc_traceback = tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } #endif /* PyObjectCallNoArg */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallNoArg(PyObject *func) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, NULL, 0); } #endif #ifdef __Pyx_CyFunction_USED if (likely(PyCFunction_Check(func) || __Pyx_CyFunction_Check(func))) #else if (likely(PyCFunction_Check(func))) #endif { if (likely(PyCFunction_GET_FLAGS(func) & METH_NOARGS)) { return __Pyx_PyObject_CallMethO(func, NULL); } } return __Pyx_PyObject_Call(func, __pyx_empty_tuple, NULL); } #endif /* PyIntBinop */ #if !CYTHON_COMPILING_IN_PYPY static PyObject* __Pyx_PyInt_AddObjC(PyObject *op1, PyObject *op2, CYTHON_UNUSED long intval, CYTHON_UNUSED int inplace) { #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(op1))) { const long b = intval; long x; long a = PyInt_AS_LONG(op1); x = (long)((unsigned long)a + b); if (likely((x^a) >= 0 || (x^b) >= 0)) return PyInt_FromLong(x); return PyLong_Type.tp_as_number->nb_add(op1, op2); } #endif #if CYTHON_USE_PYLONG_INTERNALS if (likely(PyLong_CheckExact(op1))) { const long b = intval; long a, x; #ifdef HAVE_LONG_LONG const PY_LONG_LONG llb = intval; PY_LONG_LONG lla, llx; #endif const digit* digits = ((PyLongObject*)op1)->ob_digit; const Py_ssize_t size = Py_SIZE(op1); if (likely(__Pyx_sst_abs(size) <= 1)) { a = likely(size) ? digits[0] : 0; if (size == -1) a = -a; } else { switch (size) { case -2: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { a = -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 2: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { a = (long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case -3: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { a = -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((((unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 3: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { a = (long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((((unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case -4: if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { a = -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { lla = -(PY_LONG_LONG) (((((((((unsigned PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; case 4: if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { a = (long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0])); break; #ifdef HAVE_LONG_LONG } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { lla = (PY_LONG_LONG) (((((((((unsigned PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (unsigned PY_LONG_LONG)digits[0])); goto long_long; #endif } CYTHON_FALLTHROUGH; default: return PyLong_Type.tp_as_number->nb_add(op1, op2); } } x = a + b; return PyLong_FromLong(x); #ifdef HAVE_LONG_LONG long_long: llx = lla + llb; return PyLong_FromLongLong(llx); #endif } #endif if (PyFloat_CheckExact(op1)) { const long b = intval; double a = PyFloat_AS_DOUBLE(op1); double result; PyFPE_START_PROTECT("add", return NULL) result = ((double)a) + (double)b; PyFPE_END_PROTECT(result) return PyFloat_FromDouble(result); } return (inplace ? PyNumber_InPlaceAdd : PyNumber_Add)(op1, op2); } #endif /* SliceObject */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice(PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** _py_start, PyObject** _py_stop, PyObject** _py_slice, int has_cstart, int has_cstop, CYTHON_UNUSED int wraparound) { #if CYTHON_USE_TYPE_SLOTS PyMappingMethods* mp; #if PY_MAJOR_VERSION < 3 PySequenceMethods* ms = Py_TYPE(obj)->tp_as_sequence; if (likely(ms && ms->sq_slice)) { if (!has_cstart) { if (_py_start && (*_py_start != Py_None)) { cstart = __Pyx_PyIndex_AsSsize_t(*_py_start); if ((cstart == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstart = 0; } if (!has_cstop) { if (_py_stop && (*_py_stop != Py_None)) { cstop = __Pyx_PyIndex_AsSsize_t(*_py_stop); if ((cstop == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstop = PY_SSIZE_T_MAX; } if (wraparound && unlikely((cstart < 0) | (cstop < 0)) && likely(ms->sq_length)) { Py_ssize_t l = ms->sq_length(obj); if (likely(l >= 0)) { if (cstop < 0) { cstop += l; if (cstop < 0) cstop = 0; } if (cstart < 0) { cstart += l; if (cstart < 0) cstart = 0; } } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) goto bad; PyErr_Clear(); } } return ms->sq_slice(obj, cstart, cstop); } #endif mp = Py_TYPE(obj)->tp_as_mapping; if (likely(mp && mp->mp_subscript)) #endif { PyObject* result; PyObject *py_slice, *py_start, *py_stop; if (_py_slice) { py_slice = *_py_slice; } else { PyObject* owned_start = NULL; PyObject* owned_stop = NULL; if (_py_start) { py_start = *_py_start; } else { if (has_cstart) { owned_start = py_start = PyInt_FromSsize_t(cstart); if (unlikely(!py_start)) goto bad; } else py_start = Py_None; } if (_py_stop) { py_stop = *_py_stop; } else { if (has_cstop) { owned_stop = py_stop = PyInt_FromSsize_t(cstop); if (unlikely(!py_stop)) { Py_XDECREF(owned_start); goto bad; } } else py_stop = Py_None; } py_slice = PySlice_New(py_start, py_stop, Py_None); Py_XDECREF(owned_start); Py_XDECREF(owned_stop); if (unlikely(!py_slice)) goto bad; } #if CYTHON_USE_TYPE_SLOTS result = mp->mp_subscript(obj, py_slice); #else result = PyObject_GetItem(obj, py_slice); #endif if (!_py_slice) { Py_DECREF(py_slice); } return result; } PyErr_Format(PyExc_TypeError, "'%.200s' object is unsliceable", Py_TYPE(obj)->tp_name); bad: return NULL; } /* PyObject_GenericGetAttrNoDict */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject *__Pyx_RaiseGenericGetAttributeError(PyTypeObject *tp, PyObject *attr_name) { PyErr_Format(PyExc_AttributeError, #if PY_MAJOR_VERSION >= 3 "'%.50s' object has no attribute '%U'", tp->tp_name, attr_name); #else "'%.50s' object has no attribute '%.400s'", tp->tp_name, PyString_AS_STRING(attr_name)); #endif return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name) { PyObject *descr; PyTypeObject *tp = Py_TYPE(obj); if (unlikely(!PyString_Check(attr_name))) { return PyObject_GenericGetAttr(obj, attr_name); } assert(!tp->tp_dictoffset); descr = _PyType_Lookup(tp, attr_name); if (unlikely(!descr)) { return __Pyx_RaiseGenericGetAttributeError(tp, attr_name); } Py_INCREF(descr); #if PY_MAJOR_VERSION < 3 if (likely(PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_HAVE_CLASS))) #endif { descrgetfunc f = Py_TYPE(descr)->tp_descr_get; if (unlikely(f)) { PyObject *res = f(descr, obj, (PyObject *)tp); Py_DECREF(descr); return res; } } return descr; } #endif /* PyObject_GenericGetAttr */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name) { if (unlikely(Py_TYPE(obj)->tp_dictoffset)) { return PyObject_GenericGetAttr(obj, attr_name); } return __Pyx_PyObject_GenericGetAttrNoDict(obj, attr_name); } #endif /* SetupReduce */ static int __Pyx_setup_reduce_is_named(PyObject* meth, PyObject* name) { int ret; PyObject *name_attr; name_attr = __Pyx_PyObject_GetAttrStr(meth, __pyx_n_s_name); if (likely(name_attr)) { ret = PyObject_RichCompareBool(name_attr, name, Py_EQ); } else { ret = -1; } if (unlikely(ret < 0)) { PyErr_Clear(); ret = 0; } Py_XDECREF(name_attr); return ret; } static int __Pyx_setup_reduce(PyObject* type_obj) { int ret = 0; PyObject *object_reduce = NULL; PyObject *object_reduce_ex = NULL; PyObject *reduce = NULL; PyObject *reduce_ex = NULL; PyObject *reduce_cython = NULL; PyObject *setstate = NULL; PyObject *setstate_cython = NULL; #if CYTHON_USE_PYTYPE_LOOKUP if (_PyType_Lookup((PyTypeObject*)type_obj, __pyx_n_s_getstate)) goto GOOD; #else if (PyObject_HasAttr(type_obj, __pyx_n_s_getstate)) goto GOOD; #endif #if CYTHON_USE_PYTYPE_LOOKUP object_reduce_ex = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #else object_reduce_ex = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #endif reduce_ex = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_ex); if (unlikely(!reduce_ex)) goto BAD; if (reduce_ex == object_reduce_ex) { #if CYTHON_USE_PYTYPE_LOOKUP object_reduce = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #else object_reduce = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #endif reduce = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce); if (unlikely(!reduce)) goto BAD; if (reduce == object_reduce || __Pyx_setup_reduce_is_named(reduce, __pyx_n_s_reduce_cython)) { reduce_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_cython); if (unlikely(!reduce_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce, reduce_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce_cython); if (unlikely(ret < 0)) goto BAD; setstate = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate); if (!setstate) PyErr_Clear(); if (!setstate || __Pyx_setup_reduce_is_named(setstate, __pyx_n_s_setstate_cython)) { setstate_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate_cython); if (unlikely(!setstate_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate, setstate_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate_cython); if (unlikely(ret < 0)) goto BAD; } PyType_Modified((PyTypeObject*)type_obj); } } goto GOOD; BAD: if (!PyErr_Occurred()) PyErr_Format(PyExc_RuntimeError, "Unable to initialize pickling for %s", ((PyTypeObject*)type_obj)->tp_name); ret = -1; GOOD: #if !CYTHON_USE_PYTYPE_LOOKUP Py_XDECREF(object_reduce); Py_XDECREF(object_reduce_ex); #endif Py_XDECREF(reduce); Py_XDECREF(reduce_ex); Py_XDECREF(reduce_cython); Py_XDECREF(setstate); Py_XDECREF(setstate_cython); return ret; } /* Import */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level) { PyObject *empty_list = 0; PyObject *module = 0; PyObject *global_dict = 0; PyObject *empty_dict = 0; PyObject *list; #if PY_MAJOR_VERSION < 3 PyObject *py_import; py_import = __Pyx_PyObject_GetAttrStr(__pyx_b, __pyx_n_s_import); if (!py_import) goto bad; #endif if (from_list) list = from_list; else { empty_list = PyList_New(0); if (!empty_list) goto bad; list = empty_list; } global_dict = PyModule_GetDict(__pyx_m); if (!global_dict) goto bad; empty_dict = PyDict_New(); if (!empty_dict) goto bad; { #if PY_MAJOR_VERSION >= 3 if (level == -1) { if (strchr(__Pyx_MODULE_NAME, '.')) { module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, 1); if (!module) { if (!PyErr_ExceptionMatches(PyExc_ImportError)) goto bad; PyErr_Clear(); } } level = 0; } #endif if (!module) { #if PY_MAJOR_VERSION < 3 PyObject *py_level = PyInt_FromLong(level); if (!py_level) goto bad; module = PyObject_CallFunctionObjArgs(py_import, name, global_dict, empty_dict, list, py_level, (PyObject *)NULL); Py_DECREF(py_level); #else module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, level); #endif } } bad: #if PY_MAJOR_VERSION < 3 Py_XDECREF(py_import); #endif Py_XDECREF(empty_list); Py_XDECREF(empty_dict); return module; } /* ImportFrom */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name) { PyObject* value = __Pyx_PyObject_GetAttrStr(module, name); if (unlikely(!value) && PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Format(PyExc_ImportError, #if PY_MAJOR_VERSION < 3 "cannot import name %.230s", PyString_AS_STRING(name)); #else "cannot import name %S", name); #endif } return value; } /* FetchCommonType */ static PyTypeObject* __Pyx_FetchCommonType(PyTypeObject* type) { PyObject* fake_module; PyTypeObject* cached_type = NULL; fake_module = PyImport_AddModule((char*) "_cython_" CYTHON_ABI); if (!fake_module) return NULL; Py_INCREF(fake_module); cached_type = (PyTypeObject*) PyObject_GetAttrString(fake_module, type->tp_name); if (cached_type) { if (!PyType_Check((PyObject*)cached_type)) { PyErr_Format(PyExc_TypeError, "Shared Cython type %.200s is not a type object", type->tp_name); goto bad; } if (cached_type->tp_basicsize != type->tp_basicsize) { PyErr_Format(PyExc_TypeError, "Shared Cython type %.200s has the wrong size, try recompiling", type->tp_name); goto bad; } } else { if (!PyErr_ExceptionMatches(PyExc_AttributeError)) goto bad; PyErr_Clear(); if (PyType_Ready(type) < 0) goto bad; if (PyObject_SetAttrString(fake_module, type->tp_name, (PyObject*) type) < 0) goto bad; Py_INCREF(type); cached_type = type; } done: Py_DECREF(fake_module); return cached_type; bad: Py_XDECREF(cached_type); cached_type = NULL; goto done; } /* CythonFunction */ #include static PyObject * __Pyx_CyFunction_get_doc(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *closure) { if (unlikely(op->func_doc == NULL)) { if (op->func.m_ml->ml_doc) { #if PY_MAJOR_VERSION >= 3 op->func_doc = PyUnicode_FromString(op->func.m_ml->ml_doc); #else op->func_doc = PyString_FromString(op->func.m_ml->ml_doc); #endif if (unlikely(op->func_doc == NULL)) return NULL; } else { Py_INCREF(Py_None); return Py_None; } } Py_INCREF(op->func_doc); return op->func_doc; } static int __Pyx_CyFunction_set_doc(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp = op->func_doc; if (value == NULL) { value = Py_None; } Py_INCREF(value); op->func_doc = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_name(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { if (unlikely(op->func_name == NULL)) { #if PY_MAJOR_VERSION >= 3 op->func_name = PyUnicode_InternFromString(op->func.m_ml->ml_name); #else op->func_name = PyString_InternFromString(op->func.m_ml->ml_name); #endif if (unlikely(op->func_name == NULL)) return NULL; } Py_INCREF(op->func_name); return op->func_name; } static int __Pyx_CyFunction_set_name(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp; #if PY_MAJOR_VERSION >= 3 if (unlikely(value == NULL || !PyUnicode_Check(value))) #else if (unlikely(value == NULL || !PyString_Check(value))) #endif { PyErr_SetString(PyExc_TypeError, "__name__ must be set to a string object"); return -1; } tmp = op->func_name; Py_INCREF(value); op->func_name = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_qualname(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { Py_INCREF(op->func_qualname); return op->func_qualname; } static int __Pyx_CyFunction_set_qualname(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp; #if PY_MAJOR_VERSION >= 3 if (unlikely(value == NULL || !PyUnicode_Check(value))) #else if (unlikely(value == NULL || !PyString_Check(value))) #endif { PyErr_SetString(PyExc_TypeError, "__qualname__ must be set to a string object"); return -1; } tmp = op->func_qualname; Py_INCREF(value); op->func_qualname = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_self(__pyx_CyFunctionObject *m, CYTHON_UNUSED void *closure) { PyObject *self; self = m->func_closure; if (self == NULL) self = Py_None; Py_INCREF(self); return self; } static PyObject * __Pyx_CyFunction_get_dict(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { if (unlikely(op->func_dict == NULL)) { op->func_dict = PyDict_New(); if (unlikely(op->func_dict == NULL)) return NULL; } Py_INCREF(op->func_dict); return op->func_dict; } static int __Pyx_CyFunction_set_dict(__pyx_CyFunctionObject *op, PyObject *value, CYTHON_UNUSED void *context) { PyObject *tmp; if (unlikely(value == NULL)) { PyErr_SetString(PyExc_TypeError, "function's dictionary may not be deleted"); return -1; } if (unlikely(!PyDict_Check(value))) { PyErr_SetString(PyExc_TypeError, "setting function's dictionary to a non-dict"); return -1; } tmp = op->func_dict; Py_INCREF(value); op->func_dict = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_globals(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { Py_INCREF(op->func_globals); return op->func_globals; } static PyObject * __Pyx_CyFunction_get_closure(CYTHON_UNUSED __pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { Py_INCREF(Py_None); return Py_None; } static PyObject * __Pyx_CyFunction_get_code(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = (op->func_code) ? op->func_code : Py_None; Py_INCREF(result); return result; } static int __Pyx_CyFunction_init_defaults(__pyx_CyFunctionObject *op) { int result = 0; PyObject *res = op->defaults_getter((PyObject *) op); if (unlikely(!res)) return -1; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS op->defaults_tuple = PyTuple_GET_ITEM(res, 0); Py_INCREF(op->defaults_tuple); op->defaults_kwdict = PyTuple_GET_ITEM(res, 1); Py_INCREF(op->defaults_kwdict); #else op->defaults_tuple = PySequence_ITEM(res, 0); if (unlikely(!op->defaults_tuple)) result = -1; else { op->defaults_kwdict = PySequence_ITEM(res, 1); if (unlikely(!op->defaults_kwdict)) result = -1; } #endif Py_DECREF(res); return result; } static int __Pyx_CyFunction_set_defaults(__pyx_CyFunctionObject *op, PyObject* value, CYTHON_UNUSED void *context) { PyObject* tmp; if (!value) { value = Py_None; } else if (value != Py_None && !PyTuple_Check(value)) { PyErr_SetString(PyExc_TypeError, "__defaults__ must be set to a tuple object"); return -1; } Py_INCREF(value); tmp = op->defaults_tuple; op->defaults_tuple = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_defaults(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = op->defaults_tuple; if (unlikely(!result)) { if (op->defaults_getter) { if (__Pyx_CyFunction_init_defaults(op) < 0) return NULL; result = op->defaults_tuple; } else { result = Py_None; } } Py_INCREF(result); return result; } static int __Pyx_CyFunction_set_kwdefaults(__pyx_CyFunctionObject *op, PyObject* value, CYTHON_UNUSED void *context) { PyObject* tmp; if (!value) { value = Py_None; } else if (value != Py_None && !PyDict_Check(value)) { PyErr_SetString(PyExc_TypeError, "__kwdefaults__ must be set to a dict object"); return -1; } Py_INCREF(value); tmp = op->defaults_kwdict; op->defaults_kwdict = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_kwdefaults(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = op->defaults_kwdict; if (unlikely(!result)) { if (op->defaults_getter) { if (__Pyx_CyFunction_init_defaults(op) < 0) return NULL; result = op->defaults_kwdict; } else { result = Py_None; } } Py_INCREF(result); return result; } static int __Pyx_CyFunction_set_annotations(__pyx_CyFunctionObject *op, PyObject* value, CYTHON_UNUSED void *context) { PyObject* tmp; if (!value || value == Py_None) { value = NULL; } else if (!PyDict_Check(value)) { PyErr_SetString(PyExc_TypeError, "__annotations__ must be set to a dict object"); return -1; } Py_XINCREF(value); tmp = op->func_annotations; op->func_annotations = value; Py_XDECREF(tmp); return 0; } static PyObject * __Pyx_CyFunction_get_annotations(__pyx_CyFunctionObject *op, CYTHON_UNUSED void *context) { PyObject* result = op->func_annotations; if (unlikely(!result)) { result = PyDict_New(); if (unlikely(!result)) return NULL; op->func_annotations = result; } Py_INCREF(result); return result; } static PyGetSetDef __pyx_CyFunction_getsets[] = { {(char *) "func_doc", (getter)__Pyx_CyFunction_get_doc, (setter)__Pyx_CyFunction_set_doc, 0, 0}, {(char *) "__doc__", (getter)__Pyx_CyFunction_get_doc, (setter)__Pyx_CyFunction_set_doc, 0, 0}, {(char *) "func_name", (getter)__Pyx_CyFunction_get_name, (setter)__Pyx_CyFunction_set_name, 0, 0}, {(char *) "__name__", (getter)__Pyx_CyFunction_get_name, (setter)__Pyx_CyFunction_set_name, 0, 0}, {(char *) "__qualname__", (getter)__Pyx_CyFunction_get_qualname, (setter)__Pyx_CyFunction_set_qualname, 0, 0}, {(char *) "__self__", (getter)__Pyx_CyFunction_get_self, 0, 0, 0}, {(char *) "func_dict", (getter)__Pyx_CyFunction_get_dict, (setter)__Pyx_CyFunction_set_dict, 0, 0}, {(char *) "__dict__", (getter)__Pyx_CyFunction_get_dict, (setter)__Pyx_CyFunction_set_dict, 0, 0}, {(char *) "func_globals", (getter)__Pyx_CyFunction_get_globals, 0, 0, 0}, {(char *) "__globals__", (getter)__Pyx_CyFunction_get_globals, 0, 0, 0}, {(char *) "func_closure", (getter)__Pyx_CyFunction_get_closure, 0, 0, 0}, {(char *) "__closure__", (getter)__Pyx_CyFunction_get_closure, 0, 0, 0}, {(char *) "func_code", (getter)__Pyx_CyFunction_get_code, 0, 0, 0}, {(char *) "__code__", (getter)__Pyx_CyFunction_get_code, 0, 0, 0}, {(char *) "func_defaults", (getter)__Pyx_CyFunction_get_defaults, (setter)__Pyx_CyFunction_set_defaults, 0, 0}, {(char *) "__defaults__", (getter)__Pyx_CyFunction_get_defaults, (setter)__Pyx_CyFunction_set_defaults, 0, 0}, {(char *) "__kwdefaults__", (getter)__Pyx_CyFunction_get_kwdefaults, (setter)__Pyx_CyFunction_set_kwdefaults, 0, 0}, {(char *) "__annotations__", (getter)__Pyx_CyFunction_get_annotations, (setter)__Pyx_CyFunction_set_annotations, 0, 0}, {0, 0, 0, 0, 0} }; static PyMemberDef __pyx_CyFunction_members[] = { {(char *) "__module__", T_OBJECT, offsetof(PyCFunctionObject, m_module), PY_WRITE_RESTRICTED, 0}, {0, 0, 0, 0, 0} }; static PyObject * __Pyx_CyFunction_reduce(__pyx_CyFunctionObject *m, CYTHON_UNUSED PyObject *args) { #if PY_MAJOR_VERSION >= 3 return PyUnicode_FromString(m->func.m_ml->ml_name); #else return PyString_FromString(m->func.m_ml->ml_name); #endif } static PyMethodDef __pyx_CyFunction_methods[] = { {"__reduce__", (PyCFunction)__Pyx_CyFunction_reduce, METH_VARARGS, 0}, {0, 0, 0, 0} }; #if PY_VERSION_HEX < 0x030500A0 #define __Pyx_CyFunction_weakreflist(cyfunc) ((cyfunc)->func_weakreflist) #else #define __Pyx_CyFunction_weakreflist(cyfunc) ((cyfunc)->func.m_weakreflist) #endif static PyObject *__Pyx_CyFunction_New(PyTypeObject *type, PyMethodDef *ml, int flags, PyObject* qualname, PyObject *closure, PyObject *module, PyObject* globals, PyObject* code) { __pyx_CyFunctionObject *op = PyObject_GC_New(__pyx_CyFunctionObject, type); if (op == NULL) return NULL; op->flags = flags; __Pyx_CyFunction_weakreflist(op) = NULL; op->func.m_ml = ml; op->func.m_self = (PyObject *) op; Py_XINCREF(closure); op->func_closure = closure; Py_XINCREF(module); op->func.m_module = module; op->func_dict = NULL; op->func_name = NULL; Py_INCREF(qualname); op->func_qualname = qualname; op->func_doc = NULL; op->func_classobj = NULL; op->func_globals = globals; Py_INCREF(op->func_globals); Py_XINCREF(code); op->func_code = code; op->defaults_pyobjects = 0; op->defaults = NULL; op->defaults_tuple = NULL; op->defaults_kwdict = NULL; op->defaults_getter = NULL; op->func_annotations = NULL; PyObject_GC_Track(op); return (PyObject *) op; } static int __Pyx_CyFunction_clear(__pyx_CyFunctionObject *m) { Py_CLEAR(m->func_closure); Py_CLEAR(m->func.m_module); Py_CLEAR(m->func_dict); Py_CLEAR(m->func_name); Py_CLEAR(m->func_qualname); Py_CLEAR(m->func_doc); Py_CLEAR(m->func_globals); Py_CLEAR(m->func_code); Py_CLEAR(m->func_classobj); Py_CLEAR(m->defaults_tuple); Py_CLEAR(m->defaults_kwdict); Py_CLEAR(m->func_annotations); if (m->defaults) { PyObject **pydefaults = __Pyx_CyFunction_Defaults(PyObject *, m); int i; for (i = 0; i < m->defaults_pyobjects; i++) Py_XDECREF(pydefaults[i]); PyObject_Free(m->defaults); m->defaults = NULL; } return 0; } static void __Pyx__CyFunction_dealloc(__pyx_CyFunctionObject *m) { if (__Pyx_CyFunction_weakreflist(m) != NULL) PyObject_ClearWeakRefs((PyObject *) m); __Pyx_CyFunction_clear(m); PyObject_GC_Del(m); } static void __Pyx_CyFunction_dealloc(__pyx_CyFunctionObject *m) { PyObject_GC_UnTrack(m); __Pyx__CyFunction_dealloc(m); } static int __Pyx_CyFunction_traverse(__pyx_CyFunctionObject *m, visitproc visit, void *arg) { Py_VISIT(m->func_closure); Py_VISIT(m->func.m_module); Py_VISIT(m->func_dict); Py_VISIT(m->func_name); Py_VISIT(m->func_qualname); Py_VISIT(m->func_doc); Py_VISIT(m->func_globals); Py_VISIT(m->func_code); Py_VISIT(m->func_classobj); Py_VISIT(m->defaults_tuple); Py_VISIT(m->defaults_kwdict); if (m->defaults) { PyObject **pydefaults = __Pyx_CyFunction_Defaults(PyObject *, m); int i; for (i = 0; i < m->defaults_pyobjects; i++) Py_VISIT(pydefaults[i]); } return 0; } static PyObject *__Pyx_CyFunction_descr_get(PyObject *func, PyObject *obj, PyObject *type) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; if (m->flags & __Pyx_CYFUNCTION_STATICMETHOD) { Py_INCREF(func); return func; } if (m->flags & __Pyx_CYFUNCTION_CLASSMETHOD) { if (type == NULL) type = (PyObject *)(Py_TYPE(obj)); return __Pyx_PyMethod_New(func, type, (PyObject *)(Py_TYPE(type))); } if (obj == Py_None) obj = NULL; return __Pyx_PyMethod_New(func, obj, type); } static PyObject* __Pyx_CyFunction_repr(__pyx_CyFunctionObject *op) { #if PY_MAJOR_VERSION >= 3 return PyUnicode_FromFormat("", op->func_qualname, (void *)op); #else return PyString_FromFormat("", PyString_AsString(op->func_qualname), (void *)op); #endif } static PyObject * __Pyx_CyFunction_CallMethod(PyObject *func, PyObject *self, PyObject *arg, PyObject *kw) { PyCFunctionObject* f = (PyCFunctionObject*)func; PyCFunction meth = f->m_ml->ml_meth; Py_ssize_t size; switch (f->m_ml->ml_flags & (METH_VARARGS | METH_KEYWORDS | METH_NOARGS | METH_O)) { case METH_VARARGS: if (likely(kw == NULL || PyDict_Size(kw) == 0)) return (*meth)(self, arg); break; case METH_VARARGS | METH_KEYWORDS: return (*(PyCFunctionWithKeywords)(void*)meth)(self, arg, kw); case METH_NOARGS: if (likely(kw == NULL || PyDict_Size(kw) == 0)) { size = PyTuple_GET_SIZE(arg); if (likely(size == 0)) return (*meth)(self, NULL); PyErr_Format(PyExc_TypeError, "%.200s() takes no arguments (%" CYTHON_FORMAT_SSIZE_T "d given)", f->m_ml->ml_name, size); return NULL; } break; case METH_O: if (likely(kw == NULL || PyDict_Size(kw) == 0)) { size = PyTuple_GET_SIZE(arg); if (likely(size == 1)) { PyObject *result, *arg0; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS arg0 = PyTuple_GET_ITEM(arg, 0); #else arg0 = PySequence_ITEM(arg, 0); if (unlikely(!arg0)) return NULL; #endif result = (*meth)(self, arg0); #if !(CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS) Py_DECREF(arg0); #endif return result; } PyErr_Format(PyExc_TypeError, "%.200s() takes exactly one argument (%" CYTHON_FORMAT_SSIZE_T "d given)", f->m_ml->ml_name, size); return NULL; } break; default: PyErr_SetString(PyExc_SystemError, "Bad call flags in " "__Pyx_CyFunction_Call. METH_OLDARGS is no " "longer supported!"); return NULL; } PyErr_Format(PyExc_TypeError, "%.200s() takes no keyword arguments", f->m_ml->ml_name); return NULL; } static CYTHON_INLINE PyObject *__Pyx_CyFunction_Call(PyObject *func, PyObject *arg, PyObject *kw) { return __Pyx_CyFunction_CallMethod(func, ((PyCFunctionObject*)func)->m_self, arg, kw); } static PyObject *__Pyx_CyFunction_CallAsMethod(PyObject *func, PyObject *args, PyObject *kw) { PyObject *result; __pyx_CyFunctionObject *cyfunc = (__pyx_CyFunctionObject *) func; if ((cyfunc->flags & __Pyx_CYFUNCTION_CCLASS) && !(cyfunc->flags & __Pyx_CYFUNCTION_STATICMETHOD)) { Py_ssize_t argc; PyObject *new_args; PyObject *self; argc = PyTuple_GET_SIZE(args); new_args = PyTuple_GetSlice(args, 1, argc); if (unlikely(!new_args)) return NULL; self = PyTuple_GetItem(args, 0); if (unlikely(!self)) { Py_DECREF(new_args); return NULL; } result = __Pyx_CyFunction_CallMethod(func, self, new_args, kw); Py_DECREF(new_args); } else { result = __Pyx_CyFunction_Call(func, args, kw); } return result; } static PyTypeObject __pyx_CyFunctionType_type = { PyVarObject_HEAD_INIT(0, 0) "cython_function_or_method", sizeof(__pyx_CyFunctionObject), 0, (destructor) __Pyx_CyFunction_dealloc, 0, 0, 0, #if PY_MAJOR_VERSION < 3 0, #else 0, #endif (reprfunc) __Pyx_CyFunction_repr, 0, 0, 0, 0, __Pyx_CyFunction_CallAsMethod, 0, 0, 0, 0, Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, 0, (traverseproc) __Pyx_CyFunction_traverse, (inquiry) __Pyx_CyFunction_clear, 0, #if PY_VERSION_HEX < 0x030500A0 offsetof(__pyx_CyFunctionObject, func_weakreflist), #else offsetof(PyCFunctionObject, m_weakreflist), #endif 0, 0, __pyx_CyFunction_methods, __pyx_CyFunction_members, __pyx_CyFunction_getsets, 0, 0, __Pyx_CyFunction_descr_get, 0, offsetof(__pyx_CyFunctionObject, func_dict), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, #if PY_VERSION_HEX >= 0x030400a1 0, #endif }; static int __pyx_CyFunction_init(void) { __pyx_CyFunctionType = __Pyx_FetchCommonType(&__pyx_CyFunctionType_type); if (unlikely(__pyx_CyFunctionType == NULL)) { return -1; } return 0; } static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->defaults = PyObject_Malloc(size); if (unlikely(!m->defaults)) return PyErr_NoMemory(); memset(m->defaults, 0, size); m->defaults_pyobjects = pyobjects; return m->defaults; } static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->defaults_tuple = tuple; Py_INCREF(tuple); } static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->defaults_kwdict = dict; Py_INCREF(dict); } static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) { __pyx_CyFunctionObject *m = (__pyx_CyFunctionObject *) func; m->func_annotations = dict; Py_INCREF(dict); } /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_PY_LONG_LONG(PY_LONG_LONG value) { const PY_LONG_LONG neg_one = (PY_LONG_LONG) ((PY_LONG_LONG) 0 - (PY_LONG_LONG) 1), const_zero = (PY_LONG_LONG) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(PY_LONG_LONG) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(PY_LONG_LONG) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(PY_LONG_LONG) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(PY_LONG_LONG) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(PY_LONG_LONG) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(PY_LONG_LONG), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(int) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(int) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(int) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(int), little, !is_unsigned); } } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* CIntFromPy */ static CYTHON_INLINE unsigned char __Pyx_PyInt_As_unsigned_char(PyObject *x) { const unsigned char neg_one = (unsigned char) ((unsigned char) 0 - (unsigned char) 1), const_zero = (unsigned char) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(unsigned char) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(unsigned char, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (unsigned char) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (unsigned char) 0; case 1: __PYX_VERIFY_RETURN_INT(unsigned char, digit, digits[0]) case 2: if (8 * sizeof(unsigned char) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) >= 2 * PyLong_SHIFT) { return (unsigned char) (((((unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0])); } } break; case 3: if (8 * sizeof(unsigned char) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) >= 3 * PyLong_SHIFT) { return (unsigned char) (((((((unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0])); } } break; case 4: if (8 * sizeof(unsigned char) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) >= 4 * PyLong_SHIFT) { return (unsigned char) (((((((((unsigned char)digits[3]) << PyLong_SHIFT) | (unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (unsigned char) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(unsigned char) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned char) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (unsigned char) 0; case -1: __PYX_VERIFY_RETURN_INT(unsigned char, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(unsigned char, digit, +digits[0]) case -2: if (8 * sizeof(unsigned char) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 2 * PyLong_SHIFT) { return (unsigned char) (((unsigned char)-1)*(((((unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case 2: if (8 * sizeof(unsigned char) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 2 * PyLong_SHIFT) { return (unsigned char) ((((((unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case -3: if (8 * sizeof(unsigned char) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 3 * PyLong_SHIFT) { return (unsigned char) (((unsigned char)-1)*(((((((unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case 3: if (8 * sizeof(unsigned char) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 3 * PyLong_SHIFT) { return (unsigned char) ((((((((unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case -4: if (8 * sizeof(unsigned char) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 4 * PyLong_SHIFT) { return (unsigned char) (((unsigned char)-1)*(((((((((unsigned char)digits[3]) << PyLong_SHIFT) | (unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; case 4: if (8 * sizeof(unsigned char) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(unsigned char, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(unsigned char) - 1 > 4 * PyLong_SHIFT) { return (unsigned char) ((((((((((unsigned char)digits[3]) << PyLong_SHIFT) | (unsigned char)digits[2]) << PyLong_SHIFT) | (unsigned char)digits[1]) << PyLong_SHIFT) | (unsigned char)digits[0]))); } } break; } #endif if (sizeof(unsigned char) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(unsigned char) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(unsigned char, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else unsigned char val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (unsigned char) -1; } } else { unsigned char val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (unsigned char) -1; val = __Pyx_PyInt_As_unsigned_char(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to unsigned char"); return (unsigned char) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to unsigned char"); return (unsigned char) -1; } /* CIntFromPy */ static CYTHON_INLINE PY_LONG_LONG __Pyx_PyInt_As_PY_LONG_LONG(PyObject *x) { const PY_LONG_LONG neg_one = (PY_LONG_LONG) ((PY_LONG_LONG) 0 - (PY_LONG_LONG) 1), const_zero = (PY_LONG_LONG) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(PY_LONG_LONG) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (PY_LONG_LONG) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (PY_LONG_LONG) 0; case 1: __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, digit, digits[0]) case 2: if (8 * sizeof(PY_LONG_LONG) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) >= 2 * PyLong_SHIFT) { return (PY_LONG_LONG) (((((PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0])); } } break; case 3: if (8 * sizeof(PY_LONG_LONG) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) >= 3 * PyLong_SHIFT) { return (PY_LONG_LONG) (((((((PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0])); } } break; case 4: if (8 * sizeof(PY_LONG_LONG) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) >= 4 * PyLong_SHIFT) { return (PY_LONG_LONG) (((((((((PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (PY_LONG_LONG) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(PY_LONG_LONG) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(PY_LONG_LONG, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(PY_LONG_LONG) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(PY_LONG_LONG, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (PY_LONG_LONG) 0; case -1: __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, digit, +digits[0]) case -2: if (8 * sizeof(PY_LONG_LONG) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { return (PY_LONG_LONG) (((PY_LONG_LONG)-1)*(((((PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0]))); } } break; case 2: if (8 * sizeof(PY_LONG_LONG) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { return (PY_LONG_LONG) ((((((PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0]))); } } break; case -3: if (8 * sizeof(PY_LONG_LONG) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { return (PY_LONG_LONG) (((PY_LONG_LONG)-1)*(((((((PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0]))); } } break; case 3: if (8 * sizeof(PY_LONG_LONG) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { return (PY_LONG_LONG) ((((((((PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0]))); } } break; case -4: if (8 * sizeof(PY_LONG_LONG) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { return (PY_LONG_LONG) (((PY_LONG_LONG)-1)*(((((((((PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0]))); } } break; case 4: if (8 * sizeof(PY_LONG_LONG) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(PY_LONG_LONG, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(PY_LONG_LONG) - 1 > 4 * PyLong_SHIFT) { return (PY_LONG_LONG) ((((((((((PY_LONG_LONG)digits[3]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[2]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[1]) << PyLong_SHIFT) | (PY_LONG_LONG)digits[0]))); } } break; } #endif if (sizeof(PY_LONG_LONG) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(PY_LONG_LONG, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(PY_LONG_LONG) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(PY_LONG_LONG, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else PY_LONG_LONG val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (PY_LONG_LONG) -1; } } else { PY_LONG_LONG val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (PY_LONG_LONG) -1; val = __Pyx_PyInt_As_PY_LONG_LONG(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to PY_LONG_LONG"); return (PY_LONG_LONG) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to PY_LONG_LONG"); return (PY_LONG_LONG) -1; } /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; ip) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/crypto.pyx0000644000175000017500000001555513431513133016714 0ustar useruser00000000000000"""A thin OpenSSL wrapper This could be replaced by PyCrypto maybe? """ import hashlib import hmac from math import ceil from libc.stdlib cimport malloc, free API_VERSION = '1.0_01' cdef extern from "openssl/rand.h": int RAND_bytes(unsigned char *buf, int num) cdef extern from "openssl/evp.h": ctypedef struct EVP_MD: pass ctypedef struct EVP_CIPHER: pass ctypedef struct EVP_CIPHER_CTX: pass ctypedef struct ENGINE: pass const EVP_CIPHER *EVP_aes_256_ctr() EVP_CIPHER_CTX *EVP_CIPHER_CTX_new() void EVP_CIPHER_CTX_free(EVP_CIPHER_CTX *a) int EVP_EncryptInit_ex(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *cipher, ENGINE *impl, const unsigned char *key, const unsigned char *iv) int EVP_DecryptInit_ex(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *cipher, ENGINE *impl, const unsigned char *key, const unsigned char *iv) int EVP_EncryptUpdate(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl, const unsigned char *in_, int inl) int EVP_DecryptUpdate(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl, const unsigned char *in_, int inl) int EVP_EncryptFinal_ex(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl) int EVP_DecryptFinal_ex(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl) import struct _int = struct.Struct('>I') _long = struct.Struct('>Q') _2long = struct.Struct('>QQ') bytes_to_int = lambda x, offset=0: _int.unpack_from(x, offset)[0] bytes_to_long = lambda x, offset=0: _long.unpack_from(x, offset)[0] long_to_bytes = lambda x: _long.pack(x) def bytes16_to_int(b, offset=0): h, l = _2long.unpack_from(b, offset) return (h << 64) + l def int_to_bytes16(i): max_uint64 = 0xffffffffffffffff l = i & max_uint64 h = (i >> 64) & max_uint64 return _2long.pack(h, l) def increment_iv(iv, amount=1): """ Increment the IV by the given amount (default 1). :param iv: input IV, 16 bytes (128 bit) :param amount: increment value :return: input_IV + amount, 16 bytes (128 bit) """ assert len(iv) == 16 iv = bytes16_to_int(iv) iv += amount iv = int_to_bytes16(iv) return iv def num_aes_blocks(int length): """Return the number of AES blocks required to encrypt/decrypt *length* bytes of data. Note: this is only correct for modes without padding, like AES-CTR. """ return (length + 15) // 16 cdef class AES: """A thin wrapper around the OpenSSL EVP cipher API """ cdef EVP_CIPHER_CTX *ctx cdef int is_encrypt cdef unsigned char iv_orig[16] cdef long long blocks def __cinit__(self, is_encrypt, key, iv=None): self.ctx = EVP_CIPHER_CTX_new() self.is_encrypt = is_encrypt # Set cipher type and mode cipher_mode = EVP_aes_256_ctr() if self.is_encrypt: if not EVP_EncryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): raise Exception('EVP_EncryptInit_ex failed') else: # decrypt if not EVP_DecryptInit_ex(self.ctx, cipher_mode, NULL, NULL, NULL): raise Exception('EVP_DecryptInit_ex failed') self.reset(key, iv) def __dealloc__(self): EVP_CIPHER_CTX_free(self.ctx) def reset(self, key=None, iv=None): cdef const unsigned char *key2 = NULL cdef const unsigned char *iv2 = NULL if key: key2 = key if iv: iv2 = iv assert isinstance(iv, bytes) and len(iv) == 16 for i in range(16): self.iv_orig[i] = iv[i] self.blocks = 0 # number of AES blocks encrypted starting with iv_orig # Initialise key and IV if self.is_encrypt: if not EVP_EncryptInit_ex(self.ctx, NULL, NULL, key2, iv2): raise Exception('EVP_EncryptInit_ex failed') else: # decrypt if not EVP_DecryptInit_ex(self.ctx, NULL, NULL, key2, iv2): raise Exception('EVP_DecryptInit_ex failed') @property def iv(self): return increment_iv(self.iv_orig[:16], self.blocks) def encrypt(self, data): cdef int inl = len(data) cdef int ctl = 0 cdef int outl = 0 # note: modes that use padding, need up to one extra AES block (16b) cdef unsigned char *out = malloc(inl+16) if not out: raise MemoryError try: if not EVP_EncryptUpdate(self.ctx, out, &outl, data, inl): raise Exception('EVP_EncryptUpdate failed') ctl = outl if not EVP_EncryptFinal_ex(self.ctx, out+ctl, &outl): raise Exception('EVP_EncryptFinal failed') ctl += outl self.blocks += num_aes_blocks(ctl) return out[:ctl] finally: free(out) def decrypt(self, data): cdef int inl = len(data) cdef int ptl = 0 cdef int outl = 0 # note: modes that use padding, need up to one extra AES block (16b). # This is what the openssl docs say. I am not sure this is correct, # but OTOH it will not cause any harm if our buffer is a little bigger. cdef unsigned char *out = malloc(inl+16) if not out: raise MemoryError try: if not EVP_DecryptUpdate(self.ctx, out, &outl, data, inl): raise Exception('EVP_DecryptUpdate failed') ptl = outl if EVP_DecryptFinal_ex(self.ctx, out+ptl, &outl) <= 0: # this error check is very important for modes with padding or # authentication. for them, a failure here means corrupted data. # CTR mode does not use padding nor authentication. raise Exception('EVP_DecryptFinal failed') ptl += outl self.blocks += num_aes_blocks(inl) return out[:ptl] finally: free(out) def hkdf_hmac_sha512(ikm, salt, info, output_length): """ Compute HKDF-HMAC-SHA512 with input key material *ikm*, *salt* and *info* to produce *output_length* bytes. This is the "HMAC-based Extract-and-Expand Key Derivation Function (HKDF)" (RFC 5869) instantiated with HMAC-SHA512. *output_length* must not be greater than 64 * 255 bytes. """ digest_length = 64 assert output_length <= (255 * digest_length), 'output_length must be <= 255 * 64 bytes' # Step 1. HKDF-Extract (ikm, salt) -> prk if salt is None: salt = bytes(64) prk = hmac.HMAC(salt, ikm, hashlib.sha512).digest() # Step 2. HKDF-Expand (prk, info, output_length) -> output key n = ceil(output_length / digest_length) t_n = b'' output = b'' for i in range(n): msg = t_n + info + (i + 1).to_bytes(1, 'little') t_n = hmac.HMAC(prk, msg, hashlib.sha512).digest() output += t_n return output[:output_length] borgbackup-1.0.13/borg/fuse.py0000644000175000017500000003261113431513133016136 0ustar useruser00000000000000from collections import defaultdict import errno import io import llfuse import os import stat import tempfile import time from signal import SIGINT from distutils.version import LooseVersion import msgpack from .archive import Archive from .helpers import daemonize, bigint_to_int from .logger import create_logger from .lrucache import LRUCache from .remote import RemoteRepository logger = create_logger() # Does this version of llfuse support ns precision? have_fuse_xtime_ns = hasattr(llfuse.EntryAttributes, 'st_mtime_ns') fuse_version = LooseVersion(getattr(llfuse, '__version__', '0.1')) if fuse_version >= '0.42': def fuse_main(): return llfuse.main(workers=1) else: def fuse_main(): llfuse.main(single=True) return None class ItemCache: def __init__(self): self.fd = tempfile.TemporaryFile(prefix='borg-tmp') self.offset = 1000000 def add(self, item): pos = self.fd.seek(0, io.SEEK_END) self.fd.write(msgpack.packb(item)) return pos + self.offset def get(self, inode): self.fd.seek(inode - self.offset, io.SEEK_SET) return next(msgpack.Unpacker(self.fd, read_size=1024)) class FuseOperations(llfuse.Operations): """Export archive as a fuse filesystem """ allow_damaged_files = False def __init__(self, key, repository, manifest, archive, cached_repo): super().__init__() self._inode_count = 0 self.key = key self.repository_uncached = repository self.repository = cached_repo self.items = {} self.parent = {} self.contents = defaultdict(dict) self.default_uid = os.getuid() self.default_gid = os.getgid() self.default_dir = {b'mode': 0o40755, b'mtime': int(time.time() * 1e9), b'uid': self.default_uid, b'gid': self.default_gid} self.pending_archives = {} self.accounted_chunks = {} self.cache = ItemCache() data_cache_capacity = int(os.environ.get('BORG_MOUNT_DATA_CACHE_ENTRIES', os.cpu_count() or 1)) logger.debug('mount data cache capacity: %d chunks', data_cache_capacity) self.data_cache = LRUCache(capacity=data_cache_capacity, dispose=lambda _: None) self._create_dir(parent=1) # first call, create root dir (inode == 1) if archive: self.process_archive(archive) else: for archive_name in manifest.archives: # Create archive placeholder inode archive_inode = self._create_dir(parent=1) self.contents[1][os.fsencode(archive_name)] = archive_inode self.pending_archives[archive_inode] = Archive(repository, key, manifest, archive_name) def mount(self, mountpoint, mount_options, foreground=False): """Mount filesystem on *mountpoint* with *mount_options*.""" # default_permissions enables permission checking by the kernel. Without # this, any umask (or uid/gid) would not have an effect and this could # cause security issues if used with allow_other mount option. # When not using allow_other or allow_root, access is limited to the # mounting user anyway. options = ['fsname=borgfs', 'ro', 'default_permissions'] if mount_options: options.extend(mount_options.split(',')) try: options.remove('allow_damaged_files') self.allow_damaged_files = True except ValueError: pass try: options.remove('ignore_permissions') # if above does not raise ValueError (meaning: ignore_permissions is present), # we remove default_permissions again. # in case users have a use-case that requires NOT giving "default_permissions", # this is enabled by the custom "ignore_permissions" mount option which just # removes "default_permissions" again: options.remove('default_permissions') except ValueError: pass llfuse.init(self, mountpoint, options) if not foreground: old_id, new_id = daemonize() if not isinstance(self.repository_uncached, RemoteRepository): # local repo and the locking process' PID just changed, migrate it: self.repository_uncached.migrate_lock(old_id, new_id) # If the file system crashes, we do not want to umount because in that # case the mountpoint suddenly appears to become empty. This can have # nasty consequences, imagine the user has e.g. an active rsync mirror # job - seeing the mountpoint empty, rsync would delete everything in the # mirror. umount = False try: signal = fuse_main() # no crash and no signal (or it's ^C and we're in the foreground) -> umount request umount = (signal is None or (signal == SIGINT and foreground)) finally: llfuse.close(umount) def _create_dir(self, parent): """Create directory """ ino = self.allocate_inode() self.items[ino] = self.default_dir self.parent[ino] = parent return ino def process_archive(self, archive, prefix=[]): """Build fuse inode hierarchy from archive metadata """ unpacker = msgpack.Unpacker() for key, chunk in zip(archive.metadata[b'items'], self.repository.get_many(archive.metadata[b'items'])): data = self.key.decrypt(key, chunk) unpacker.feed(data) for item in unpacker: try: # This can happen if an archive was created with a command line like # $ borg create ... dir1/file dir1 # In this case the code below will have created a default_dir inode for dir1 already. inode = self._find_inode(item[b'path'], prefix) except KeyError: pass else: self.items[inode] = item continue path = item.pop(b'path') segments = prefix + os.fsencode(os.path.normpath(path)).split(b'/') num_segments = len(segments) parent = 1 for i, segment in enumerate(segments, 1): # Leaf segment? if i == num_segments: if b'source' in item and stat.S_ISREG(item[b'mode']): try: inode = self._find_inode(item[b'source'], prefix) except KeyError: file = path.decode(errors='surrogateescape') source = item[b'source'].decode(errors='surrogateescape') logger.warning('Skipping broken hard link: %s -> %s', file, source) continue item = self.cache.get(inode) item[b'nlink'] = item.get(b'nlink', 1) + 1 self.items[inode] = item else: inode = self.cache.add(item) self.parent[inode] = parent if segment: self.contents[parent][segment] = inode elif segment in self.contents[parent]: parent = self.contents[parent][segment] else: inode = self._create_dir(parent) if segment: self.contents[parent][segment] = inode parent = inode def allocate_inode(self): self._inode_count += 1 return self._inode_count def statfs(self, ctx=None): stat_ = llfuse.StatvfsData() stat_.f_bsize = 512 stat_.f_frsize = 512 stat_.f_blocks = 0 stat_.f_bfree = 0 stat_.f_bavail = 0 stat_.f_files = 0 stat_.f_ffree = 0 stat_.f_favail = 0 return stat_ def get_item(self, inode): try: return self.items[inode] except KeyError: return self.cache.get(inode) def _find_inode(self, path, prefix=[]): segments = prefix + os.fsencode(os.path.normpath(path)).split(b'/') inode = 1 for segment in segments: inode = self.contents[inode][segment] return inode def getattr(self, inode, ctx=None): item = self.get_item(inode) size = 0 dsize = 0 try: for key, chunksize, _ in item[b'chunks']: size += chunksize if self.accounted_chunks.get(key, inode) == inode: self.accounted_chunks[key] = inode dsize += chunksize except KeyError: pass entry = llfuse.EntryAttributes() entry.st_ino = inode entry.generation = 0 entry.entry_timeout = 300 entry.attr_timeout = 300 entry.st_mode = item[b'mode'] entry.st_nlink = item.get(b'nlink', 1) entry.st_uid = item[b'uid'] if item[b'uid'] >= 0 else self.default_uid entry.st_gid = item[b'gid'] if item[b'gid'] >= 0 else self.default_gid entry.st_rdev = item.get(b'rdev', 0) entry.st_size = size entry.st_blksize = 512 entry.st_blocks = (dsize + entry.st_blksize - 1) // entry.st_blksize # note: older archives only have mtime (not atime nor ctime) if have_fuse_xtime_ns: entry.st_mtime_ns = bigint_to_int(item[b'mtime']) if b'atime' in item: entry.st_atime_ns = bigint_to_int(item[b'atime']) else: entry.st_atime_ns = bigint_to_int(item[b'mtime']) if b'ctime' in item: entry.st_ctime_ns = bigint_to_int(item[b'ctime']) else: entry.st_ctime_ns = bigint_to_int(item[b'mtime']) else: entry.st_mtime = bigint_to_int(item[b'mtime']) / 1e9 if b'atime' in item: entry.st_atime = bigint_to_int(item[b'atime']) / 1e9 else: entry.st_atime = bigint_to_int(item[b'mtime']) / 1e9 if b'ctime' in item: entry.st_ctime = bigint_to_int(item[b'ctime']) / 1e9 else: entry.st_ctime = bigint_to_int(item[b'mtime']) / 1e9 return entry def listxattr(self, inode, ctx=None): item = self.get_item(inode) return item.get(b'xattrs', {}).keys() def getxattr(self, inode, name, ctx=None): item = self.get_item(inode) try: return item.get(b'xattrs', {})[name] or b'' except KeyError: raise llfuse.FUSEError(llfuse.ENOATTR) from None def _load_pending_archive(self, inode): # Check if this is an archive we need to load archive = self.pending_archives.pop(inode, None) if archive: self.process_archive(archive, [os.fsencode(archive.name)]) def lookup(self, parent_inode, name, ctx=None): self._load_pending_archive(parent_inode) if name == b'.': inode = parent_inode elif name == b'..': inode = self.parent[parent_inode] else: inode = self.contents[parent_inode].get(name) if not inode: raise llfuse.FUSEError(errno.ENOENT) return self.getattr(inode) def open(self, inode, flags, ctx=None): if not self.allow_damaged_files: item = self.get_item(inode) if b'chunks_healthy' in item: # Processed archive items don't carry the path anymore; for converting the inode # to the path we'd either have to store the inverse of the current structure, # or search the entire archive. So we just don't print it. It's easy to correlate anyway. logger.warning('File has damaged (all-zero) chunks. Try running borg check --repair. ' 'Mount with allow_damaged_files to read damaged files.') raise llfuse.FUSEError(errno.EIO) return inode def opendir(self, inode, ctx=None): self._load_pending_archive(inode) return inode def read(self, fh, offset, size): parts = [] item = self.get_item(fh) for id, s, csize in item[b'chunks']: if s < offset: offset -= s continue n = min(size, s - offset) if id in self.data_cache: data = self.data_cache[id] if offset + n == len(data): # evict fully read chunk from cache del self.data_cache[id] else: data = self.key.decrypt(id, self.repository.get(id)) if offset + n < len(data): # chunk was only partially read, cache it self.data_cache[id] = data parts.append(data[offset:offset + n]) offset = 0 size -= n if not size: break return b''.join(parts) def readdir(self, fh, off): entries = [(b'.', fh), (b'..', self.parent[fh])] entries.extend(self.contents[fh].items()) for i, (name, inode) in enumerate(entries[off:], off): yield name, self.getattr(inode), i + 1 def readlink(self, inode, ctx=None): item = self.get_item(inode) return os.fsencode(item[b'source']) borgbackup-1.0.13/borg/hashindex.c0000644000175000017500000145114013431514213016744 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__hashindex #define __PYX_HAVE_API__borg__hashindex /* Early includes */ #include #include "_hashindex.c" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "borg/hashindex.pyx", "stringsource", }; /*--- Type declarations ---*/ struct __pyx_obj_4borg_9hashindex_IndexBase; struct __pyx_obj_4borg_9hashindex_NSIndex; struct __pyx_obj_4borg_9hashindex_NSKeyIterator; struct __pyx_obj_4borg_9hashindex_ChunkIndex; struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator; /* "borg/hashindex.pyx":54 * * @cython.internal * cdef class IndexBase: # <<<<<<<<<<<<<< * cdef HashIndex *index * cdef int key_size */ struct __pyx_obj_4borg_9hashindex_IndexBase { PyObject_HEAD HashIndex *index; int key_size; }; /* "borg/hashindex.pyx":121 * * * cdef class NSIndex(IndexBase): # <<<<<<<<<<<<<< * * value_size = 8 */ struct __pyx_obj_4borg_9hashindex_NSIndex { struct __pyx_obj_4borg_9hashindex_IndexBase __pyx_base; }; /* "borg/hashindex.pyx":166 * * * cdef class NSKeyIterator: # <<<<<<<<<<<<<< * cdef NSIndex idx * cdef HashIndex *index */ struct __pyx_obj_4borg_9hashindex_NSKeyIterator { PyObject_HEAD struct __pyx_obj_4borg_9hashindex_NSIndex *idx; HashIndex *index; void const *key; int key_size; int exhausted; }; /* "borg/hashindex.pyx":194 * * * cdef class ChunkIndex(IndexBase): # <<<<<<<<<<<<<< * """ * Mapping of 32 byte keys to (refcount, size, csize), which are all 32-bit unsigned. */ struct __pyx_obj_4borg_9hashindex_ChunkIndex { struct __pyx_obj_4borg_9hashindex_IndexBase __pyx_base; struct __pyx_vtabstruct_4borg_9hashindex_ChunkIndex *__pyx_vtab; }; /* "borg/hashindex.pyx":333 * * * cdef class ChunkKeyIterator: # <<<<<<<<<<<<<< * cdef ChunkIndex idx * cdef HashIndex *index */ struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator { PyObject_HEAD struct __pyx_obj_4borg_9hashindex_ChunkIndex *idx; HashIndex *index; void const *key; int key_size; int exhausted; }; /* "borg/hashindex.pyx":194 * * * cdef class ChunkIndex(IndexBase): # <<<<<<<<<<<<<< * """ * Mapping of 32 byte keys to (refcount, size, csize), which are all 32-bit unsigned. */ struct __pyx_vtabstruct_4borg_9hashindex_ChunkIndex { PyObject *(*_add)(struct __pyx_obj_4borg_9hashindex_ChunkIndex *, void *, uint32_t *); }; static struct __pyx_vtabstruct_4borg_9hashindex_ChunkIndex *__pyx_vtabptr_4borg_9hashindex_ChunkIndex; /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* GetModuleGlobalName.proto */ #if CYTHON_USE_DICT_VERSIONS #define __Pyx_GetModuleGlobalName(var, name) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } #define __Pyx_GetModuleGlobalNameUncached(var, name) {\ PY_UINT64_T __pyx_dict_version;\ PyObject *__pyx_dict_cached_value;\ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value); #else #define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name) #define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name) static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name); #endif /* PyCFunctionFastCall.proto */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs); #else #define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL) #endif /* PyFunctionFastCall.proto */ #if CYTHON_FAST_PYCALL #define __Pyx_PyFunction_FastCall(func, args, nargs)\ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL) #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #else #define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs) #endif #define __Pyx_BUILD_ASSERT_EXPR(cond)\ (sizeof(char [1 - 2*!(cond)]) - 1) #ifndef Py_MEMBER_SIZE #define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) #endif static size_t __pyx_pyframe_localsplus_offset = 0; #include "frameobject.h" #define __Pxy_PyFrame_Initialize_Offsets()\ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\ (void)(__pyx_pyframe_localsplus_offset = PyFrame_Type.tp_basicsize - Py_MEMBER_SIZE(PyFrameObject, f_localsplus))) #define __Pyx_PyFrame_GetLocalsplus(frame)\ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset)) #endif /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* PyObjectCall2Args.proto */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2); /* PyObjectCallMethO.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg); #endif /* PyObjectCallOneArg.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg); /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* RaiseException.proto */ static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause); /* PySequenceContains.proto */ static CYTHON_INLINE int __Pyx_PySequence_ContainsTF(PyObject* item, PyObject* seq, int eq) { int result = PySequence_Contains(seq, item); return unlikely(result < 0) ? result : (result == (eq == Py_EQ)); } /* GetItemInt.proto */ #define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\ __Pyx_GetItemInt_Generic(o, to_py_func(i)))) #define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); #define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j); static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, int wraparound, int boundscheck); /* ObjectGetItem.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key); #else #define __Pyx_PyObject_GetItem(obj, key) PyObject_GetItem(obj, key) #endif /* GetTopmostException.proto */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate); #endif /* SaveResetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSave(type, value, tb) __Pyx__ExceptionSave(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #define __Pyx_ExceptionReset(type, value, tb) __Pyx__ExceptionReset(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); #else #define __Pyx_ExceptionSave(type, value, tb) PyErr_GetExcInfo(type, value, tb) #define __Pyx_ExceptionReset(type, value, tb) PyErr_SetExcInfo(type, value, tb) #endif /* PyErrExceptionMatches.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_ExceptionMatches(err) __Pyx_PyErr_ExceptionMatchesInState(__pyx_tstate, err) static CYTHON_INLINE int __Pyx_PyErr_ExceptionMatchesInState(PyThreadState* tstate, PyObject* err); #else #define __Pyx_PyErr_ExceptionMatches(err) PyErr_ExceptionMatches(err) #endif /* GetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_GetException(type, value, tb) __Pyx__GetException(__pyx_tstate, type, value, tb) static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb); #endif /* ArgTypeTest.proto */ #define __Pyx_ArgTypeTest(obj, type, none_allowed, name, exact)\ ((likely((Py_TYPE(obj) == type) | (none_allowed && (obj == Py_None)))) ? 1 :\ __Pyx__ArgTypeTest(obj, type, name, exact)) static int __Pyx__ArgTypeTest(PyObject *obj, PyTypeObject *type, const char *name, int exact); /* PyObject_GenericGetAttrNoDict.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttrNoDict PyObject_GenericGetAttr #endif /* PyObject_GenericGetAttr.proto */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GenericGetAttr PyObject_GenericGetAttr #endif /* SetupReduce.proto */ static int __Pyx_setup_reduce(PyObject* type_obj); /* SetVTable.proto */ static int __Pyx_SetVtable(PyObject *dict, void *vtable); /* Import.proto */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level); /* PyObjectCallNoArg.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallNoArg(PyObject *func); #else #define __Pyx_PyObject_CallNoArg(func) __Pyx_PyObject_Call(func, __pyx_empty_tuple, NULL) #endif /* None.proto */ static CYTHON_INLINE long __Pyx_mod_long(long, long); /* ClassMethod.proto */ #include "descrobject.h" static PyObject* __Pyx_Method_ClassMethod(PyObject *method); /* GetNameInClass.proto */ #define __Pyx_GetNameInClass(var, nmspace, name) (var) = __Pyx__GetNameInClass(nmspace, name) static PyObject *__Pyx__GetNameInClass(PyObject *nmspace, PyObject *name); /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_uint32_t(uint32_t value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_uint64_t(uint64_t value); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE uint32_t __Pyx_PyInt_As_uint32_t(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); static PyObject *__pyx_f_4borg_9hashindex_10ChunkIndex__add(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, void *__pyx_v_key, uint32_t *__pyx_v_data); /* proto*/ /* Module declarations from 'cython' */ /* Module declarations from 'libc.stdint' */ /* Module declarations from 'borg.hashindex' */ static PyTypeObject *__pyx_ptype_4borg_9hashindex_IndexBase = 0; static PyTypeObject *__pyx_ptype_4borg_9hashindex_NSIndex = 0; static PyTypeObject *__pyx_ptype_4borg_9hashindex_NSKeyIterator = 0; static PyTypeObject *__pyx_ptype_4borg_9hashindex_ChunkIndex = 0; static PyTypeObject *__pyx_ptype_4borg_9hashindex_ChunkKeyIterator = 0; static PyObject *__pyx_v_4borg_9hashindex__NoDefault = 0; static uint32_t __pyx_v_4borg_9hashindex__MAX_VALUE; #define __Pyx_MODULE_NAME "borg.hashindex" extern int __pyx_module_is_main_borg__hashindex; int __pyx_module_is_main_borg__hashindex = 0; /* Implementation of 'borg.hashindex' */ static PyObject *__pyx_builtin_object; static PyObject *__pyx_builtin_KeyError; static PyObject *__pyx_builtin_TypeError; static PyObject *__pyx_builtin_IndexError; static PyObject *__pyx_builtin_StopIteration; static const char __pyx_k_os[] = "os"; static const char __pyx_k_key[] = "key"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_name[] = "__name__"; static const char __pyx_k_path[] = "path"; static const char __pyx_k_read[] = "read"; static const char __pyx_k_refs[] = "refs"; static const char __pyx_k_size[] = "size"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_csize[] = "csize"; static const char __pyx_k_value[] = "value"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_import[] = "__import__"; static const char __pyx_k_marker[] = "marker"; static const char __pyx_k_object[] = "object"; static const char __pyx_k_reduce[] = "__reduce__"; static const char __pyx_k_NSIndex[] = "NSIndex"; static const char __pyx_k_default[] = "default"; static const char __pyx_k_KeyError[] = "KeyError"; static const char __pyx_k_capacity[] = "capacity"; static const char __pyx_k_fsencode[] = "fsencode"; static const char __pyx_k_getstate[] = "__getstate__"; static const char __pyx_k_key_size[] = "key_size"; static const char __pyx_k_setstate[] = "__setstate__"; static const char __pyx_k_MAX_VALUE[] = "MAX_VALUE"; static const char __pyx_k_TypeError[] = "TypeError"; static const char __pyx_k_reduce_ex[] = "__reduce_ex__"; static const char __pyx_k_ChunkIndex[] = "ChunkIndex"; static const char __pyx_k_IndexError[] = "IndexError"; static const char __pyx_k_pyx_vtable[] = "__pyx_vtable__"; static const char __pyx_k_value_size[] = "value_size"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_NSKeyIterator[] = "NSKeyIterator"; static const char __pyx_k_StopIteration[] = "StopIteration"; static const char __pyx_k_reduce_cython[] = "__reduce_cython__"; static const char __pyx_k_MAX_LOAD_FACTOR[] = "MAX_LOAD_FACTOR"; static const char __pyx_k_setstate_cython[] = "__setstate_cython__"; static const char __pyx_k_ChunkKeyIterator[] = "ChunkKeyIterator"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_hashindex_set_failed[] = "hashindex_set failed"; static const char __pyx_k_hashindex_init_failed[] = "hashindex_init failed"; static const char __pyx_k_hashindex_read_failed[] = "hashindex_read failed"; static const char __pyx_k_hashindex_write_failed[] = "hashindex_write failed"; static const char __pyx_k_hashindex_delete_failed[] = "hashindex_delete failed"; static const char __pyx_k_invalid_reference_count[] = "invalid reference count"; static const char __pyx_k_maximum_number_of_segments_reach[] = "maximum number of segments reached"; static const char __pyx_k_no_default___reduce___due_to_non[] = "no default __reduce__ due to non-trivial __cinit__"; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_n_s_ChunkIndex; static PyObject *__pyx_n_s_ChunkKeyIterator; static PyObject *__pyx_n_s_IndexError; static PyObject *__pyx_n_s_KeyError; static PyObject *__pyx_n_s_MAX_LOAD_FACTOR; static PyObject *__pyx_n_s_MAX_VALUE; static PyObject *__pyx_n_s_NSIndex; static PyObject *__pyx_n_s_NSKeyIterator; static PyObject *__pyx_n_s_StopIteration; static PyObject *__pyx_n_s_TypeError; static PyObject *__pyx_n_s_capacity; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_csize; static PyObject *__pyx_n_s_default; static PyObject *__pyx_n_s_fsencode; static PyObject *__pyx_n_s_getstate; static PyObject *__pyx_kp_s_hashindex_delete_failed; static PyObject *__pyx_kp_s_hashindex_init_failed; static PyObject *__pyx_kp_s_hashindex_read_failed; static PyObject *__pyx_kp_s_hashindex_set_failed; static PyObject *__pyx_kp_s_hashindex_write_failed; static PyObject *__pyx_n_s_import; static PyObject *__pyx_kp_s_invalid_reference_count; static PyObject *__pyx_n_s_key; static PyObject *__pyx_n_s_key_size; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_marker; static PyObject *__pyx_kp_s_maximum_number_of_segments_reach; static PyObject *__pyx_n_s_name; static PyObject *__pyx_kp_s_no_default___reduce___due_to_non; static PyObject *__pyx_n_s_object; static PyObject *__pyx_n_s_os; static PyObject *__pyx_n_s_path; static PyObject *__pyx_n_s_pyx_vtable; static PyObject *__pyx_n_s_read; static PyObject *__pyx_n_s_reduce; static PyObject *__pyx_n_s_reduce_cython; static PyObject *__pyx_n_s_reduce_ex; static PyObject *__pyx_n_s_refs; static PyObject *__pyx_n_s_setstate; static PyObject *__pyx_n_s_setstate_cython; static PyObject *__pyx_n_s_size; static PyObject *__pyx_n_s_test; static PyObject *__pyx_n_s_value; static PyObject *__pyx_n_s_value_size; static int __pyx_pf_4borg_9hashindex_9IndexBase___cinit__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_capacity, PyObject *__pyx_v_path, PyObject *__pyx_v_key_size); /* proto */ static void __pyx_pf_4borg_9hashindex_9IndexBase_2__dealloc__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_4read(PyTypeObject *__pyx_v_cls, PyObject *__pyx_v_path); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_6write(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_path); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_8clear(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_10setdefault(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value); /* proto */ static int __pyx_pf_4borg_9hashindex_9IndexBase_12__delitem__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_14get(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_default); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_16pop(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_default); /* proto */ static Py_ssize_t __pyx_pf_4borg_9hashindex_9IndexBase_18__len__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_20__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_22__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex___getitem__(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static int __pyx_pf_4borg_9hashindex_7NSIndex_2__setitem__(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value); /* proto */ static int __pyx_pf_4borg_9hashindex_7NSIndex_4__contains__(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex_6iteritems(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_marker); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex_8__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex_10__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static int __pyx_pf_4borg_9hashindex_13NSKeyIterator___cinit__(struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self, PyObject *__pyx_v_key_size); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_2__iter__(struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_4__next__(struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_6__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_8__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex___getitem__(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static int __pyx_pf_4borg_9hashindex_10ChunkIndex_2__setitem__(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value); /* proto */ static int __pyx_pf_4borg_9hashindex_10ChunkIndex_4__contains__(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_6incref(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_8decref(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_10iteritems(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_marker); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_12summarize(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_14add(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_refs, PyObject *__pyx_v_size, PyObject *__pyx_v_csize); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_16merge(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_other); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_18__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_20__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static int __pyx_pf_4borg_9hashindex_16ChunkKeyIterator___cinit__(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self, PyObject *__pyx_v_key_size); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_2__iter__(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_4__next__(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_6__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self); /* proto */ static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_8__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state); /* proto */ static PyObject *__pyx_tp_new_4borg_9hashindex_IndexBase(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_tp_new_4borg_9hashindex_NSIndex(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_tp_new_4borg_9hashindex_NSKeyIterator(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_tp_new_4borg_9hashindex_ChunkIndex(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_tp_new_4borg_9hashindex_ChunkKeyIterator(PyTypeObject *t, PyObject *a, PyObject *k); /*proto*/ static PyObject *__pyx_int_0; static PyObject *__pyx_int_8; static PyObject *__pyx_int_12; static PyObject *__pyx_int_32; static PyObject *__pyx_int_4294967295; static PyObject *__pyx_k__5; static PyObject *__pyx_tuple_; static PyObject *__pyx_tuple__2; static PyObject *__pyx_tuple__3; static PyObject *__pyx_tuple__4; static PyObject *__pyx_tuple__6; static PyObject *__pyx_tuple__7; static PyObject *__pyx_tuple__8; static PyObject *__pyx_tuple__9; static PyObject *__pyx_tuple__10; static PyObject *__pyx_tuple__11; static PyObject *__pyx_tuple__12; static PyObject *__pyx_tuple__13; static PyObject *__pyx_tuple__14; static PyObject *__pyx_tuple__15; static PyObject *__pyx_tuple__16; /* Late includes */ /* "borg/hashindex.pyx":61 * MAX_VALUE = _MAX_VALUE * * def __cinit__(self, capacity=0, path=None, key_size=32): # <<<<<<<<<<<<<< * self.key_size = key_size * if path: */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_9IndexBase_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static int __pyx_pw_4borg_9hashindex_9IndexBase_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_capacity = 0; PyObject *__pyx_v_path = 0; PyObject *__pyx_v_key_size = 0; int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_capacity,&__pyx_n_s_path,&__pyx_n_s_key_size,0}; PyObject* values[3] = {0,0,0}; values[0] = ((PyObject *)__pyx_int_0); values[1] = ((PyObject *)Py_None); values[2] = ((PyObject *)__pyx_int_32); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_capacity); if (value) { values[0] = value; kw_args--; } } CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path); if (value) { values[1] = value; kw_args--; } } CYTHON_FALLTHROUGH; case 2: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key_size); if (value) { values[2] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(0, 61, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_capacity = values[0]; __pyx_v_path = values[1]; __pyx_v_key_size = values[2]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__cinit__", 0, 0, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 61, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.IndexBase.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return -1; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase___cinit__(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), __pyx_v_capacity, __pyx_v_path, __pyx_v_key_size); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_9IndexBase___cinit__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_capacity, PyObject *__pyx_v_path, PyObject *__pyx_v_key_size) { int __pyx_r; __Pyx_RefNannyDeclarations int __pyx_t_1; int __pyx_t_2; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; char *__pyx_t_6; int __pyx_t_7; __Pyx_RefNannySetupContext("__cinit__", 0); __Pyx_INCREF(__pyx_v_path); /* "borg/hashindex.pyx":62 * * def __cinit__(self, capacity=0, path=None, key_size=32): * self.key_size = key_size # <<<<<<<<<<<<<< * if path: * path = os.fsencode(path) */ __pyx_t_1 = __Pyx_PyInt_As_int(__pyx_v_key_size); if (unlikely((__pyx_t_1 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 62, __pyx_L1_error) __pyx_v_self->key_size = __pyx_t_1; /* "borg/hashindex.pyx":63 * def __cinit__(self, capacity=0, path=None, key_size=32): * self.key_size = key_size * if path: # <<<<<<<<<<<<<< * path = os.fsencode(path) * self.index = hashindex_read(path) */ __pyx_t_2 = __Pyx_PyObject_IsTrue(__pyx_v_path); if (unlikely(__pyx_t_2 < 0)) __PYX_ERR(0, 63, __pyx_L1_error) if (__pyx_t_2) { /* "borg/hashindex.pyx":64 * self.key_size = key_size * if path: * path = os.fsencode(path) # <<<<<<<<<<<<<< * self.index = hashindex_read(path) * if not self.index: */ __Pyx_GetModuleGlobalName(__pyx_t_4, __pyx_n_s_os); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 64, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_t_4, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 64, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_5))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_3 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_4, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_path); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 64, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_DECREF_SET(__pyx_v_path, __pyx_t_3); __pyx_t_3 = 0; /* "borg/hashindex.pyx":65 * if path: * path = os.fsencode(path) * self.index = hashindex_read(path) # <<<<<<<<<<<<<< * if not self.index: * raise Exception('hashindex_read failed') */ __pyx_t_6 = __Pyx_PyObject_AsWritableString(__pyx_v_path); if (unlikely((!__pyx_t_6) && PyErr_Occurred())) __PYX_ERR(0, 65, __pyx_L1_error) __pyx_v_self->index = hashindex_read(__pyx_t_6); /* "borg/hashindex.pyx":66 * path = os.fsencode(path) * self.index = hashindex_read(path) * if not self.index: # <<<<<<<<<<<<<< * raise Exception('hashindex_read failed') * else: */ __pyx_t_2 = ((!(__pyx_v_self->index != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/hashindex.pyx":67 * self.index = hashindex_read(path) * if not self.index: * raise Exception('hashindex_read failed') # <<<<<<<<<<<<<< * else: * self.index = hashindex_init(capacity, self.key_size, self.value_size) */ __pyx_t_3 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple_, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 67, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_Raise(__pyx_t_3, 0, 0, 0); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __PYX_ERR(0, 67, __pyx_L1_error) /* "borg/hashindex.pyx":66 * path = os.fsencode(path) * self.index = hashindex_read(path) * if not self.index: # <<<<<<<<<<<<<< * raise Exception('hashindex_read failed') * else: */ } /* "borg/hashindex.pyx":63 * def __cinit__(self, capacity=0, path=None, key_size=32): * self.key_size = key_size * if path: # <<<<<<<<<<<<<< * path = os.fsencode(path) * self.index = hashindex_read(path) */ goto __pyx_L3; } /* "borg/hashindex.pyx":69 * raise Exception('hashindex_read failed') * else: * self.index = hashindex_init(capacity, self.key_size, self.value_size) # <<<<<<<<<<<<<< * if not self.index: * raise Exception('hashindex_init failed') */ /*else*/ { __pyx_t_1 = __Pyx_PyInt_As_int(__pyx_v_capacity); if (unlikely((__pyx_t_1 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 69, __pyx_L1_error) __pyx_t_3 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_value_size); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_7 = __Pyx_PyInt_As_int(__pyx_t_3); if (unlikely((__pyx_t_7 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_v_self->index = hashindex_init(__pyx_t_1, __pyx_v_self->key_size, __pyx_t_7); /* "borg/hashindex.pyx":70 * else: * self.index = hashindex_init(capacity, self.key_size, self.value_size) * if not self.index: # <<<<<<<<<<<<<< * raise Exception('hashindex_init failed') * */ __pyx_t_2 = ((!(__pyx_v_self->index != 0)) != 0); if (unlikely(__pyx_t_2)) { /* "borg/hashindex.pyx":71 * self.index = hashindex_init(capacity, self.key_size, self.value_size) * if not self.index: * raise Exception('hashindex_init failed') # <<<<<<<<<<<<<< * * def __dealloc__(self): */ __pyx_t_3 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 71, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_Raise(__pyx_t_3, 0, 0, 0); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __PYX_ERR(0, 71, __pyx_L1_error) /* "borg/hashindex.pyx":70 * else: * self.index = hashindex_init(capacity, self.key_size, self.value_size) * if not self.index: # <<<<<<<<<<<<<< * raise Exception('hashindex_init failed') * */ } } __pyx_L3:; /* "borg/hashindex.pyx":61 * MAX_VALUE = _MAX_VALUE * * def __cinit__(self, capacity=0, path=None, key_size=32): # <<<<<<<<<<<<<< * self.key_size = key_size * if path: */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.hashindex.IndexBase.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_XDECREF(__pyx_v_path); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":73 * raise Exception('hashindex_init failed') * * def __dealloc__(self): # <<<<<<<<<<<<<< * if self.index: * hashindex_free(self.index) */ /* Python wrapper */ static void __pyx_pw_4borg_9hashindex_9IndexBase_3__dealloc__(PyObject *__pyx_v_self); /*proto*/ static void __pyx_pw_4borg_9hashindex_9IndexBase_3__dealloc__(PyObject *__pyx_v_self) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__dealloc__ (wrapper)", 0); __pyx_pf_4borg_9hashindex_9IndexBase_2__dealloc__(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); } static void __pyx_pf_4borg_9hashindex_9IndexBase_2__dealloc__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self) { __Pyx_RefNannyDeclarations int __pyx_t_1; __Pyx_RefNannySetupContext("__dealloc__", 0); /* "borg/hashindex.pyx":74 * * def __dealloc__(self): * if self.index: # <<<<<<<<<<<<<< * hashindex_free(self.index) * */ __pyx_t_1 = (__pyx_v_self->index != 0); if (__pyx_t_1) { /* "borg/hashindex.pyx":75 * def __dealloc__(self): * if self.index: * hashindex_free(self.index) # <<<<<<<<<<<<<< * * @classmethod */ hashindex_free(__pyx_v_self->index); /* "borg/hashindex.pyx":74 * * def __dealloc__(self): * if self.index: # <<<<<<<<<<<<<< * hashindex_free(self.index) * */ } /* "borg/hashindex.pyx":73 * raise Exception('hashindex_init failed') * * def __dealloc__(self): # <<<<<<<<<<<<<< * if self.index: * hashindex_free(self.index) */ /* function exit code */ __Pyx_RefNannyFinishContext(); } /* "borg/hashindex.pyx":78 * * @classmethod * def read(cls, path): # <<<<<<<<<<<<<< * return cls(path=path) * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_5read(PyObject *__pyx_v_cls, PyObject *__pyx_v_path); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_5read(PyObject *__pyx_v_cls, PyObject *__pyx_v_path) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("read (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_4read(((PyTypeObject*)__pyx_v_cls), ((PyObject *)__pyx_v_path)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_4read(PyTypeObject *__pyx_v_cls, PyObject *__pyx_v_path) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannySetupContext("read", 0); /* "borg/hashindex.pyx":79 * @classmethod * def read(cls, path): * return cls(path=path) # <<<<<<<<<<<<<< * * def write(self, path): */ __Pyx_XDECREF(__pyx_r); __pyx_t_1 = __Pyx_PyDict_NewPresized(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 79, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_t_1, __pyx_n_s_path, __pyx_v_path) < 0) __PYX_ERR(0, 79, __pyx_L1_error) __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)__pyx_v_cls), __pyx_empty_tuple, __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 79, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_2; __pyx_t_2 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":78 * * @classmethod * def read(cls, path): # <<<<<<<<<<<<<< * return cls(path=path) * */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.hashindex.IndexBase.read", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":81 * return cls(path=path) * * def write(self, path): # <<<<<<<<<<<<<< * path = os.fsencode(path) * if not hashindex_write(self.index, path): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_7write(PyObject *__pyx_v_self, PyObject *__pyx_v_path); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_7write(PyObject *__pyx_v_self, PyObject *__pyx_v_path) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("write (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_6write(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), ((PyObject *)__pyx_v_path)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_6write(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_path) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; char *__pyx_t_4; int __pyx_t_5; __Pyx_RefNannySetupContext("write", 0); __Pyx_INCREF(__pyx_v_path); /* "borg/hashindex.pyx":82 * * def write(self, path): * path = os.fsencode(path) # <<<<<<<<<<<<<< * if not hashindex_write(self.index, path): * raise Exception('hashindex_write failed') */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_os); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 82, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 82, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_path); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 82, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF_SET(__pyx_v_path, __pyx_t_1); __pyx_t_1 = 0; /* "borg/hashindex.pyx":83 * def write(self, path): * path = os.fsencode(path) * if not hashindex_write(self.index, path): # <<<<<<<<<<<<<< * raise Exception('hashindex_write failed') * */ __pyx_t_4 = __Pyx_PyObject_AsWritableString(__pyx_v_path); if (unlikely((!__pyx_t_4) && PyErr_Occurred())) __PYX_ERR(0, 83, __pyx_L1_error) __pyx_t_5 = ((!(hashindex_write(__pyx_v_self->index, __pyx_t_4) != 0)) != 0); if (unlikely(__pyx_t_5)) { /* "borg/hashindex.pyx":84 * path = os.fsencode(path) * if not hashindex_write(self.index, path): * raise Exception('hashindex_write failed') # <<<<<<<<<<<<<< * * def clear(self): */ __pyx_t_1 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__3, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 84, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(0, 84, __pyx_L1_error) /* "borg/hashindex.pyx":83 * def write(self, path): * path = os.fsencode(path) * if not hashindex_write(self.index, path): # <<<<<<<<<<<<<< * raise Exception('hashindex_write failed') * */ } /* "borg/hashindex.pyx":81 * return cls(path=path) * * def write(self, path): # <<<<<<<<<<<<<< * path = os.fsencode(path) * if not hashindex_write(self.index, path): */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.hashindex.IndexBase.write", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_path); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":86 * raise Exception('hashindex_write failed') * * def clear(self): # <<<<<<<<<<<<<< * hashindex_free(self.index) * self.index = hashindex_init(0, self.key_size, self.value_size) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_9clear(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_9clear(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("clear (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_8clear(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_8clear(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; int __pyx_t_2; int __pyx_t_3; __Pyx_RefNannySetupContext("clear", 0); /* "borg/hashindex.pyx":87 * * def clear(self): * hashindex_free(self.index) # <<<<<<<<<<<<<< * self.index = hashindex_init(0, self.key_size, self.value_size) * if not self.index: */ hashindex_free(__pyx_v_self->index); /* "borg/hashindex.pyx":88 * def clear(self): * hashindex_free(self.index) * self.index = hashindex_init(0, self.key_size, self.value_size) # <<<<<<<<<<<<<< * if not self.index: * raise Exception('hashindex_init failed') */ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(((PyObject *)__pyx_v_self), __pyx_n_s_value_size); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 88, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyInt_As_int(__pyx_t_1); if (unlikely((__pyx_t_2 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 88, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_self->index = hashindex_init(0, __pyx_v_self->key_size, __pyx_t_2); /* "borg/hashindex.pyx":89 * hashindex_free(self.index) * self.index = hashindex_init(0, self.key_size, self.value_size) * if not self.index: # <<<<<<<<<<<<<< * raise Exception('hashindex_init failed') * */ __pyx_t_3 = ((!(__pyx_v_self->index != 0)) != 0); if (unlikely(__pyx_t_3)) { /* "borg/hashindex.pyx":90 * self.index = hashindex_init(0, self.key_size, self.value_size) * if not self.index: * raise Exception('hashindex_init failed') # <<<<<<<<<<<<<< * * def setdefault(self, key, value): */ __pyx_t_1 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 90, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(0, 90, __pyx_L1_error) /* "borg/hashindex.pyx":89 * hashindex_free(self.index) * self.index = hashindex_init(0, self.key_size, self.value_size) * if not self.index: # <<<<<<<<<<<<<< * raise Exception('hashindex_init failed') * */ } /* "borg/hashindex.pyx":86 * raise Exception('hashindex_write failed') * * def clear(self): # <<<<<<<<<<<<<< * hashindex_free(self.index) * self.index = hashindex_init(0, self.key_size, self.value_size) */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.IndexBase.clear", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":92 * raise Exception('hashindex_init failed') * * def setdefault(self, key, value): # <<<<<<<<<<<<<< * if not key in self: * self[key] = value */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_11setdefault(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_11setdefault(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key = 0; PyObject *__pyx_v_value = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("setdefault (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key,&__pyx_n_s_value,0}; PyObject* values[2] = {0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_value)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("setdefault", 1, 2, 2, 1); __PYX_ERR(0, 92, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "setdefault") < 0)) __PYX_ERR(0, 92, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 2) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); } __pyx_v_key = values[0]; __pyx_v_value = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("setdefault", 1, 2, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 92, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.IndexBase.setdefault", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_10setdefault(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), __pyx_v_key, __pyx_v_value); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_10setdefault(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; int __pyx_t_2; __Pyx_RefNannySetupContext("setdefault", 0); /* "borg/hashindex.pyx":93 * * def setdefault(self, key, value): * if not key in self: # <<<<<<<<<<<<<< * self[key] = value * */ __pyx_t_1 = (__Pyx_PySequence_ContainsTF(__pyx_v_key, ((PyObject *)__pyx_v_self), Py_NE)); if (unlikely(__pyx_t_1 < 0)) __PYX_ERR(0, 93, __pyx_L1_error) __pyx_t_2 = (__pyx_t_1 != 0); if (__pyx_t_2) { /* "borg/hashindex.pyx":94 * def setdefault(self, key, value): * if not key in self: * self[key] = value # <<<<<<<<<<<<<< * * def __delitem__(self, key): */ if (unlikely(PyObject_SetItem(((PyObject *)__pyx_v_self), __pyx_v_key, __pyx_v_value) < 0)) __PYX_ERR(0, 94, __pyx_L1_error) /* "borg/hashindex.pyx":93 * * def setdefault(self, key, value): * if not key in self: # <<<<<<<<<<<<<< * self[key] = value * */ } /* "borg/hashindex.pyx":92 * raise Exception('hashindex_init failed') * * def setdefault(self, key, value): # <<<<<<<<<<<<<< * if not key in self: * self[key] = value */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_AddTraceback("borg.hashindex.IndexBase.setdefault", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":96 * self[key] = value * * def __delitem__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * if not hashindex_delete(self.index, key): */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_9IndexBase_13__delitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static int __pyx_pw_4borg_9hashindex_9IndexBase_13__delitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__delitem__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_12__delitem__(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_9IndexBase_12__delitem__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key) { int __pyx_r; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("__delitem__", 0); /* "borg/hashindex.pyx":97 * * def __delitem__(self, key): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * if not hashindex_delete(self.index, key): * raise Exception('hashindex_delete failed') */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 97, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 97, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":98 * def __delitem__(self, key): * assert len(key) == self.key_size * if not hashindex_delete(self.index, key): # <<<<<<<<<<<<<< * raise Exception('hashindex_delete failed') * */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 98, __pyx_L1_error) __pyx_t_3 = ((!(hashindex_delete(__pyx_v_self->index, ((char *)__pyx_t_2)) != 0)) != 0); if (unlikely(__pyx_t_3)) { /* "borg/hashindex.pyx":99 * assert len(key) == self.key_size * if not hashindex_delete(self.index, key): * raise Exception('hashindex_delete failed') # <<<<<<<<<<<<<< * * def get(self, key, default=None): */ __pyx_t_4 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__4, NULL); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 99, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 99, __pyx_L1_error) /* "borg/hashindex.pyx":98 * def __delitem__(self, key): * assert len(key) == self.key_size * if not hashindex_delete(self.index, key): # <<<<<<<<<<<<<< * raise Exception('hashindex_delete failed') * */ } /* "borg/hashindex.pyx":96 * self[key] = value * * def __delitem__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * if not hashindex_delete(self.index, key): */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.hashindex.IndexBase.__delitem__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":101 * raise Exception('hashindex_delete failed') * * def get(self, key, default=None): # <<<<<<<<<<<<<< * try: * return self[key] */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_15get(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_15get(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key = 0; PyObject *__pyx_v_default = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("get (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key,&__pyx_n_s_default,0}; PyObject* values[2] = {0,0}; values[1] = ((PyObject *)Py_None); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_default); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "get") < 0)) __PYX_ERR(0, 101, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_key = values[0]; __pyx_v_default = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("get", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 101, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.IndexBase.get", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_14get(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), __pyx_v_key, __pyx_v_default); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_14get(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_default) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; int __pyx_t_5; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; __Pyx_RefNannySetupContext("get", 0); /* "borg/hashindex.pyx":102 * * def get(self, key, default=None): * try: # <<<<<<<<<<<<<< * return self[key] * except KeyError: */ { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3); __Pyx_XGOTREF(__pyx_t_1); __Pyx_XGOTREF(__pyx_t_2); __Pyx_XGOTREF(__pyx_t_3); /*try:*/ { /* "borg/hashindex.pyx":103 * def get(self, key, default=None): * try: * return self[key] # <<<<<<<<<<<<<< * except KeyError: * return default */ __Pyx_XDECREF(__pyx_r); __pyx_t_4 = __Pyx_PyObject_GetItem(((PyObject *)__pyx_v_self), __pyx_v_key); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 103, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_4); __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L7_try_return; /* "borg/hashindex.pyx":102 * * def get(self, key, default=None): * try: # <<<<<<<<<<<<<< * return self[key] * except KeyError: */ } __pyx_L3_error:; __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/hashindex.pyx":104 * try: * return self[key] * except KeyError: # <<<<<<<<<<<<<< * return default * */ __pyx_t_5 = __Pyx_PyErr_ExceptionMatches(__pyx_builtin_KeyError); if (__pyx_t_5) { __Pyx_AddTraceback("borg.hashindex.IndexBase.get", __pyx_clineno, __pyx_lineno, __pyx_filename); if (__Pyx_GetException(&__pyx_t_4, &__pyx_t_6, &__pyx_t_7) < 0) __PYX_ERR(0, 104, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_GOTREF(__pyx_t_6); __Pyx_GOTREF(__pyx_t_7); /* "borg/hashindex.pyx":105 * return self[key] * except KeyError: * return default # <<<<<<<<<<<<<< * * def pop(self, key, default=_NoDefault): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(__pyx_v_default); __pyx_r = __pyx_v_default; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; goto __pyx_L6_except_return; } goto __pyx_L5_except_error; __pyx_L5_except_error:; /* "borg/hashindex.pyx":102 * * def get(self, key, default=None): * try: # <<<<<<<<<<<<<< * return self[key] * except KeyError: */ __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L1_error; __pyx_L7_try_return:; __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L0; __pyx_L6_except_return:; __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L0; } /* "borg/hashindex.pyx":101 * raise Exception('hashindex_delete failed') * * def get(self, key, default=None): # <<<<<<<<<<<<<< * try: * return self[key] */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_AddTraceback("borg.hashindex.IndexBase.get", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":107 * return default * * def pop(self, key, default=_NoDefault): # <<<<<<<<<<<<<< * try: * value = self[key] */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_17pop(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_17pop(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key = 0; PyObject *__pyx_v_default = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("pop (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key,&__pyx_n_s_default,0}; PyObject* values[2] = {0,0}; values[1] = __pyx_k__5; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_default); if (value) { values[1] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "pop") < 0)) __PYX_ERR(0, 107, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_key = values[0]; __pyx_v_default = values[1]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("pop", 0, 1, 2, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 107, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.IndexBase.pop", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_16pop(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), __pyx_v_key, __pyx_v_default); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_16pop(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_default) { PyObject *__pyx_v_value = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; int __pyx_t_5; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; int __pyx_t_9; __Pyx_RefNannySetupContext("pop", 0); /* "borg/hashindex.pyx":108 * * def pop(self, key, default=_NoDefault): * try: # <<<<<<<<<<<<<< * value = self[key] * del self[key] */ { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3); __Pyx_XGOTREF(__pyx_t_1); __Pyx_XGOTREF(__pyx_t_2); __Pyx_XGOTREF(__pyx_t_3); /*try:*/ { /* "borg/hashindex.pyx":109 * def pop(self, key, default=_NoDefault): * try: * value = self[key] # <<<<<<<<<<<<<< * del self[key] * return value */ __pyx_t_4 = __Pyx_PyObject_GetItem(((PyObject *)__pyx_v_self), __pyx_v_key); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 109, __pyx_L3_error) __Pyx_GOTREF(__pyx_t_4); __pyx_v_value = __pyx_t_4; __pyx_t_4 = 0; /* "borg/hashindex.pyx":110 * try: * value = self[key] * del self[key] # <<<<<<<<<<<<<< * return value * except KeyError: */ if (unlikely(PyObject_DelItem(((PyObject *)__pyx_v_self), __pyx_v_key) < 0)) __PYX_ERR(0, 110, __pyx_L3_error) /* "borg/hashindex.pyx":111 * value = self[key] * del self[key] * return value # <<<<<<<<<<<<<< * except KeyError: * if default != _NoDefault: */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(__pyx_v_value); __pyx_r = __pyx_v_value; goto __pyx_L7_try_return; /* "borg/hashindex.pyx":108 * * def pop(self, key, default=_NoDefault): * try: # <<<<<<<<<<<<<< * value = self[key] * del self[key] */ } __pyx_L3_error:; __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/hashindex.pyx":112 * del self[key] * return value * except KeyError: # <<<<<<<<<<<<<< * if default != _NoDefault: * return default */ __pyx_t_5 = __Pyx_PyErr_ExceptionMatches(__pyx_builtin_KeyError); if (__pyx_t_5) { __Pyx_AddTraceback("borg.hashindex.IndexBase.pop", __pyx_clineno, __pyx_lineno, __pyx_filename); if (__Pyx_GetException(&__pyx_t_4, &__pyx_t_6, &__pyx_t_7) < 0) __PYX_ERR(0, 112, __pyx_L5_except_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_GOTREF(__pyx_t_6); __Pyx_GOTREF(__pyx_t_7); /* "borg/hashindex.pyx":113 * return value * except KeyError: * if default != _NoDefault: # <<<<<<<<<<<<<< * return default * raise */ __pyx_t_8 = PyObject_RichCompare(__pyx_v_default, __pyx_v_4borg_9hashindex__NoDefault, Py_NE); __Pyx_XGOTREF(__pyx_t_8); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 113, __pyx_L5_except_error) __pyx_t_9 = __Pyx_PyObject_IsTrue(__pyx_t_8); if (unlikely(__pyx_t_9 < 0)) __PYX_ERR(0, 113, __pyx_L5_except_error) __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; if (__pyx_t_9) { /* "borg/hashindex.pyx":114 * except KeyError: * if default != _NoDefault: * return default # <<<<<<<<<<<<<< * raise * */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(__pyx_v_default); __pyx_r = __pyx_v_default; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; goto __pyx_L6_except_return; /* "borg/hashindex.pyx":113 * return value * except KeyError: * if default != _NoDefault: # <<<<<<<<<<<<<< * return default * raise */ } /* "borg/hashindex.pyx":115 * if default != _NoDefault: * return default * raise # <<<<<<<<<<<<<< * * def __len__(self): */ __Pyx_GIVEREF(__pyx_t_4); __Pyx_GIVEREF(__pyx_t_6); __Pyx_XGIVEREF(__pyx_t_7); __Pyx_ErrRestoreWithState(__pyx_t_4, __pyx_t_6, __pyx_t_7); __pyx_t_4 = 0; __pyx_t_6 = 0; __pyx_t_7 = 0; __PYX_ERR(0, 115, __pyx_L5_except_error) } goto __pyx_L5_except_error; __pyx_L5_except_error:; /* "borg/hashindex.pyx":108 * * def pop(self, key, default=_NoDefault): * try: # <<<<<<<<<<<<<< * value = self[key] * del self[key] */ __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L1_error; __pyx_L7_try_return:; __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L0; __pyx_L6_except_return:; __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L0; } /* "borg/hashindex.pyx":107 * return default * * def pop(self, key, default=_NoDefault): # <<<<<<<<<<<<<< * try: * value = self[key] */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.hashindex.IndexBase.pop", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_value); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":117 * raise * * def __len__(self): # <<<<<<<<<<<<<< * return hashindex_get_size(self.index) * */ /* Python wrapper */ static Py_ssize_t __pyx_pw_4borg_9hashindex_9IndexBase_19__len__(PyObject *__pyx_v_self); /*proto*/ static Py_ssize_t __pyx_pw_4borg_9hashindex_9IndexBase_19__len__(PyObject *__pyx_v_self) { Py_ssize_t __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__len__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_18__len__(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static Py_ssize_t __pyx_pf_4borg_9hashindex_9IndexBase_18__len__(struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self) { Py_ssize_t __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__len__", 0); /* "borg/hashindex.pyx":118 * * def __len__(self): * return hashindex_get_size(self.index) # <<<<<<<<<<<<<< * * */ __pyx_r = hashindex_get_size(__pyx_v_self->index); goto __pyx_L0; /* "borg/hashindex.pyx":117 * raise * * def __len__(self): # <<<<<<<<<<<<<< * return hashindex_get_size(self.index) * */ /* function exit code */ __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_21__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_21__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_20__reduce_cython__(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_20__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__6, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.IndexBase.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_23__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_9IndexBase_23__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_9IndexBase_22__setstate_cython__(((struct __pyx_obj_4borg_9hashindex_IndexBase *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_9IndexBase_22__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_IndexBase *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__7, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.IndexBase.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":125 * value_size = 8 * * def __getitem__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * data = hashindex_get(self.index, key) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_1__getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_1__getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__getitem__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_7NSIndex___getitem__(((struct __pyx_obj_4borg_9hashindex_NSIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex___getitem__(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_key) { uint32_t *__pyx_v_data; uint32_t __pyx_v_segment; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; __Pyx_RefNannySetupContext("__getitem__", 0); /* "borg/hashindex.pyx":126 * * def __getitem__(self, key): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * data = hashindex_get(self.index, key) * if not data: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 126, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 126, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":127 * def __getitem__(self, key): * assert len(key) == self.key_size * data = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if not data: * raise KeyError(key) */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 127, __pyx_L1_error) __pyx_v_data = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_2))); /* "borg/hashindex.pyx":128 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t segment = _le32toh(data[0]) */ __pyx_t_3 = ((!(__pyx_v_data != 0)) != 0); if (unlikely(__pyx_t_3)) { /* "borg/hashindex.pyx":129 * data = hashindex_get(self.index, key) * if not data: * raise KeyError(key) # <<<<<<<<<<<<<< * cdef uint32_t segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" */ __pyx_t_4 = __Pyx_PyObject_CallOneArg(__pyx_builtin_KeyError, __pyx_v_key); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 129, __pyx_L1_error) /* "borg/hashindex.pyx":128 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t segment = _le32toh(data[0]) */ } /* "borg/hashindex.pyx":130 * if not data: * raise KeyError(key) * cdef uint32_t segment = _le32toh(data[0]) # <<<<<<<<<<<<<< * assert segment <= _MAX_VALUE, "maximum number of segments reached" * return segment, _le32toh(data[1]) */ __pyx_v_segment = _le32toh((__pyx_v_data[0])); /* "borg/hashindex.pyx":131 * raise KeyError(key) * cdef uint32_t segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" # <<<<<<<<<<<<<< * return segment, _le32toh(data[1]) * */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_segment <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_maximum_number_of_segments_reach); __PYX_ERR(0, 131, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":132 * cdef uint32_t segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" * return segment, _le32toh(data[1]) # <<<<<<<<<<<<<< * * def __setitem__(self, key, value): */ __Pyx_XDECREF(__pyx_r); __pyx_t_4 = __Pyx_PyInt_From_uint32_t(__pyx_v_segment); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[1]))); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = PyTuple_New(2); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 132, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_6, 1, __pyx_t_5); __pyx_t_4 = 0; __pyx_t_5 = 0; __pyx_r = __pyx_t_6; __pyx_t_6 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":125 * value_size = 8 * * def __getitem__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * data = hashindex_get(self.index, key) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.hashindex.NSIndex.__getitem__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":134 * return segment, _le32toh(data[1]) * * def __setitem__(self, key, value): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * cdef uint32_t[2] data */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_7NSIndex_3__setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value); /*proto*/ static int __pyx_pw_4borg_9hashindex_7NSIndex_3__setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value) { int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setitem__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_7NSIndex_2__setitem__(((struct __pyx_obj_4borg_9hashindex_NSIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key), ((PyObject *)__pyx_v_value)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_7NSIndex_2__setitem__(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value) { uint32_t __pyx_v_data[2]; uint32_t __pyx_v_segment; int __pyx_r; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; PyObject *__pyx_t_2 = NULL; uint32_t __pyx_t_3; char *__pyx_t_4; int __pyx_t_5; __Pyx_RefNannySetupContext("__setitem__", 0); /* "borg/hashindex.pyx":135 * * def __setitem__(self, key, value): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * cdef uint32_t[2] data * cdef uint32_t segment = value[0] */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 135, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 135, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":137 * assert len(key) == self.key_size * cdef uint32_t[2] data * cdef uint32_t segment = value[0] # <<<<<<<<<<<<<< * assert segment <= _MAX_VALUE, "maximum number of segments reached" * data[0] = _htole32(segment) */ __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_value, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 137, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 137, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_segment = __pyx_t_3; /* "borg/hashindex.pyx":138 * cdef uint32_t[2] data * cdef uint32_t segment = value[0] * assert segment <= _MAX_VALUE, "maximum number of segments reached" # <<<<<<<<<<<<<< * data[0] = _htole32(segment) * data[1] = _htole32(value[1]) */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_segment <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_maximum_number_of_segments_reach); __PYX_ERR(0, 138, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":139 * cdef uint32_t segment = value[0] * assert segment <= _MAX_VALUE, "maximum number of segments reached" * data[0] = _htole32(segment) # <<<<<<<<<<<<<< * data[1] = _htole32(value[1]) * if not hashindex_set(self.index, key, data): */ (__pyx_v_data[0]) = _htole32(__pyx_v_segment); /* "borg/hashindex.pyx":140 * assert segment <= _MAX_VALUE, "maximum number of segments reached" * data[0] = _htole32(segment) * data[1] = _htole32(value[1]) # <<<<<<<<<<<<<< * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') */ __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_value, 1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 140, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 140, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; (__pyx_v_data[1]) = _htole32(__pyx_t_3); /* "borg/hashindex.pyx":141 * data[0] = _htole32(segment) * data[1] = _htole32(value[1]) * if not hashindex_set(self.index, key, data): # <<<<<<<<<<<<<< * raise Exception('hashindex_set failed') * */ __pyx_t_4 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_4) && PyErr_Occurred())) __PYX_ERR(0, 141, __pyx_L1_error) __pyx_t_5 = ((!(hashindex_set(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_4), __pyx_v_data) != 0)) != 0); if (unlikely(__pyx_t_5)) { /* "borg/hashindex.pyx":142 * data[1] = _htole32(value[1]) * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') # <<<<<<<<<<<<<< * * def __contains__(self, key): */ __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__8, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 142, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_Raise(__pyx_t_2, 0, 0, 0); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __PYX_ERR(0, 142, __pyx_L1_error) /* "borg/hashindex.pyx":141 * data[0] = _htole32(segment) * data[1] = _htole32(value[1]) * if not hashindex_set(self.index, key, data): # <<<<<<<<<<<<<< * raise Exception('hashindex_set failed') * */ } /* "borg/hashindex.pyx":134 * return segment, _le32toh(data[1]) * * def __setitem__(self, key, value): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * cdef uint32_t[2] data */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.hashindex.NSIndex.__setitem__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":144 * raise Exception('hashindex_set failed') * * def __contains__(self, key): # <<<<<<<<<<<<<< * cdef uint32_t segment * assert len(key) == self.key_size */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_7NSIndex_5__contains__(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static int __pyx_pw_4borg_9hashindex_7NSIndex_5__contains__(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__contains__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_7NSIndex_4__contains__(((struct __pyx_obj_4borg_9hashindex_NSIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_7NSIndex_4__contains__(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_key) { uint32_t __pyx_v_segment; uint32_t *__pyx_v_data; int __pyx_r; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; __Pyx_RefNannySetupContext("__contains__", 0); /* "borg/hashindex.pyx":146 * def __contains__(self, key): * cdef uint32_t segment * assert len(key) == self.key_size # <<<<<<<<<<<<<< * data = hashindex_get(self.index, key) * if data != NULL: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 146, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 146, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":147 * cdef uint32_t segment * assert len(key) == self.key_size * data = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if data != NULL: * segment = _le32toh(data[0]) */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 147, __pyx_L1_error) __pyx_v_data = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_2))); /* "borg/hashindex.pyx":148 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if data != NULL: # <<<<<<<<<<<<<< * segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" */ __pyx_t_3 = ((__pyx_v_data != NULL) != 0); if (__pyx_t_3) { /* "borg/hashindex.pyx":149 * data = hashindex_get(self.index, key) * if data != NULL: * segment = _le32toh(data[0]) # <<<<<<<<<<<<<< * assert segment <= _MAX_VALUE, "maximum number of segments reached" * return data != NULL */ __pyx_v_segment = _le32toh((__pyx_v_data[0])); /* "borg/hashindex.pyx":150 * if data != NULL: * segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" # <<<<<<<<<<<<<< * return data != NULL * */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_segment <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_maximum_number_of_segments_reach); __PYX_ERR(0, 150, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":148 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if data != NULL: # <<<<<<<<<<<<<< * segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" */ } /* "borg/hashindex.pyx":151 * segment = _le32toh(data[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" * return data != NULL # <<<<<<<<<<<<<< * * def iteritems(self, marker=None): */ __pyx_r = (__pyx_v_data != NULL); goto __pyx_L0; /* "borg/hashindex.pyx":144 * raise Exception('hashindex_set failed') * * def __contains__(self, key): # <<<<<<<<<<<<<< * cdef uint32_t segment * assert len(key) == self.key_size */ /* function exit code */ __pyx_L1_error:; __Pyx_AddTraceback("borg.hashindex.NSIndex.__contains__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":153 * return data != NULL * * def iteritems(self, marker=None): # <<<<<<<<<<<<<< * cdef const void *key * iter = NSKeyIterator(self.key_size) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_7iteritems(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_7iteritems(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_marker = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("iteritems (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_marker,0}; PyObject* values[1] = {0}; values[0] = ((PyObject *)Py_None); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_marker); if (value) { values[0] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "iteritems") < 0)) __PYX_ERR(0, 153, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_marker = values[0]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("iteritems", 0, 0, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 153, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.NSIndex.iteritems", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_7NSIndex_6iteritems(((struct __pyx_obj_4borg_9hashindex_NSIndex *)__pyx_v_self), __pyx_v_marker); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex_6iteritems(struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, PyObject *__pyx_v_marker) { void const *__pyx_v_key; struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_iter = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; HashIndex *__pyx_t_3; int __pyx_t_4; char *__pyx_t_5; int __pyx_t_6; __Pyx_RefNannySetupContext("iteritems", 0); /* "borg/hashindex.pyx":155 * def iteritems(self, marker=None): * cdef const void *key * iter = NSKeyIterator(self.key_size) # <<<<<<<<<<<<<< * iter.idx = self * iter.index = self.index */ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->__pyx_base.key_size); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 155, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_CallOneArg(((PyObject *)__pyx_ptype_4borg_9hashindex_NSKeyIterator), __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 155, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_iter = ((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)__pyx_t_2); __pyx_t_2 = 0; /* "borg/hashindex.pyx":156 * cdef const void *key * iter = NSKeyIterator(self.key_size) * iter.idx = self # <<<<<<<<<<<<<< * iter.index = self.index * if marker: */ __Pyx_INCREF(((PyObject *)__pyx_v_self)); __Pyx_GIVEREF(((PyObject *)__pyx_v_self)); __Pyx_GOTREF(__pyx_v_iter->idx); __Pyx_DECREF(((PyObject *)__pyx_v_iter->idx)); __pyx_v_iter->idx = __pyx_v_self; /* "borg/hashindex.pyx":157 * iter = NSKeyIterator(self.key_size) * iter.idx = self * iter.index = self.index # <<<<<<<<<<<<<< * if marker: * key = hashindex_get(self.index, marker) */ __pyx_t_3 = __pyx_v_self->__pyx_base.index; __pyx_v_iter->index = __pyx_t_3; /* "borg/hashindex.pyx":158 * iter.idx = self * iter.index = self.index * if marker: # <<<<<<<<<<<<<< * key = hashindex_get(self.index, marker) * if marker is None: */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_marker); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 158, __pyx_L1_error) if (__pyx_t_4) { /* "borg/hashindex.pyx":159 * iter.index = self.index * if marker: * key = hashindex_get(self.index, marker) # <<<<<<<<<<<<<< * if marker is None: * raise IndexError */ __pyx_t_5 = __Pyx_PyObject_AsWritableString(__pyx_v_marker); if (unlikely((!__pyx_t_5) && PyErr_Occurred())) __PYX_ERR(0, 159, __pyx_L1_error) __pyx_v_key = hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_5)); /* "borg/hashindex.pyx":160 * if marker: * key = hashindex_get(self.index, marker) * if marker is None: # <<<<<<<<<<<<<< * raise IndexError * iter.key = key - self.key_size */ __pyx_t_4 = (__pyx_v_marker == Py_None); __pyx_t_6 = (__pyx_t_4 != 0); if (unlikely(__pyx_t_6)) { /* "borg/hashindex.pyx":161 * key = hashindex_get(self.index, marker) * if marker is None: * raise IndexError # <<<<<<<<<<<<<< * iter.key = key - self.key_size * return iter */ __Pyx_Raise(__pyx_builtin_IndexError, 0, 0, 0); __PYX_ERR(0, 161, __pyx_L1_error) /* "borg/hashindex.pyx":160 * if marker: * key = hashindex_get(self.index, marker) * if marker is None: # <<<<<<<<<<<<<< * raise IndexError * iter.key = key - self.key_size */ } /* "borg/hashindex.pyx":162 * if marker is None: * raise IndexError * iter.key = key - self.key_size # <<<<<<<<<<<<<< * return iter * */ __pyx_v_iter->key = (__pyx_v_key - __pyx_v_self->__pyx_base.key_size); /* "borg/hashindex.pyx":158 * iter.idx = self * iter.index = self.index * if marker: # <<<<<<<<<<<<<< * key = hashindex_get(self.index, marker) * if marker is None: */ } /* "borg/hashindex.pyx":163 * raise IndexError * iter.key = key - self.key_size * return iter # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(((PyObject *)__pyx_v_iter)); __pyx_r = ((PyObject *)__pyx_v_iter); goto __pyx_L0; /* "borg/hashindex.pyx":153 * return data != NULL * * def iteritems(self, marker=None): # <<<<<<<<<<<<<< * cdef const void *key * iter = NSKeyIterator(self.key_size) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.hashindex.NSIndex.iteritems", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF((PyObject *)__pyx_v_iter); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_9__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_9__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_7NSIndex_8__reduce_cython__(((struct __pyx_obj_4borg_9hashindex_NSIndex *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex_8__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__9, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.NSIndex.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_11__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_7NSIndex_11__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_7NSIndex_10__setstate_cython__(((struct __pyx_obj_4borg_9hashindex_NSIndex *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_7NSIndex_10__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSIndex *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__10, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.NSIndex.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":173 * cdef int exhausted * * def __cinit__(self, key_size): # <<<<<<<<<<<<<< * self.key = NULL * self.key_size = key_size */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_13NSKeyIterator_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static int __pyx_pw_4borg_9hashindex_13NSKeyIterator_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key_size = 0; int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key_size,0}; PyObject* values[1] = {0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key_size)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(0, 173, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 1) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); } __pyx_v_key_size = values[0]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 1, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 173, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.NSKeyIterator.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return -1; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_13NSKeyIterator___cinit__(((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)__pyx_v_self), __pyx_v_key_size); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_13NSKeyIterator___cinit__(struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self, PyObject *__pyx_v_key_size) { int __pyx_r; __Pyx_RefNannyDeclarations int __pyx_t_1; __Pyx_RefNannySetupContext("__cinit__", 0); /* "borg/hashindex.pyx":174 * * def __cinit__(self, key_size): * self.key = NULL # <<<<<<<<<<<<<< * self.key_size = key_size * self.exhausted = 0 */ __pyx_v_self->key = NULL; /* "borg/hashindex.pyx":175 * def __cinit__(self, key_size): * self.key = NULL * self.key_size = key_size # <<<<<<<<<<<<<< * self.exhausted = 0 * */ __pyx_t_1 = __Pyx_PyInt_As_int(__pyx_v_key_size); if (unlikely((__pyx_t_1 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 175, __pyx_L1_error) __pyx_v_self->key_size = __pyx_t_1; /* "borg/hashindex.pyx":176 * self.key = NULL * self.key_size = key_size * self.exhausted = 0 # <<<<<<<<<<<<<< * * def __iter__(self): */ __pyx_v_self->exhausted = 0; /* "borg/hashindex.pyx":173 * cdef int exhausted * * def __cinit__(self, key_size): # <<<<<<<<<<<<<< * self.key = NULL * self.key_size = key_size */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_AddTraceback("borg.hashindex.NSKeyIterator.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":178 * self.exhausted = 0 * * def __iter__(self): # <<<<<<<<<<<<<< * return self * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_3__iter__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_3__iter__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__iter__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_13NSKeyIterator_2__iter__(((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_2__iter__(struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__iter__", 0); /* "borg/hashindex.pyx":179 * * def __iter__(self): * return self # <<<<<<<<<<<<<< * * def __next__(self): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(((PyObject *)__pyx_v_self)); __pyx_r = ((PyObject *)__pyx_v_self); goto __pyx_L0; /* "borg/hashindex.pyx":178 * self.exhausted = 0 * * def __iter__(self): # <<<<<<<<<<<<<< * return self * */ /* function exit code */ __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":181 * return self * * def __next__(self): # <<<<<<<<<<<<<< * if self.exhausted: * raise StopIteration */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_5__next__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_5__next__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__next__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_13NSKeyIterator_4__next__(((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_4__next__(struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self) { uint32_t *__pyx_v_value; uint32_t __pyx_v_segment; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("__next__", 0); /* "borg/hashindex.pyx":182 * * def __next__(self): * if self.exhausted: # <<<<<<<<<<<<<< * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) */ __pyx_t_1 = (__pyx_v_self->exhausted != 0); if (unlikely(__pyx_t_1)) { /* "borg/hashindex.pyx":183 * def __next__(self): * if self.exhausted: * raise StopIteration # <<<<<<<<<<<<<< * self.key = hashindex_next_key(self.index, self.key) * if not self.key: */ __Pyx_Raise(__pyx_builtin_StopIteration, 0, 0, 0); __PYX_ERR(0, 183, __pyx_L1_error) /* "borg/hashindex.pyx":182 * * def __next__(self): * if self.exhausted: # <<<<<<<<<<<<<< * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) */ } /* "borg/hashindex.pyx":184 * if self.exhausted: * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) # <<<<<<<<<<<<<< * if not self.key: * self.exhausted = 1 */ __pyx_v_self->key = hashindex_next_key(__pyx_v_self->index, ((char *)__pyx_v_self->key)); /* "borg/hashindex.pyx":185 * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) * if not self.key: # <<<<<<<<<<<<<< * self.exhausted = 1 * raise StopIteration */ __pyx_t_1 = ((!(__pyx_v_self->key != 0)) != 0); if (unlikely(__pyx_t_1)) { /* "borg/hashindex.pyx":186 * self.key = hashindex_next_key(self.index, self.key) * if not self.key: * self.exhausted = 1 # <<<<<<<<<<<<<< * raise StopIteration * cdef uint32_t *value = (self.key + self.key_size) */ __pyx_v_self->exhausted = 1; /* "borg/hashindex.pyx":187 * if not self.key: * self.exhausted = 1 * raise StopIteration # <<<<<<<<<<<<<< * cdef uint32_t *value = (self.key + self.key_size) * cdef uint32_t segment = _le32toh(value[0]) */ __Pyx_Raise(__pyx_builtin_StopIteration, 0, 0, 0); __PYX_ERR(0, 187, __pyx_L1_error) /* "borg/hashindex.pyx":185 * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) * if not self.key: # <<<<<<<<<<<<<< * self.exhausted = 1 * raise StopIteration */ } /* "borg/hashindex.pyx":188 * self.exhausted = 1 * raise StopIteration * cdef uint32_t *value = (self.key + self.key_size) # <<<<<<<<<<<<<< * cdef uint32_t segment = _le32toh(value[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" */ __pyx_v_value = ((uint32_t *)(__pyx_v_self->key + __pyx_v_self->key_size)); /* "borg/hashindex.pyx":189 * raise StopIteration * cdef uint32_t *value = (self.key + self.key_size) * cdef uint32_t segment = _le32toh(value[0]) # <<<<<<<<<<<<<< * assert segment <= _MAX_VALUE, "maximum number of segments reached" * return (self.key)[:self.key_size], (segment, _le32toh(value[1])) */ __pyx_v_segment = _le32toh((__pyx_v_value[0])); /* "borg/hashindex.pyx":190 * cdef uint32_t *value = (self.key + self.key_size) * cdef uint32_t segment = _le32toh(value[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" # <<<<<<<<<<<<<< * return (self.key)[:self.key_size], (segment, _le32toh(value[1])) * */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_segment <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_maximum_number_of_segments_reach); __PYX_ERR(0, 190, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":191 * cdef uint32_t segment = _le32toh(value[0]) * assert segment <= _MAX_VALUE, "maximum number of segments reached" * return (self.key)[:self.key_size], (segment, _le32toh(value[1])) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyBytes_FromStringAndSize(((char *)__pyx_v_self->key) + 0, __pyx_v_self->key_size - 0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 191, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_From_uint32_t(__pyx_v_segment); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 191, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_value[1]))); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 191, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = PyTuple_New(2); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 191, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_3); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_4); __pyx_t_3 = 0; __pyx_t_4 = 0; __pyx_t_4 = PyTuple_New(2); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 191, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_4, 0, __pyx_t_2); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_4, 1, __pyx_t_5); __pyx_t_2 = 0; __pyx_t_5 = 0; __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":181 * return self * * def __next__(self): # <<<<<<<<<<<<<< * if self.exhausted: * raise StopIteration */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.hashindex.NSKeyIterator.__next__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_7__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_7__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_13NSKeyIterator_6__reduce_cython__(((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_6__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__11, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.NSKeyIterator.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_9__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_13NSKeyIterator_9__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_13NSKeyIterator_8__setstate_cython__(((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_13NSKeyIterator_8__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_NSKeyIterator *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__12, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.NSKeyIterator.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":213 * value_size = 12 * * def __getitem__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * data = hashindex_get(self.index, key) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_1__getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_1__getitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__getitem__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex___getitem__(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex___getitem__(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key) { uint32_t *__pyx_v_data; uint32_t __pyx_v_refcount; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; __Pyx_RefNannySetupContext("__getitem__", 0); /* "borg/hashindex.pyx":214 * * def __getitem__(self, key): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * data = hashindex_get(self.index, key) * if not data: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 214, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 214, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":215 * def __getitem__(self, key): * assert len(key) == self.key_size * data = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if not data: * raise KeyError(key) */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 215, __pyx_L1_error) __pyx_v_data = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_2))); /* "borg/hashindex.pyx":216 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) */ __pyx_t_3 = ((!(__pyx_v_data != 0)) != 0); if (unlikely(__pyx_t_3)) { /* "borg/hashindex.pyx":217 * data = hashindex_get(self.index, key) * if not data: * raise KeyError(key) # <<<<<<<<<<<<<< * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" */ __pyx_t_4 = __Pyx_PyObject_CallOneArg(__pyx_builtin_KeyError, __pyx_v_key); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 217, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 217, __pyx_L1_error) /* "borg/hashindex.pyx":216 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) */ } /* "borg/hashindex.pyx":218 * if not data: * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) # <<<<<<<<<<<<<< * assert refcount <= _MAX_VALUE, "invalid reference count" * return refcount, _le32toh(data[1]), _le32toh(data[2]) */ __pyx_v_refcount = _le32toh((__pyx_v_data[0])); /* "borg/hashindex.pyx":219 * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * return refcount, _le32toh(data[1]), _le32toh(data[2]) * */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 219, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":220 * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" * return refcount, _le32toh(data[1]), _le32toh(data[2]) # <<<<<<<<<<<<<< * * def __setitem__(self, key, value): */ __Pyx_XDECREF(__pyx_r); __pyx_t_4 = __Pyx_PyInt_From_uint32_t(__pyx_v_refcount); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 220, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[1]))); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 220, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[2]))); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 220, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = PyTuple_New(3); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 220, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_7, 0, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_7, 1, __pyx_t_5); __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_7, 2, __pyx_t_6); __pyx_t_4 = 0; __pyx_t_5 = 0; __pyx_t_6 = 0; __pyx_r = __pyx_t_7; __pyx_t_7 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":213 * value_size = 12 * * def __getitem__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * data = hashindex_get(self.index, key) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.__getitem__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":222 * return refcount, _le32toh(data[1]), _le32toh(data[2]) * * def __setitem__(self, key, value): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * cdef uint32_t[3] data */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_10ChunkIndex_3__setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value); /*proto*/ static int __pyx_pw_4borg_9hashindex_10ChunkIndex_3__setitem__(PyObject *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value) { int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setitem__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_2__setitem__(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key), ((PyObject *)__pyx_v_value)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_10ChunkIndex_2__setitem__(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_value) { uint32_t __pyx_v_data[3]; uint32_t __pyx_v_refcount; int __pyx_r; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; PyObject *__pyx_t_2 = NULL; uint32_t __pyx_t_3; char *__pyx_t_4; int __pyx_t_5; __Pyx_RefNannySetupContext("__setitem__", 0); /* "borg/hashindex.pyx":223 * * def __setitem__(self, key, value): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * cdef uint32_t[3] data * cdef uint32_t refcount = value[0] */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 223, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 223, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":225 * assert len(key) == self.key_size * cdef uint32_t[3] data * cdef uint32_t refcount = value[0] # <<<<<<<<<<<<<< * assert refcount <= _MAX_VALUE, "invalid reference count" * data[0] = _htole32(refcount) */ __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_value, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 225, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 225, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_refcount = __pyx_t_3; /* "borg/hashindex.pyx":226 * cdef uint32_t[3] data * cdef uint32_t refcount = value[0] * assert refcount <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * data[0] = _htole32(refcount) * data[1] = _htole32(value[1]) */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 226, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":227 * cdef uint32_t refcount = value[0] * assert refcount <= _MAX_VALUE, "invalid reference count" * data[0] = _htole32(refcount) # <<<<<<<<<<<<<< * data[1] = _htole32(value[1]) * data[2] = _htole32(value[2]) */ (__pyx_v_data[0]) = _htole32(__pyx_v_refcount); /* "borg/hashindex.pyx":228 * assert refcount <= _MAX_VALUE, "invalid reference count" * data[0] = _htole32(refcount) * data[1] = _htole32(value[1]) # <<<<<<<<<<<<<< * data[2] = _htole32(value[2]) * if not hashindex_set(self.index, key, data): */ __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_value, 1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 228, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 228, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; (__pyx_v_data[1]) = _htole32(__pyx_t_3); /* "borg/hashindex.pyx":229 * data[0] = _htole32(refcount) * data[1] = _htole32(value[1]) * data[2] = _htole32(value[2]) # <<<<<<<<<<<<<< * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') */ __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_value, 2, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 229, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_As_uint32_t(__pyx_t_2); if (unlikely((__pyx_t_3 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 229, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; (__pyx_v_data[2]) = _htole32(__pyx_t_3); /* "borg/hashindex.pyx":230 * data[1] = _htole32(value[1]) * data[2] = _htole32(value[2]) * if not hashindex_set(self.index, key, data): # <<<<<<<<<<<<<< * raise Exception('hashindex_set failed') * */ __pyx_t_4 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_4) && PyErr_Occurred())) __PYX_ERR(0, 230, __pyx_L1_error) __pyx_t_5 = ((!(hashindex_set(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_4), __pyx_v_data) != 0)) != 0); if (unlikely(__pyx_t_5)) { /* "borg/hashindex.pyx":231 * data[2] = _htole32(value[2]) * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') # <<<<<<<<<<<<<< * * def __contains__(self, key): */ __pyx_t_2 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__8, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 231, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_Raise(__pyx_t_2, 0, 0, 0); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __PYX_ERR(0, 231, __pyx_L1_error) /* "borg/hashindex.pyx":230 * data[1] = _htole32(value[1]) * data[2] = _htole32(value[2]) * if not hashindex_set(self.index, key, data): # <<<<<<<<<<<<<< * raise Exception('hashindex_set failed') * */ } /* "borg/hashindex.pyx":222 * return refcount, _le32toh(data[1]), _le32toh(data[2]) * * def __setitem__(self, key, value): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * cdef uint32_t[3] data */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.__setitem__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":233 * raise Exception('hashindex_set failed') * * def __contains__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * data = hashindex_get(self.index, key) */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_10ChunkIndex_5__contains__(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static int __pyx_pw_4borg_9hashindex_10ChunkIndex_5__contains__(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__contains__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_4__contains__(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_10ChunkIndex_4__contains__(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key) { uint32_t *__pyx_v_data; int __pyx_r; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; __Pyx_RefNannySetupContext("__contains__", 0); /* "borg/hashindex.pyx":234 * * def __contains__(self, key): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * data = hashindex_get(self.index, key) * if data != NULL: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 234, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 234, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":235 * def __contains__(self, key): * assert len(key) == self.key_size * data = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if data != NULL: * assert _le32toh(data[0]) <= _MAX_VALUE, "invalid reference count" */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 235, __pyx_L1_error) __pyx_v_data = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_2))); /* "borg/hashindex.pyx":236 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if data != NULL: # <<<<<<<<<<<<<< * assert _le32toh(data[0]) <= _MAX_VALUE, "invalid reference count" * return data != NULL */ __pyx_t_3 = ((__pyx_v_data != NULL) != 0); if (__pyx_t_3) { /* "borg/hashindex.pyx":237 * data = hashindex_get(self.index, key) * if data != NULL: * assert _le32toh(data[0]) <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * return data != NULL * */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((_le32toh((__pyx_v_data[0])) <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 237, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":236 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if data != NULL: # <<<<<<<<<<<<<< * assert _le32toh(data[0]) <= _MAX_VALUE, "invalid reference count" * return data != NULL */ } /* "borg/hashindex.pyx":238 * if data != NULL: * assert _le32toh(data[0]) <= _MAX_VALUE, "invalid reference count" * return data != NULL # <<<<<<<<<<<<<< * * def incref(self, key): */ __pyx_r = (__pyx_v_data != NULL); goto __pyx_L0; /* "borg/hashindex.pyx":233 * raise Exception('hashindex_set failed') * * def __contains__(self, key): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * data = hashindex_get(self.index, key) */ /* function exit code */ __pyx_L1_error:; __Pyx_AddTraceback("borg.hashindex.ChunkIndex.__contains__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":240 * return data != NULL * * def incref(self, key): # <<<<<<<<<<<<<< * """Increase refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_7incref(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static char __pyx_doc_4borg_9hashindex_10ChunkIndex_6incref[] = "Increase refcount for 'key', return (refcount, size, csize)"; static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_7incref(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("incref (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_6incref(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_6incref(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key) { uint32_t *__pyx_v_data; uint32_t __pyx_v_refcount; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; __Pyx_RefNannySetupContext("incref", 0); /* "borg/hashindex.pyx":242 * def incref(self, key): * """Increase refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size # <<<<<<<<<<<<<< * data = hashindex_get(self.index, key) * if not data: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 242, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 242, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":243 * """Increase refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size * data = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if not data: * raise KeyError(key) */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 243, __pyx_L1_error) __pyx_v_data = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_2))); /* "borg/hashindex.pyx":244 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) */ __pyx_t_3 = ((!(__pyx_v_data != 0)) != 0); if (unlikely(__pyx_t_3)) { /* "borg/hashindex.pyx":245 * data = hashindex_get(self.index, key) * if not data: * raise KeyError(key) # <<<<<<<<<<<<<< * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" */ __pyx_t_4 = __Pyx_PyObject_CallOneArg(__pyx_builtin_KeyError, __pyx_v_key); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 245, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 245, __pyx_L1_error) /* "borg/hashindex.pyx":244 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) */ } /* "borg/hashindex.pyx":246 * if not data: * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) # <<<<<<<<<<<<<< * assert refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: */ __pyx_v_refcount = _le32toh((__pyx_v_data[0])); /* "borg/hashindex.pyx":247 * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * if refcount != _MAX_VALUE: * refcount += 1 */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 247, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":248 * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: # <<<<<<<<<<<<<< * refcount += 1 * data[0] = _htole32(refcount) */ __pyx_t_3 = ((__pyx_v_refcount != __pyx_v_4borg_9hashindex__MAX_VALUE) != 0); if (__pyx_t_3) { /* "borg/hashindex.pyx":249 * assert refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: * refcount += 1 # <<<<<<<<<<<<<< * data[0] = _htole32(refcount) * return refcount, _le32toh(data[1]), _le32toh(data[2]) */ __pyx_v_refcount = (__pyx_v_refcount + 1); /* "borg/hashindex.pyx":248 * cdef uint32_t refcount = _le32toh(data[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: # <<<<<<<<<<<<<< * refcount += 1 * data[0] = _htole32(refcount) */ } /* "borg/hashindex.pyx":250 * if refcount != _MAX_VALUE: * refcount += 1 * data[0] = _htole32(refcount) # <<<<<<<<<<<<<< * return refcount, _le32toh(data[1]), _le32toh(data[2]) * */ (__pyx_v_data[0]) = _htole32(__pyx_v_refcount); /* "borg/hashindex.pyx":251 * refcount += 1 * data[0] = _htole32(refcount) * return refcount, _le32toh(data[1]), _le32toh(data[2]) # <<<<<<<<<<<<<< * * def decref(self, key): */ __Pyx_XDECREF(__pyx_r); __pyx_t_4 = __Pyx_PyInt_From_uint32_t(__pyx_v_refcount); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 251, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[1]))); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 251, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[2]))); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 251, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = PyTuple_New(3); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 251, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_7, 0, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_7, 1, __pyx_t_5); __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_7, 2, __pyx_t_6); __pyx_t_4 = 0; __pyx_t_5 = 0; __pyx_t_6 = 0; __pyx_r = __pyx_t_7; __pyx_t_7 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":240 * return data != NULL * * def incref(self, key): # <<<<<<<<<<<<<< * """Increase refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.incref", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":253 * return refcount, _le32toh(data[1]), _le32toh(data[2]) * * def decref(self, key): # <<<<<<<<<<<<<< * """Decrease refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_9decref(PyObject *__pyx_v_self, PyObject *__pyx_v_key); /*proto*/ static char __pyx_doc_4borg_9hashindex_10ChunkIndex_8decref[] = "Decrease refcount for 'key', return (refcount, size, csize)"; static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_9decref(PyObject *__pyx_v_self, PyObject *__pyx_v_key) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("decref (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_8decref(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((PyObject *)__pyx_v_key)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_8decref(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key) { uint32_t *__pyx_v_data; uint32_t __pyx_v_refcount; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; char *__pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; __Pyx_RefNannySetupContext("decref", 0); /* "borg/hashindex.pyx":255 * def decref(self, key): * """Decrease refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size # <<<<<<<<<<<<<< * data = hashindex_get(self.index, key) * if not data: */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 255, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 255, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":256 * """Decrease refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size * data = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if not data: * raise KeyError(key) */ __pyx_t_2 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_2) && PyErr_Occurred())) __PYX_ERR(0, 256, __pyx_L1_error) __pyx_v_data = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_2))); /* "borg/hashindex.pyx":257 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) */ __pyx_t_3 = ((!(__pyx_v_data != 0)) != 0); if (unlikely(__pyx_t_3)) { /* "borg/hashindex.pyx":258 * data = hashindex_get(self.index, key) * if not data: * raise KeyError(key) # <<<<<<<<<<<<<< * cdef uint32_t refcount = _le32toh(data[0]) * # Never decrease a reference count of zero */ __pyx_t_4 = __Pyx_PyObject_CallOneArg(__pyx_builtin_KeyError, __pyx_v_key); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 258, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_Raise(__pyx_t_4, 0, 0, 0); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __PYX_ERR(0, 258, __pyx_L1_error) /* "borg/hashindex.pyx":257 * assert len(key) == self.key_size * data = hashindex_get(self.index, key) * if not data: # <<<<<<<<<<<<<< * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) */ } /* "borg/hashindex.pyx":259 * if not data: * raise KeyError(key) * cdef uint32_t refcount = _le32toh(data[0]) # <<<<<<<<<<<<<< * # Never decrease a reference count of zero * assert 0 < refcount <= _MAX_VALUE, "invalid reference count" */ __pyx_v_refcount = _le32toh((__pyx_v_data[0])); /* "borg/hashindex.pyx":261 * cdef uint32_t refcount = _le32toh(data[0]) * # Never decrease a reference count of zero * assert 0 < refcount <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * if refcount != _MAX_VALUE: * refcount -= 1 */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_3 = (0 < __pyx_v_refcount); if (__pyx_t_3) { __pyx_t_3 = (__pyx_v_refcount <= __pyx_v_4borg_9hashindex__MAX_VALUE); } if (unlikely(!(__pyx_t_3 != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 261, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":262 * # Never decrease a reference count of zero * assert 0 < refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: # <<<<<<<<<<<<<< * refcount -= 1 * data[0] = _htole32(refcount) */ __pyx_t_3 = ((__pyx_v_refcount != __pyx_v_4borg_9hashindex__MAX_VALUE) != 0); if (__pyx_t_3) { /* "borg/hashindex.pyx":263 * assert 0 < refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: * refcount -= 1 # <<<<<<<<<<<<<< * data[0] = _htole32(refcount) * return refcount, _le32toh(data[1]), _le32toh(data[2]) */ __pyx_v_refcount = (__pyx_v_refcount - 1); /* "borg/hashindex.pyx":262 * # Never decrease a reference count of zero * assert 0 < refcount <= _MAX_VALUE, "invalid reference count" * if refcount != _MAX_VALUE: # <<<<<<<<<<<<<< * refcount -= 1 * data[0] = _htole32(refcount) */ } /* "borg/hashindex.pyx":264 * if refcount != _MAX_VALUE: * refcount -= 1 * data[0] = _htole32(refcount) # <<<<<<<<<<<<<< * return refcount, _le32toh(data[1]), _le32toh(data[2]) * */ (__pyx_v_data[0]) = _htole32(__pyx_v_refcount); /* "borg/hashindex.pyx":265 * refcount -= 1 * data[0] = _htole32(refcount) * return refcount, _le32toh(data[1]), _le32toh(data[2]) # <<<<<<<<<<<<<< * * def iteritems(self, marker=None): */ __Pyx_XDECREF(__pyx_r); __pyx_t_4 = __Pyx_PyInt_From_uint32_t(__pyx_v_refcount); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 265, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[1]))); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 265, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_data[2]))); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 265, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = PyTuple_New(3); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 265, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_7, 0, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_7, 1, __pyx_t_5); __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_7, 2, __pyx_t_6); __pyx_t_4 = 0; __pyx_t_5 = 0; __pyx_t_6 = 0; __pyx_r = __pyx_t_7; __pyx_t_7 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":253 * return refcount, _le32toh(data[1]), _le32toh(data[2]) * * def decref(self, key): # <<<<<<<<<<<<<< * """Decrease refcount for 'key', return (refcount, size, csize)""" * assert len(key) == self.key_size */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.decref", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":267 * return refcount, _le32toh(data[1]), _le32toh(data[2]) * * def iteritems(self, marker=None): # <<<<<<<<<<<<<< * cdef const void *key * iter = ChunkKeyIterator(self.key_size) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_11iteritems(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_11iteritems(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_marker = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("iteritems (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_marker,0}; PyObject* values[1] = {0}; values[0] = ((PyObject *)Py_None); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_marker); if (value) { values[0] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "iteritems") < 0)) __PYX_ERR(0, 267, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_marker = values[0]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("iteritems", 0, 0, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 267, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.ChunkIndex.iteritems", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_10iteritems(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), __pyx_v_marker); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_10iteritems(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_marker) { void const *__pyx_v_key; struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_iter = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; HashIndex *__pyx_t_3; int __pyx_t_4; char *__pyx_t_5; int __pyx_t_6; __Pyx_RefNannySetupContext("iteritems", 0); /* "borg/hashindex.pyx":269 * def iteritems(self, marker=None): * cdef const void *key * iter = ChunkKeyIterator(self.key_size) # <<<<<<<<<<<<<< * iter.idx = self * iter.index = self.index */ __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->__pyx_base.key_size); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 269, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_CallOneArg(((PyObject *)__pyx_ptype_4borg_9hashindex_ChunkKeyIterator), __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 269, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_iter = ((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)__pyx_t_2); __pyx_t_2 = 0; /* "borg/hashindex.pyx":270 * cdef const void *key * iter = ChunkKeyIterator(self.key_size) * iter.idx = self # <<<<<<<<<<<<<< * iter.index = self.index * if marker: */ __Pyx_INCREF(((PyObject *)__pyx_v_self)); __Pyx_GIVEREF(((PyObject *)__pyx_v_self)); __Pyx_GOTREF(__pyx_v_iter->idx); __Pyx_DECREF(((PyObject *)__pyx_v_iter->idx)); __pyx_v_iter->idx = __pyx_v_self; /* "borg/hashindex.pyx":271 * iter = ChunkKeyIterator(self.key_size) * iter.idx = self * iter.index = self.index # <<<<<<<<<<<<<< * if marker: * key = hashindex_get(self.index, marker) */ __pyx_t_3 = __pyx_v_self->__pyx_base.index; __pyx_v_iter->index = __pyx_t_3; /* "borg/hashindex.pyx":272 * iter.idx = self * iter.index = self.index * if marker: # <<<<<<<<<<<<<< * key = hashindex_get(self.index, marker) * if marker is None: */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_marker); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 272, __pyx_L1_error) if (__pyx_t_4) { /* "borg/hashindex.pyx":273 * iter.index = self.index * if marker: * key = hashindex_get(self.index, marker) # <<<<<<<<<<<<<< * if marker is None: * raise IndexError */ __pyx_t_5 = __Pyx_PyObject_AsWritableString(__pyx_v_marker); if (unlikely((!__pyx_t_5) && PyErr_Occurred())) __PYX_ERR(0, 273, __pyx_L1_error) __pyx_v_key = hashindex_get(__pyx_v_self->__pyx_base.index, ((char *)__pyx_t_5)); /* "borg/hashindex.pyx":274 * if marker: * key = hashindex_get(self.index, marker) * if marker is None: # <<<<<<<<<<<<<< * raise IndexError * iter.key = key - self.key_size */ __pyx_t_4 = (__pyx_v_marker == Py_None); __pyx_t_6 = (__pyx_t_4 != 0); if (unlikely(__pyx_t_6)) { /* "borg/hashindex.pyx":275 * key = hashindex_get(self.index, marker) * if marker is None: * raise IndexError # <<<<<<<<<<<<<< * iter.key = key - self.key_size * return iter */ __Pyx_Raise(__pyx_builtin_IndexError, 0, 0, 0); __PYX_ERR(0, 275, __pyx_L1_error) /* "borg/hashindex.pyx":274 * if marker: * key = hashindex_get(self.index, marker) * if marker is None: # <<<<<<<<<<<<<< * raise IndexError * iter.key = key - self.key_size */ } /* "borg/hashindex.pyx":276 * if marker is None: * raise IndexError * iter.key = key - self.key_size # <<<<<<<<<<<<<< * return iter * */ __pyx_v_iter->key = (__pyx_v_key - __pyx_v_self->__pyx_base.key_size); /* "borg/hashindex.pyx":272 * iter.idx = self * iter.index = self.index * if marker: # <<<<<<<<<<<<<< * key = hashindex_get(self.index, marker) * if marker is None: */ } /* "borg/hashindex.pyx":277 * raise IndexError * iter.key = key - self.key_size * return iter # <<<<<<<<<<<<<< * * def summarize(self): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(((PyObject *)__pyx_v_iter)); __pyx_r = ((PyObject *)__pyx_v_iter); goto __pyx_L0; /* "borg/hashindex.pyx":267 * return refcount, _le32toh(data[1]), _le32toh(data[2]) * * def iteritems(self, marker=None): # <<<<<<<<<<<<<< * cdef const void *key * iter = ChunkKeyIterator(self.key_size) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.iteritems", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF((PyObject *)__pyx_v_iter); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":279 * return iter * * def summarize(self): # <<<<<<<<<<<<<< * cdef uint64_t size = 0, csize = 0, unique_size = 0, unique_csize = 0, chunks = 0, unique_chunks = 0 * cdef uint32_t *values */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_13summarize(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_13summarize(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("summarize (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_12summarize(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_12summarize(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self) { uint64_t __pyx_v_size; uint64_t __pyx_v_csize; uint64_t __pyx_v_unique_size; uint64_t __pyx_v_unique_csize; uint64_t __pyx_v_chunks; uint64_t __pyx_v_unique_chunks; uint32_t *__pyx_v_values; uint32_t __pyx_v_refcount; void *__pyx_v_key; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; __Pyx_RefNannySetupContext("summarize", 0); /* "borg/hashindex.pyx":280 * * def summarize(self): * cdef uint64_t size = 0, csize = 0, unique_size = 0, unique_csize = 0, chunks = 0, unique_chunks = 0 # <<<<<<<<<<<<<< * cdef uint32_t *values * cdef uint32_t refcount */ __pyx_v_size = 0; __pyx_v_csize = 0; __pyx_v_unique_size = 0; __pyx_v_unique_csize = 0; __pyx_v_chunks = 0; __pyx_v_unique_chunks = 0; /* "borg/hashindex.pyx":283 * cdef uint32_t *values * cdef uint32_t refcount * cdef void *key = NULL # <<<<<<<<<<<<<< * * while True: */ __pyx_v_key = NULL; /* "borg/hashindex.pyx":285 * cdef void *key = NULL * * while True: # <<<<<<<<<<<<<< * key = hashindex_next_key(self.index, key) * if not key: */ while (1) { /* "borg/hashindex.pyx":286 * * while True: * key = hashindex_next_key(self.index, key) # <<<<<<<<<<<<<< * if not key: * break */ __pyx_v_key = hashindex_next_key(__pyx_v_self->__pyx_base.index, __pyx_v_key); /* "borg/hashindex.pyx":287 * while True: * key = hashindex_next_key(self.index, key) * if not key: # <<<<<<<<<<<<<< * break * unique_chunks += 1 */ __pyx_t_1 = ((!(__pyx_v_key != 0)) != 0); if (__pyx_t_1) { /* "borg/hashindex.pyx":288 * key = hashindex_next_key(self.index, key) * if not key: * break # <<<<<<<<<<<<<< * unique_chunks += 1 * values = (key + self.key_size) */ goto __pyx_L4_break; /* "borg/hashindex.pyx":287 * while True: * key = hashindex_next_key(self.index, key) * if not key: # <<<<<<<<<<<<<< * break * unique_chunks += 1 */ } /* "borg/hashindex.pyx":289 * if not key: * break * unique_chunks += 1 # <<<<<<<<<<<<<< * values = (key + self.key_size) * refcount = _le32toh(values[0]) */ __pyx_v_unique_chunks = (__pyx_v_unique_chunks + 1); /* "borg/hashindex.pyx":290 * break * unique_chunks += 1 * values = (key + self.key_size) # <<<<<<<<<<<<<< * refcount = _le32toh(values[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" */ __pyx_v_values = ((uint32_t *)(__pyx_v_key + __pyx_v_self->__pyx_base.key_size)); /* "borg/hashindex.pyx":291 * unique_chunks += 1 * values = (key + self.key_size) * refcount = _le32toh(values[0]) # <<<<<<<<<<<<<< * assert refcount <= _MAX_VALUE, "invalid reference count" * chunks += refcount */ __pyx_v_refcount = _le32toh((__pyx_v_values[0])); /* "borg/hashindex.pyx":292 * values = (key + self.key_size) * refcount = _le32toh(values[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * chunks += refcount * unique_size += _le32toh(values[1]) */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 292, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":293 * refcount = _le32toh(values[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" * chunks += refcount # <<<<<<<<<<<<<< * unique_size += _le32toh(values[1]) * unique_csize += _le32toh(values[2]) */ __pyx_v_chunks = (__pyx_v_chunks + __pyx_v_refcount); /* "borg/hashindex.pyx":294 * assert refcount <= _MAX_VALUE, "invalid reference count" * chunks += refcount * unique_size += _le32toh(values[1]) # <<<<<<<<<<<<<< * unique_csize += _le32toh(values[2]) * size += _le32toh(values[1]) * _le32toh(values[0]) */ __pyx_v_unique_size = (__pyx_v_unique_size + _le32toh((__pyx_v_values[1]))); /* "borg/hashindex.pyx":295 * chunks += refcount * unique_size += _le32toh(values[1]) * unique_csize += _le32toh(values[2]) # <<<<<<<<<<<<<< * size += _le32toh(values[1]) * _le32toh(values[0]) * csize += _le32toh(values[2]) * _le32toh(values[0]) */ __pyx_v_unique_csize = (__pyx_v_unique_csize + _le32toh((__pyx_v_values[2]))); /* "borg/hashindex.pyx":296 * unique_size += _le32toh(values[1]) * unique_csize += _le32toh(values[2]) * size += _le32toh(values[1]) * _le32toh(values[0]) # <<<<<<<<<<<<<< * csize += _le32toh(values[2]) * _le32toh(values[0]) * */ __pyx_v_size = (__pyx_v_size + (((uint64_t)_le32toh((__pyx_v_values[1]))) * _le32toh((__pyx_v_values[0])))); /* "borg/hashindex.pyx":297 * unique_csize += _le32toh(values[2]) * size += _le32toh(values[1]) * _le32toh(values[0]) * csize += _le32toh(values[2]) * _le32toh(values[0]) # <<<<<<<<<<<<<< * * return size, csize, unique_size, unique_csize, unique_chunks, chunks */ __pyx_v_csize = (__pyx_v_csize + (((uint64_t)_le32toh((__pyx_v_values[2]))) * _le32toh((__pyx_v_values[0])))); } __pyx_L4_break:; /* "borg/hashindex.pyx":299 * csize += _le32toh(values[2]) * _le32toh(values[0]) * * return size, csize, unique_size, unique_csize, unique_chunks, chunks # <<<<<<<<<<<<<< * * def add(self, key, refs, size, csize): */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyInt_From_uint64_t(__pyx_v_size); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_From_uint64_t(__pyx_v_csize); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyInt_From_uint64_t(__pyx_v_unique_size); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyInt_From_uint64_t(__pyx_v_unique_csize); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyInt_From_uint64_t(__pyx_v_unique_chunks); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_7 = __Pyx_PyInt_From_uint64_t(__pyx_v_chunks); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = PyTuple_New(6); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 299, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_8, 0, __pyx_t_2); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_8, 1, __pyx_t_3); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_8, 2, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_8, 3, __pyx_t_5); __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_8, 4, __pyx_t_6); __Pyx_GIVEREF(__pyx_t_7); PyTuple_SET_ITEM(__pyx_t_8, 5, __pyx_t_7); __pyx_t_2 = 0; __pyx_t_3 = 0; __pyx_t_4 = 0; __pyx_t_5 = 0; __pyx_t_6 = 0; __pyx_t_7 = 0; __pyx_r = __pyx_t_8; __pyx_t_8 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":279 * return iter * * def summarize(self): # <<<<<<<<<<<<<< * cdef uint64_t size = 0, csize = 0, unique_size = 0, unique_csize = 0, chunks = 0, unique_chunks = 0 * cdef uint32_t *values */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.summarize", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":301 * return size, csize, unique_size, unique_csize, unique_chunks, chunks * * def add(self, key, refs, size, csize): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * cdef uint32_t[3] data */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_15add(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_15add(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key = 0; PyObject *__pyx_v_refs = 0; PyObject *__pyx_v_size = 0; PyObject *__pyx_v_csize = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("add (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key,&__pyx_n_s_refs,&__pyx_n_s_size,&__pyx_n_s_csize,0}; PyObject* values[4] = {0,0,0,0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_refs)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("add", 1, 4, 4, 1); __PYX_ERR(0, 301, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_size)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("add", 1, 4, 4, 2); __PYX_ERR(0, 301, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (likely((values[3] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_csize)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("add", 1, 4, 4, 3); __PYX_ERR(0, 301, __pyx_L3_error) } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "add") < 0)) __PYX_ERR(0, 301, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 4) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[3] = PyTuple_GET_ITEM(__pyx_args, 3); } __pyx_v_key = values[0]; __pyx_v_refs = values[1]; __pyx_v_size = values[2]; __pyx_v_csize = values[3]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("add", 1, 4, 4, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 301, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.ChunkIndex.add", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_14add(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), __pyx_v_key, __pyx_v_refs, __pyx_v_size, __pyx_v_csize); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_14add(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, PyObject *__pyx_v_key, PyObject *__pyx_v_refs, PyObject *__pyx_v_size, PyObject *__pyx_v_csize) { uint32_t __pyx_v_data[3]; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations Py_ssize_t __pyx_t_1; uint32_t __pyx_t_2; char *__pyx_t_3; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("add", 0); /* "borg/hashindex.pyx":302 * * def add(self, key, refs, size, csize): * assert len(key) == self.key_size # <<<<<<<<<<<<<< * cdef uint32_t[3] data * data[0] = _htole32(refs) */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = PyObject_Length(__pyx_v_key); if (unlikely(__pyx_t_1 == ((Py_ssize_t)-1))) __PYX_ERR(0, 302, __pyx_L1_error) if (unlikely(!((__pyx_t_1 == __pyx_v_self->__pyx_base.key_size) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 302, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":304 * assert len(key) == self.key_size * cdef uint32_t[3] data * data[0] = _htole32(refs) # <<<<<<<<<<<<<< * data[1] = _htole32(size) * data[2] = _htole32(csize) */ __pyx_t_2 = __Pyx_PyInt_As_uint32_t(__pyx_v_refs); if (unlikely((__pyx_t_2 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 304, __pyx_L1_error) (__pyx_v_data[0]) = _htole32(__pyx_t_2); /* "borg/hashindex.pyx":305 * cdef uint32_t[3] data * data[0] = _htole32(refs) * data[1] = _htole32(size) # <<<<<<<<<<<<<< * data[2] = _htole32(csize) * self._add( key, data) */ __pyx_t_2 = __Pyx_PyInt_As_uint32_t(__pyx_v_size); if (unlikely((__pyx_t_2 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 305, __pyx_L1_error) (__pyx_v_data[1]) = _htole32(__pyx_t_2); /* "borg/hashindex.pyx":306 * data[0] = _htole32(refs) * data[1] = _htole32(size) * data[2] = _htole32(csize) # <<<<<<<<<<<<<< * self._add( key, data) * */ __pyx_t_2 = __Pyx_PyInt_As_uint32_t(__pyx_v_csize); if (unlikely((__pyx_t_2 == ((uint32_t)-1)) && PyErr_Occurred())) __PYX_ERR(0, 306, __pyx_L1_error) (__pyx_v_data[2]) = _htole32(__pyx_t_2); /* "borg/hashindex.pyx":307 * data[1] = _htole32(size) * data[2] = _htole32(csize) * self._add( key, data) # <<<<<<<<<<<<<< * * cdef _add(self, void *key, uint32_t *data): */ __pyx_t_3 = __Pyx_PyObject_AsWritableString(__pyx_v_key); if (unlikely((!__pyx_t_3) && PyErr_Occurred())) __PYX_ERR(0, 307, __pyx_L1_error) __pyx_t_4 = ((struct __pyx_vtabstruct_4borg_9hashindex_ChunkIndex *)__pyx_v_self->__pyx_vtab)->_add(__pyx_v_self, ((char *)__pyx_t_3), __pyx_v_data); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 307, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/hashindex.pyx":301 * return size, csize, unique_size, unique_csize, unique_chunks, chunks * * def add(self, key, refs, size, csize): # <<<<<<<<<<<<<< * assert len(key) == self.key_size * cdef uint32_t[3] data */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.add", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":309 * self._add( key, data) * * cdef _add(self, void *key, uint32_t *data): # <<<<<<<<<<<<<< * cdef uint64_t refcount1, refcount2, result64 * values = hashindex_get(self.index, key) */ static PyObject *__pyx_f_4borg_9hashindex_10ChunkIndex__add(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, void *__pyx_v_key, uint32_t *__pyx_v_data) { uint64_t __pyx_v_refcount1; uint64_t __pyx_v_refcount2; uint64_t __pyx_v_result64; uint32_t *__pyx_v_values; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; uint32_t __pyx_t_2; uint64_t __pyx_t_3; uint64_t __pyx_t_4; PyObject *__pyx_t_5 = NULL; __Pyx_RefNannySetupContext("_add", 0); /* "borg/hashindex.pyx":311 * cdef _add(self, void *key, uint32_t *data): * cdef uint64_t refcount1, refcount2, result64 * values = hashindex_get(self.index, key) # <<<<<<<<<<<<<< * if values: * refcount1 = _le32toh(values[0]) */ __pyx_v_values = ((uint32_t *)hashindex_get(__pyx_v_self->__pyx_base.index, __pyx_v_key)); /* "borg/hashindex.pyx":312 * cdef uint64_t refcount1, refcount2, result64 * values = hashindex_get(self.index, key) * if values: # <<<<<<<<<<<<<< * refcount1 = _le32toh(values[0]) * refcount2 = _le32toh(data[0]) */ __pyx_t_1 = (__pyx_v_values != 0); if (__pyx_t_1) { /* "borg/hashindex.pyx":313 * values = hashindex_get(self.index, key) * if values: * refcount1 = _le32toh(values[0]) # <<<<<<<<<<<<<< * refcount2 = _le32toh(data[0]) * assert refcount1 <= _MAX_VALUE, "invalid reference count" */ __pyx_v_refcount1 = _le32toh((__pyx_v_values[0])); /* "borg/hashindex.pyx":314 * if values: * refcount1 = _le32toh(values[0]) * refcount2 = _le32toh(data[0]) # <<<<<<<<<<<<<< * assert refcount1 <= _MAX_VALUE, "invalid reference count" * assert refcount2 <= _MAX_VALUE, "invalid reference count" */ __pyx_v_refcount2 = _le32toh((__pyx_v_data[0])); /* "borg/hashindex.pyx":315 * refcount1 = _le32toh(values[0]) * refcount2 = _le32toh(data[0]) * assert refcount1 <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * assert refcount2 <= _MAX_VALUE, "invalid reference count" * result64 = refcount1 + refcount2 */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount1 <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 315, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":316 * refcount2 = _le32toh(data[0]) * assert refcount1 <= _MAX_VALUE, "invalid reference count" * assert refcount2 <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * result64 = refcount1 + refcount2 * values[0] = _htole32(min(result64, _MAX_VALUE)) */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount2 <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 316, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":317 * assert refcount1 <= _MAX_VALUE, "invalid reference count" * assert refcount2 <= _MAX_VALUE, "invalid reference count" * result64 = refcount1 + refcount2 # <<<<<<<<<<<<<< * values[0] = _htole32(min(result64, _MAX_VALUE)) * else: */ __pyx_v_result64 = (__pyx_v_refcount1 + __pyx_v_refcount2); /* "borg/hashindex.pyx":318 * assert refcount2 <= _MAX_VALUE, "invalid reference count" * result64 = refcount1 + refcount2 * values[0] = _htole32(min(result64, _MAX_VALUE)) # <<<<<<<<<<<<<< * else: * if not hashindex_set(self.index, key, data): */ __pyx_t_2 = __pyx_v_4borg_9hashindex__MAX_VALUE; __pyx_t_3 = __pyx_v_result64; if (((__pyx_t_2 < __pyx_t_3) != 0)) { __pyx_t_4 = __pyx_t_2; } else { __pyx_t_4 = __pyx_t_3; } (__pyx_v_values[0]) = _htole32(__pyx_t_4); /* "borg/hashindex.pyx":312 * cdef uint64_t refcount1, refcount2, result64 * values = hashindex_get(self.index, key) * if values: # <<<<<<<<<<<<<< * refcount1 = _le32toh(values[0]) * refcount2 = _le32toh(data[0]) */ goto __pyx_L3; } /* "borg/hashindex.pyx":320 * values[0] = _htole32(min(result64, _MAX_VALUE)) * else: * if not hashindex_set(self.index, key, data): # <<<<<<<<<<<<<< * raise Exception('hashindex_set failed') * */ /*else*/ { __pyx_t_1 = ((!(hashindex_set(__pyx_v_self->__pyx_base.index, __pyx_v_key, __pyx_v_data) != 0)) != 0); if (unlikely(__pyx_t_1)) { /* "borg/hashindex.pyx":321 * else: * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') # <<<<<<<<<<<<<< * * def merge(self, ChunkIndex other): */ __pyx_t_5 = __Pyx_PyObject_Call(((PyObject *)(&((PyTypeObject*)PyExc_Exception)[0])), __pyx_tuple__8, NULL); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 321, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_Raise(__pyx_t_5, 0, 0, 0); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __PYX_ERR(0, 321, __pyx_L1_error) /* "borg/hashindex.pyx":320 * values[0] = _htole32(min(result64, _MAX_VALUE)) * else: * if not hashindex_set(self.index, key, data): # <<<<<<<<<<<<<< * raise Exception('hashindex_set failed') * */ } } __pyx_L3:; /* "borg/hashindex.pyx":309 * self._add( key, data) * * cdef _add(self, void *key, uint32_t *data): # <<<<<<<<<<<<<< * cdef uint64_t refcount1, refcount2, result64 * values = hashindex_get(self.index, key) */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.hashindex.ChunkIndex._add", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":323 * raise Exception('hashindex_set failed') * * def merge(self, ChunkIndex other): # <<<<<<<<<<<<<< * cdef void *key = NULL * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_17merge(PyObject *__pyx_v_self, PyObject *__pyx_v_other); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_17merge(PyObject *__pyx_v_self, PyObject *__pyx_v_other) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("merge (wrapper)", 0); if (unlikely(!__Pyx_ArgTypeTest(((PyObject *)__pyx_v_other), __pyx_ptype_4borg_9hashindex_ChunkIndex, 1, "other", 0))) __PYX_ERR(0, 323, __pyx_L1_error) __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_16merge(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_other)); /* function exit code */ goto __pyx_L0; __pyx_L1_error:; __pyx_r = NULL; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_16merge(struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_other) { void *__pyx_v_key; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannySetupContext("merge", 0); /* "borg/hashindex.pyx":324 * * def merge(self, ChunkIndex other): * cdef void *key = NULL # <<<<<<<<<<<<<< * * while True: */ __pyx_v_key = NULL; /* "borg/hashindex.pyx":326 * cdef void *key = NULL * * while True: # <<<<<<<<<<<<<< * key = hashindex_next_key(other.index, key) * if not key: */ while (1) { /* "borg/hashindex.pyx":327 * * while True: * key = hashindex_next_key(other.index, key) # <<<<<<<<<<<<<< * if not key: * break */ __pyx_v_key = hashindex_next_key(__pyx_v_other->__pyx_base.index, __pyx_v_key); /* "borg/hashindex.pyx":328 * while True: * key = hashindex_next_key(other.index, key) * if not key: # <<<<<<<<<<<<<< * break * self._add(key, (key + self.key_size)) */ __pyx_t_1 = ((!(__pyx_v_key != 0)) != 0); if (__pyx_t_1) { /* "borg/hashindex.pyx":329 * key = hashindex_next_key(other.index, key) * if not key: * break # <<<<<<<<<<<<<< * self._add(key, (key + self.key_size)) * */ goto __pyx_L4_break; /* "borg/hashindex.pyx":328 * while True: * key = hashindex_next_key(other.index, key) * if not key: # <<<<<<<<<<<<<< * break * self._add(key, (key + self.key_size)) */ } /* "borg/hashindex.pyx":330 * if not key: * break * self._add(key, (key + self.key_size)) # <<<<<<<<<<<<<< * * */ __pyx_t_2 = ((struct __pyx_vtabstruct_4borg_9hashindex_ChunkIndex *)__pyx_v_self->__pyx_vtab)->_add(__pyx_v_self, __pyx_v_key, ((uint32_t *)(__pyx_v_key + __pyx_v_self->__pyx_base.key_size))); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 330, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; } __pyx_L4_break:; /* "borg/hashindex.pyx":323 * raise Exception('hashindex_set failed') * * def merge(self, ChunkIndex other): # <<<<<<<<<<<<<< * cdef void *key = NULL * */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.merge", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_19__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_19__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_18__reduce_cython__(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_18__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__13, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_21__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_10ChunkIndex_21__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_10ChunkIndex_20__setstate_cython__(((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_10ChunkIndex_20__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkIndex *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__14, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.ChunkIndex.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":340 * cdef int exhausted * * def __cinit__(self, key_size): # <<<<<<<<<<<<<< * self.key = NULL * self.key_size = key_size */ /* Python wrapper */ static int __pyx_pw_4borg_9hashindex_16ChunkKeyIterator_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static int __pyx_pw_4borg_9hashindex_16ChunkKeyIterator_1__cinit__(PyObject *__pyx_v_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_key_size = 0; int __pyx_r; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__cinit__ (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_key_size,0}; PyObject* values[1] = {0}; if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_key_size)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "__cinit__") < 0)) __PYX_ERR(0, 340, __pyx_L3_error) } } else if (PyTuple_GET_SIZE(__pyx_args) != 1) { goto __pyx_L5_argtuple_error; } else { values[0] = PyTuple_GET_ITEM(__pyx_args, 0); } __pyx_v_key_size = values[0]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("__cinit__", 1, 1, 1, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 340, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.hashindex.ChunkKeyIterator.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return -1; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_9hashindex_16ChunkKeyIterator___cinit__(((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)__pyx_v_self), __pyx_v_key_size); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static int __pyx_pf_4borg_9hashindex_16ChunkKeyIterator___cinit__(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self, PyObject *__pyx_v_key_size) { int __pyx_r; __Pyx_RefNannyDeclarations int __pyx_t_1; __Pyx_RefNannySetupContext("__cinit__", 0); /* "borg/hashindex.pyx":341 * * def __cinit__(self, key_size): * self.key = NULL # <<<<<<<<<<<<<< * self.key_size = key_size * self.exhausted = 0 */ __pyx_v_self->key = NULL; /* "borg/hashindex.pyx":342 * def __cinit__(self, key_size): * self.key = NULL * self.key_size = key_size # <<<<<<<<<<<<<< * self.exhausted = 0 * */ __pyx_t_1 = __Pyx_PyInt_As_int(__pyx_v_key_size); if (unlikely((__pyx_t_1 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 342, __pyx_L1_error) __pyx_v_self->key_size = __pyx_t_1; /* "borg/hashindex.pyx":343 * self.key = NULL * self.key_size = key_size * self.exhausted = 0 # <<<<<<<<<<<<<< * * def __iter__(self): */ __pyx_v_self->exhausted = 0; /* "borg/hashindex.pyx":340 * cdef int exhausted * * def __cinit__(self, key_size): # <<<<<<<<<<<<<< * self.key = NULL * self.key_size = key_size */ /* function exit code */ __pyx_r = 0; goto __pyx_L0; __pyx_L1_error:; __Pyx_AddTraceback("borg.hashindex.ChunkKeyIterator.__cinit__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = -1; __pyx_L0:; __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":345 * self.exhausted = 0 * * def __iter__(self): # <<<<<<<<<<<<<< * return self * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_3__iter__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_3__iter__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__iter__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_16ChunkKeyIterator_2__iter__(((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_2__iter__(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__iter__", 0); /* "borg/hashindex.pyx":346 * * def __iter__(self): * return self # <<<<<<<<<<<<<< * * def __next__(self): */ __Pyx_XDECREF(__pyx_r); __Pyx_INCREF(((PyObject *)__pyx_v_self)); __pyx_r = ((PyObject *)__pyx_v_self); goto __pyx_L0; /* "borg/hashindex.pyx":345 * self.exhausted = 0 * * def __iter__(self): # <<<<<<<<<<<<<< * return self * */ /* function exit code */ __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/hashindex.pyx":348 * return self * * def __next__(self): # <<<<<<<<<<<<<< * if self.exhausted: * raise StopIteration */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_5__next__(PyObject *__pyx_v_self); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_5__next__(PyObject *__pyx_v_self) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__next__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_16ChunkKeyIterator_4__next__(((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_4__next__(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self) { uint32_t *__pyx_v_value; uint32_t __pyx_v_refcount; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations int __pyx_t_1; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; __Pyx_RefNannySetupContext("__next__", 0); /* "borg/hashindex.pyx":349 * * def __next__(self): * if self.exhausted: # <<<<<<<<<<<<<< * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) */ __pyx_t_1 = (__pyx_v_self->exhausted != 0); if (unlikely(__pyx_t_1)) { /* "borg/hashindex.pyx":350 * def __next__(self): * if self.exhausted: * raise StopIteration # <<<<<<<<<<<<<< * self.key = hashindex_next_key(self.index, self.key) * if not self.key: */ __Pyx_Raise(__pyx_builtin_StopIteration, 0, 0, 0); __PYX_ERR(0, 350, __pyx_L1_error) /* "borg/hashindex.pyx":349 * * def __next__(self): * if self.exhausted: # <<<<<<<<<<<<<< * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) */ } /* "borg/hashindex.pyx":351 * if self.exhausted: * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) # <<<<<<<<<<<<<< * if not self.key: * self.exhausted = 1 */ __pyx_v_self->key = hashindex_next_key(__pyx_v_self->index, ((char *)__pyx_v_self->key)); /* "borg/hashindex.pyx":352 * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) * if not self.key: # <<<<<<<<<<<<<< * self.exhausted = 1 * raise StopIteration */ __pyx_t_1 = ((!(__pyx_v_self->key != 0)) != 0); if (unlikely(__pyx_t_1)) { /* "borg/hashindex.pyx":353 * self.key = hashindex_next_key(self.index, self.key) * if not self.key: * self.exhausted = 1 # <<<<<<<<<<<<<< * raise StopIteration * cdef uint32_t *value = (self.key + self.key_size) */ __pyx_v_self->exhausted = 1; /* "borg/hashindex.pyx":354 * if not self.key: * self.exhausted = 1 * raise StopIteration # <<<<<<<<<<<<<< * cdef uint32_t *value = (self.key + self.key_size) * cdef uint32_t refcount = _le32toh(value[0]) */ __Pyx_Raise(__pyx_builtin_StopIteration, 0, 0, 0); __PYX_ERR(0, 354, __pyx_L1_error) /* "borg/hashindex.pyx":352 * raise StopIteration * self.key = hashindex_next_key(self.index, self.key) * if not self.key: # <<<<<<<<<<<<<< * self.exhausted = 1 * raise StopIteration */ } /* "borg/hashindex.pyx":355 * self.exhausted = 1 * raise StopIteration * cdef uint32_t *value = (self.key + self.key_size) # <<<<<<<<<<<<<< * cdef uint32_t refcount = _le32toh(value[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" */ __pyx_v_value = ((uint32_t *)(__pyx_v_self->key + __pyx_v_self->key_size)); /* "borg/hashindex.pyx":356 * raise StopIteration * cdef uint32_t *value = (self.key + self.key_size) * cdef uint32_t refcount = _le32toh(value[0]) # <<<<<<<<<<<<<< * assert refcount <= _MAX_VALUE, "invalid reference count" * return (self.key)[:self.key_size], (refcount, _le32toh(value[1]), _le32toh(value[2])) */ __pyx_v_refcount = _le32toh((__pyx_v_value[0])); /* "borg/hashindex.pyx":357 * cdef uint32_t *value = (self.key + self.key_size) * cdef uint32_t refcount = _le32toh(value[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" # <<<<<<<<<<<<<< * return (self.key)[:self.key_size], (refcount, _le32toh(value[1]), _le32toh(value[2])) */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__pyx_v_refcount <= __pyx_v_4borg_9hashindex__MAX_VALUE) != 0))) { PyErr_SetObject(PyExc_AssertionError, __pyx_kp_s_invalid_reference_count); __PYX_ERR(0, 357, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":358 * cdef uint32_t refcount = _le32toh(value[0]) * assert refcount <= _MAX_VALUE, "invalid reference count" * return (self.key)[:self.key_size], (refcount, _le32toh(value[1]), _le32toh(value[2])) # <<<<<<<<<<<<<< */ __Pyx_XDECREF(__pyx_r); __pyx_t_2 = __Pyx_PyBytes_FromStringAndSize(((char *)__pyx_v_self->key) + 0, __pyx_v_self->key_size - 0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 358, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyInt_From_uint32_t(__pyx_v_refcount); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 358, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_value[1]))); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 358, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_5 = __Pyx_PyInt_From_uint32_t(_le32toh((__pyx_v_value[2]))); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 358, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = PyTuple_New(3); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 358, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_3); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_6, 1, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_6, 2, __pyx_t_5); __pyx_t_3 = 0; __pyx_t_4 = 0; __pyx_t_5 = 0; __pyx_t_5 = PyTuple_New(2); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 358, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_5, 0, __pyx_t_2); __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_5, 1, __pyx_t_6); __pyx_t_2 = 0; __pyx_t_6 = 0; __pyx_r = __pyx_t_5; __pyx_t_5 = 0; goto __pyx_L0; /* "borg/hashindex.pyx":348 * return self * * def __next__(self): # <<<<<<<<<<<<<< * if self.exhausted: * raise StopIteration */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.hashindex.ChunkKeyIterator.__next__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_7__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_7__reduce_cython__(PyObject *__pyx_v_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__reduce_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_16ChunkKeyIterator_6__reduce_cython__(((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)__pyx_v_self)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_6__reduce_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__reduce_cython__", 0); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__15, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 2, __pyx_L1_error) /* "(tree fragment)":1 * def __reduce_cython__(self): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.ChunkKeyIterator.__reduce_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_9__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state); /*proto*/ static PyObject *__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_9__setstate_cython__(PyObject *__pyx_v_self, PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__setstate_cython__ (wrapper)", 0); __pyx_r = __pyx_pf_4borg_9hashindex_16ChunkKeyIterator_8__setstate_cython__(((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)__pyx_v_self), ((PyObject *)__pyx_v___pyx_state)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_9hashindex_16ChunkKeyIterator_8__setstate_cython__(CYTHON_UNUSED struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *__pyx_v_self, CYTHON_UNUSED PyObject *__pyx_v___pyx_state) { PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; __Pyx_RefNannySetupContext("__setstate_cython__", 0); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_TypeError, __pyx_tuple__16, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_Raise(__pyx_t_1, 0, 0, 0); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __PYX_ERR(1, 4, __pyx_L1_error) /* "(tree fragment)":3 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): # <<<<<<<<<<<<<< * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_AddTraceback("borg.hashindex.ChunkKeyIterator.__setstate_cython__", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_tp_new_4borg_9hashindex_IndexBase(PyTypeObject *t, PyObject *a, PyObject *k) { PyObject *o; if (likely((t->tp_flags & Py_TPFLAGS_IS_ABSTRACT) == 0)) { o = (*t->tp_alloc)(t, 0); } else { o = (PyObject *) PyBaseObject_Type.tp_new(t, __pyx_empty_tuple, 0); } if (unlikely(!o)) return 0; if (unlikely(__pyx_pw_4borg_9hashindex_9IndexBase_1__cinit__(o, a, k) < 0)) goto bad; return o; bad: Py_DECREF(o); o = 0; return NULL; } static void __pyx_tp_dealloc_4borg_9hashindex_IndexBase(PyObject *o) { #if CYTHON_USE_TP_FINALIZE if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && (!PyType_IS_GC(Py_TYPE(o)) || !_PyGC_FINALIZED(o))) { if (PyObject_CallFinalizerFromDealloc(o)) return; } #endif { PyObject *etype, *eval, *etb; PyErr_Fetch(&etype, &eval, &etb); ++Py_REFCNT(o); __pyx_pw_4borg_9hashindex_9IndexBase_3__dealloc__(o); --Py_REFCNT(o); PyErr_Restore(etype, eval, etb); } (*Py_TYPE(o)->tp_free)(o); } static int __pyx_mp_ass_subscript_4borg_9hashindex_IndexBase(PyObject *o, PyObject *i, PyObject *v) { if (v) { PyErr_Format(PyExc_NotImplementedError, "Subscript assignment not supported by %.200s", Py_TYPE(o)->tp_name); return -1; } else { return __pyx_pw_4borg_9hashindex_9IndexBase_13__delitem__(o, i); } } static PyMethodDef __pyx_methods_4borg_9hashindex_IndexBase[] = { {"read", (PyCFunction)__pyx_pw_4borg_9hashindex_9IndexBase_5read, METH_O, 0}, {"write", (PyCFunction)__pyx_pw_4borg_9hashindex_9IndexBase_7write, METH_O, 0}, {"clear", (PyCFunction)__pyx_pw_4borg_9hashindex_9IndexBase_9clear, METH_NOARGS, 0}, {"setdefault", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_9hashindex_9IndexBase_11setdefault, METH_VARARGS|METH_KEYWORDS, 0}, {"get", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_9hashindex_9IndexBase_15get, METH_VARARGS|METH_KEYWORDS, 0}, {"pop", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_9hashindex_9IndexBase_17pop, METH_VARARGS|METH_KEYWORDS, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_9IndexBase_21__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_9IndexBase_23__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PySequenceMethods __pyx_tp_as_sequence_IndexBase = { __pyx_pw_4borg_9hashindex_9IndexBase_19__len__, /*sq_length*/ 0, /*sq_concat*/ 0, /*sq_repeat*/ 0, /*sq_item*/ 0, /*sq_slice*/ 0, /*sq_ass_item*/ 0, /*sq_ass_slice*/ 0, /*sq_contains*/ 0, /*sq_inplace_concat*/ 0, /*sq_inplace_repeat*/ }; static PyMappingMethods __pyx_tp_as_mapping_IndexBase = { __pyx_pw_4borg_9hashindex_9IndexBase_19__len__, /*mp_length*/ 0, /*mp_subscript*/ __pyx_mp_ass_subscript_4borg_9hashindex_IndexBase, /*mp_ass_subscript*/ }; static PyTypeObject __pyx_type_4borg_9hashindex_IndexBase = { PyVarObject_HEAD_INIT(0, 0) "borg.hashindex.IndexBase", /*tp_name*/ sizeof(struct __pyx_obj_4borg_9hashindex_IndexBase), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_9hashindex_IndexBase, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ &__pyx_tp_as_sequence_IndexBase, /*tp_as_sequence*/ &__pyx_tp_as_mapping_IndexBase, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE, /*tp_flags*/ 0, /*tp_doc*/ 0, /*tp_traverse*/ 0, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ 0, /*tp_iter*/ 0, /*tp_iternext*/ __pyx_methods_4borg_9hashindex_IndexBase, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_9hashindex_IndexBase, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyObject *__pyx_tp_new_4borg_9hashindex_NSIndex(PyTypeObject *t, PyObject *a, PyObject *k) { PyObject *o = __pyx_tp_new_4borg_9hashindex_IndexBase(t, a, k); if (unlikely(!o)) return 0; return o; } static PyObject *__pyx_sq_item_4borg_9hashindex_NSIndex(PyObject *o, Py_ssize_t i) { PyObject *r; PyObject *x = PyInt_FromSsize_t(i); if(!x) return 0; r = Py_TYPE(o)->tp_as_mapping->mp_subscript(o, x); Py_DECREF(x); return r; } static int __pyx_mp_ass_subscript_4borg_9hashindex_NSIndex(PyObject *o, PyObject *i, PyObject *v) { if (v) { return __pyx_pw_4borg_9hashindex_7NSIndex_3__setitem__(o, i, v); } else { if (__pyx_ptype_4borg_9hashindex_IndexBase->tp_as_mapping && __pyx_ptype_4borg_9hashindex_IndexBase->tp_as_mapping->mp_ass_subscript) return __pyx_ptype_4borg_9hashindex_IndexBase->tp_as_mapping->mp_ass_subscript(o, i, v); PyErr_Format(PyExc_NotImplementedError, "Subscript deletion not supported by %.200s", Py_TYPE(o)->tp_name); return -1; } } static PyMethodDef __pyx_methods_4borg_9hashindex_NSIndex[] = { {"iteritems", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_9hashindex_7NSIndex_7iteritems, METH_VARARGS|METH_KEYWORDS, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_7NSIndex_9__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_7NSIndex_11__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PySequenceMethods __pyx_tp_as_sequence_NSIndex = { #if CYTHON_COMPILING_IN_PYPY __pyx_pw_4borg_9hashindex_9IndexBase_19__len__, /*sq_length*/ #else 0, /*sq_length*/ #endif 0, /*sq_concat*/ 0, /*sq_repeat*/ __pyx_sq_item_4borg_9hashindex_NSIndex, /*sq_item*/ 0, /*sq_slice*/ 0, /*sq_ass_item*/ 0, /*sq_ass_slice*/ __pyx_pw_4borg_9hashindex_7NSIndex_5__contains__, /*sq_contains*/ 0, /*sq_inplace_concat*/ 0, /*sq_inplace_repeat*/ }; static PyMappingMethods __pyx_tp_as_mapping_NSIndex = { #if CYTHON_COMPILING_IN_PYPY __pyx_pw_4borg_9hashindex_9IndexBase_19__len__, /*mp_length*/ #else 0, /*mp_length*/ #endif __pyx_pw_4borg_9hashindex_7NSIndex_1__getitem__, /*mp_subscript*/ __pyx_mp_ass_subscript_4borg_9hashindex_NSIndex, /*mp_ass_subscript*/ }; static PyTypeObject __pyx_type_4borg_9hashindex_NSIndex = { PyVarObject_HEAD_INIT(0, 0) "borg.hashindex.NSIndex", /*tp_name*/ sizeof(struct __pyx_obj_4borg_9hashindex_NSIndex), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_9hashindex_IndexBase, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ &__pyx_tp_as_sequence_NSIndex, /*tp_as_sequence*/ &__pyx_tp_as_mapping_NSIndex, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE, /*tp_flags*/ 0, /*tp_doc*/ 0, /*tp_traverse*/ 0, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ 0, /*tp_iter*/ 0, /*tp_iternext*/ __pyx_methods_4borg_9hashindex_NSIndex, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_9hashindex_NSIndex, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyObject *__pyx_tp_new_4borg_9hashindex_NSKeyIterator(PyTypeObject *t, PyObject *a, PyObject *k) { struct __pyx_obj_4borg_9hashindex_NSKeyIterator *p; PyObject *o; if (likely((t->tp_flags & Py_TPFLAGS_IS_ABSTRACT) == 0)) { o = (*t->tp_alloc)(t, 0); } else { o = (PyObject *) PyBaseObject_Type.tp_new(t, __pyx_empty_tuple, 0); } if (unlikely(!o)) return 0; p = ((struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)o); p->idx = ((struct __pyx_obj_4borg_9hashindex_NSIndex *)Py_None); Py_INCREF(Py_None); if (unlikely(__pyx_pw_4borg_9hashindex_13NSKeyIterator_1__cinit__(o, a, k) < 0)) goto bad; return o; bad: Py_DECREF(o); o = 0; return NULL; } static void __pyx_tp_dealloc_4borg_9hashindex_NSKeyIterator(PyObject *o) { struct __pyx_obj_4borg_9hashindex_NSKeyIterator *p = (struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)o; #if CYTHON_USE_TP_FINALIZE if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && !_PyGC_FINALIZED(o)) { if (PyObject_CallFinalizerFromDealloc(o)) return; } #endif PyObject_GC_UnTrack(o); Py_CLEAR(p->idx); (*Py_TYPE(o)->tp_free)(o); } static int __pyx_tp_traverse_4borg_9hashindex_NSKeyIterator(PyObject *o, visitproc v, void *a) { int e; struct __pyx_obj_4borg_9hashindex_NSKeyIterator *p = (struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)o; if (p->idx) { e = (*v)(((PyObject *)p->idx), a); if (e) return e; } return 0; } static int __pyx_tp_clear_4borg_9hashindex_NSKeyIterator(PyObject *o) { PyObject* tmp; struct __pyx_obj_4borg_9hashindex_NSKeyIterator *p = (struct __pyx_obj_4borg_9hashindex_NSKeyIterator *)o; tmp = ((PyObject*)p->idx); p->idx = ((struct __pyx_obj_4borg_9hashindex_NSIndex *)Py_None); Py_INCREF(Py_None); Py_XDECREF(tmp); return 0; } static PyObject *__pyx_specialmethod___pyx_pw_4borg_9hashindex_13NSKeyIterator_5__next__(PyObject *self, CYTHON_UNUSED PyObject *arg) {return __pyx_pw_4borg_9hashindex_13NSKeyIterator_5__next__(self);} static PyMethodDef __pyx_methods_4borg_9hashindex_NSKeyIterator[] = { {"__next__", (PyCFunction)__pyx_specialmethod___pyx_pw_4borg_9hashindex_13NSKeyIterator_5__next__, METH_NOARGS|METH_COEXIST, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_13NSKeyIterator_7__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_13NSKeyIterator_9__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PyTypeObject __pyx_type_4borg_9hashindex_NSKeyIterator = { PyVarObject_HEAD_INIT(0, 0) "borg.hashindex.NSKeyIterator", /*tp_name*/ sizeof(struct __pyx_obj_4borg_9hashindex_NSKeyIterator), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_9hashindex_NSKeyIterator, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE|Py_TPFLAGS_HAVE_GC, /*tp_flags*/ 0, /*tp_doc*/ __pyx_tp_traverse_4borg_9hashindex_NSKeyIterator, /*tp_traverse*/ __pyx_tp_clear_4borg_9hashindex_NSKeyIterator, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ __pyx_pw_4borg_9hashindex_13NSKeyIterator_3__iter__, /*tp_iter*/ __pyx_pw_4borg_9hashindex_13NSKeyIterator_5__next__, /*tp_iternext*/ __pyx_methods_4borg_9hashindex_NSKeyIterator, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_9hashindex_NSKeyIterator, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static struct __pyx_vtabstruct_4borg_9hashindex_ChunkIndex __pyx_vtable_4borg_9hashindex_ChunkIndex; static PyObject *__pyx_tp_new_4borg_9hashindex_ChunkIndex(PyTypeObject *t, PyObject *a, PyObject *k) { struct __pyx_obj_4borg_9hashindex_ChunkIndex *p; PyObject *o = __pyx_tp_new_4borg_9hashindex_IndexBase(t, a, k); if (unlikely(!o)) return 0; p = ((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)o); p->__pyx_vtab = __pyx_vtabptr_4borg_9hashindex_ChunkIndex; return o; } static PyObject *__pyx_sq_item_4borg_9hashindex_ChunkIndex(PyObject *o, Py_ssize_t i) { PyObject *r; PyObject *x = PyInt_FromSsize_t(i); if(!x) return 0; r = Py_TYPE(o)->tp_as_mapping->mp_subscript(o, x); Py_DECREF(x); return r; } static int __pyx_mp_ass_subscript_4borg_9hashindex_ChunkIndex(PyObject *o, PyObject *i, PyObject *v) { if (v) { return __pyx_pw_4borg_9hashindex_10ChunkIndex_3__setitem__(o, i, v); } else { if (__pyx_ptype_4borg_9hashindex_IndexBase->tp_as_mapping && __pyx_ptype_4borg_9hashindex_IndexBase->tp_as_mapping->mp_ass_subscript) return __pyx_ptype_4borg_9hashindex_IndexBase->tp_as_mapping->mp_ass_subscript(o, i, v); PyErr_Format(PyExc_NotImplementedError, "Subscript deletion not supported by %.200s", Py_TYPE(o)->tp_name); return -1; } } static PyMethodDef __pyx_methods_4borg_9hashindex_ChunkIndex[] = { {"incref", (PyCFunction)__pyx_pw_4borg_9hashindex_10ChunkIndex_7incref, METH_O, __pyx_doc_4borg_9hashindex_10ChunkIndex_6incref}, {"decref", (PyCFunction)__pyx_pw_4borg_9hashindex_10ChunkIndex_9decref, METH_O, __pyx_doc_4borg_9hashindex_10ChunkIndex_8decref}, {"iteritems", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_9hashindex_10ChunkIndex_11iteritems, METH_VARARGS|METH_KEYWORDS, 0}, {"summarize", (PyCFunction)__pyx_pw_4borg_9hashindex_10ChunkIndex_13summarize, METH_NOARGS, 0}, {"add", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_9hashindex_10ChunkIndex_15add, METH_VARARGS|METH_KEYWORDS, 0}, {"merge", (PyCFunction)__pyx_pw_4borg_9hashindex_10ChunkIndex_17merge, METH_O, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_10ChunkIndex_19__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_10ChunkIndex_21__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PySequenceMethods __pyx_tp_as_sequence_ChunkIndex = { #if CYTHON_COMPILING_IN_PYPY __pyx_pw_4borg_9hashindex_9IndexBase_19__len__, /*sq_length*/ #else 0, /*sq_length*/ #endif 0, /*sq_concat*/ 0, /*sq_repeat*/ __pyx_sq_item_4borg_9hashindex_ChunkIndex, /*sq_item*/ 0, /*sq_slice*/ 0, /*sq_ass_item*/ 0, /*sq_ass_slice*/ __pyx_pw_4borg_9hashindex_10ChunkIndex_5__contains__, /*sq_contains*/ 0, /*sq_inplace_concat*/ 0, /*sq_inplace_repeat*/ }; static PyMappingMethods __pyx_tp_as_mapping_ChunkIndex = { #if CYTHON_COMPILING_IN_PYPY __pyx_pw_4borg_9hashindex_9IndexBase_19__len__, /*mp_length*/ #else 0, /*mp_length*/ #endif __pyx_pw_4borg_9hashindex_10ChunkIndex_1__getitem__, /*mp_subscript*/ __pyx_mp_ass_subscript_4borg_9hashindex_ChunkIndex, /*mp_ass_subscript*/ }; static PyTypeObject __pyx_type_4borg_9hashindex_ChunkIndex = { PyVarObject_HEAD_INIT(0, 0) "borg.hashindex.ChunkIndex", /*tp_name*/ sizeof(struct __pyx_obj_4borg_9hashindex_ChunkIndex), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_9hashindex_IndexBase, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ &__pyx_tp_as_sequence_ChunkIndex, /*tp_as_sequence*/ &__pyx_tp_as_mapping_ChunkIndex, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE, /*tp_flags*/ "\n Mapping of 32 byte keys to (refcount, size, csize), which are all 32-bit unsigned.\n\n The reference count cannot overflow. If an overflow would occur, the refcount\n is fixed to MAX_VALUE and will neither increase nor decrease by incref(), decref()\n or add().\n\n Prior signed 32-bit overflow is handled correctly for most cases: All values\n from UINT32_MAX (2**32-1, inclusive) to MAX_VALUE (exclusive) are reserved and either\n cause silent data loss (-1, -2) or will raise an AssertionError when accessed.\n Other values are handled correctly. Note that previously the refcount could also reach\n 0 by *increasing* it.\n\n Assigning refcounts in this reserved range is an invalid operation and raises AssertionError.\n ", /*tp_doc*/ 0, /*tp_traverse*/ 0, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ 0, /*tp_iter*/ 0, /*tp_iternext*/ __pyx_methods_4borg_9hashindex_ChunkIndex, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_9hashindex_ChunkIndex, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyObject *__pyx_tp_new_4borg_9hashindex_ChunkKeyIterator(PyTypeObject *t, PyObject *a, PyObject *k) { struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *p; PyObject *o; if (likely((t->tp_flags & Py_TPFLAGS_IS_ABSTRACT) == 0)) { o = (*t->tp_alloc)(t, 0); } else { o = (PyObject *) PyBaseObject_Type.tp_new(t, __pyx_empty_tuple, 0); } if (unlikely(!o)) return 0; p = ((struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)o); p->idx = ((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)Py_None); Py_INCREF(Py_None); if (unlikely(__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_1__cinit__(o, a, k) < 0)) goto bad; return o; bad: Py_DECREF(o); o = 0; return NULL; } static void __pyx_tp_dealloc_4borg_9hashindex_ChunkKeyIterator(PyObject *o) { struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *p = (struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)o; #if CYTHON_USE_TP_FINALIZE if (unlikely(PyType_HasFeature(Py_TYPE(o), Py_TPFLAGS_HAVE_FINALIZE) && Py_TYPE(o)->tp_finalize) && !_PyGC_FINALIZED(o)) { if (PyObject_CallFinalizerFromDealloc(o)) return; } #endif PyObject_GC_UnTrack(o); Py_CLEAR(p->idx); (*Py_TYPE(o)->tp_free)(o); } static int __pyx_tp_traverse_4borg_9hashindex_ChunkKeyIterator(PyObject *o, visitproc v, void *a) { int e; struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *p = (struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)o; if (p->idx) { e = (*v)(((PyObject *)p->idx), a); if (e) return e; } return 0; } static int __pyx_tp_clear_4borg_9hashindex_ChunkKeyIterator(PyObject *o) { PyObject* tmp; struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *p = (struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator *)o; tmp = ((PyObject*)p->idx); p->idx = ((struct __pyx_obj_4borg_9hashindex_ChunkIndex *)Py_None); Py_INCREF(Py_None); Py_XDECREF(tmp); return 0; } static PyObject *__pyx_specialmethod___pyx_pw_4borg_9hashindex_16ChunkKeyIterator_5__next__(PyObject *self, CYTHON_UNUSED PyObject *arg) {return __pyx_pw_4borg_9hashindex_16ChunkKeyIterator_5__next__(self);} static PyMethodDef __pyx_methods_4borg_9hashindex_ChunkKeyIterator[] = { {"__next__", (PyCFunction)__pyx_specialmethod___pyx_pw_4borg_9hashindex_16ChunkKeyIterator_5__next__, METH_NOARGS|METH_COEXIST, 0}, {"__reduce_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_7__reduce_cython__, METH_NOARGS, 0}, {"__setstate_cython__", (PyCFunction)__pyx_pw_4borg_9hashindex_16ChunkKeyIterator_9__setstate_cython__, METH_O, 0}, {0, 0, 0, 0} }; static PyTypeObject __pyx_type_4borg_9hashindex_ChunkKeyIterator = { PyVarObject_HEAD_INIT(0, 0) "borg.hashindex.ChunkKeyIterator", /*tp_name*/ sizeof(struct __pyx_obj_4borg_9hashindex_ChunkKeyIterator), /*tp_basicsize*/ 0, /*tp_itemsize*/ __pyx_tp_dealloc_4borg_9hashindex_ChunkKeyIterator, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ #if PY_MAJOR_VERSION < 3 0, /*tp_compare*/ #endif #if PY_MAJOR_VERSION >= 3 0, /*tp_as_async*/ #endif 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash*/ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT|Py_TPFLAGS_HAVE_VERSION_TAG|Py_TPFLAGS_CHECKTYPES|Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_BASETYPE|Py_TPFLAGS_HAVE_GC, /*tp_flags*/ 0, /*tp_doc*/ __pyx_tp_traverse_4borg_9hashindex_ChunkKeyIterator, /*tp_traverse*/ __pyx_tp_clear_4borg_9hashindex_ChunkKeyIterator, /*tp_clear*/ 0, /*tp_richcompare*/ 0, /*tp_weaklistoffset*/ __pyx_pw_4borg_9hashindex_16ChunkKeyIterator_3__iter__, /*tp_iter*/ __pyx_pw_4borg_9hashindex_16ChunkKeyIterator_5__next__, /*tp_iternext*/ __pyx_methods_4borg_9hashindex_ChunkKeyIterator, /*tp_methods*/ 0, /*tp_members*/ 0, /*tp_getset*/ 0, /*tp_base*/ 0, /*tp_dict*/ 0, /*tp_descr_get*/ 0, /*tp_descr_set*/ 0, /*tp_dictoffset*/ 0, /*tp_init*/ 0, /*tp_alloc*/ __pyx_tp_new_4borg_9hashindex_ChunkKeyIterator, /*tp_new*/ 0, /*tp_free*/ 0, /*tp_is_gc*/ 0, /*tp_bases*/ 0, /*tp_mro*/ 0, /*tp_cache*/ 0, /*tp_subclasses*/ 0, /*tp_weaklist*/ 0, /*tp_del*/ 0, /*tp_version_tag*/ #if PY_VERSION_HEX >= 0x030400a1 0, /*tp_finalize*/ #endif }; static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_hashindex(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_hashindex}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "hashindex", 0, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_n_s_ChunkIndex, __pyx_k_ChunkIndex, sizeof(__pyx_k_ChunkIndex), 0, 0, 1, 1}, {&__pyx_n_s_ChunkKeyIterator, __pyx_k_ChunkKeyIterator, sizeof(__pyx_k_ChunkKeyIterator), 0, 0, 1, 1}, {&__pyx_n_s_IndexError, __pyx_k_IndexError, sizeof(__pyx_k_IndexError), 0, 0, 1, 1}, {&__pyx_n_s_KeyError, __pyx_k_KeyError, sizeof(__pyx_k_KeyError), 0, 0, 1, 1}, {&__pyx_n_s_MAX_LOAD_FACTOR, __pyx_k_MAX_LOAD_FACTOR, sizeof(__pyx_k_MAX_LOAD_FACTOR), 0, 0, 1, 1}, {&__pyx_n_s_MAX_VALUE, __pyx_k_MAX_VALUE, sizeof(__pyx_k_MAX_VALUE), 0, 0, 1, 1}, {&__pyx_n_s_NSIndex, __pyx_k_NSIndex, sizeof(__pyx_k_NSIndex), 0, 0, 1, 1}, {&__pyx_n_s_NSKeyIterator, __pyx_k_NSKeyIterator, sizeof(__pyx_k_NSKeyIterator), 0, 0, 1, 1}, {&__pyx_n_s_StopIteration, __pyx_k_StopIteration, sizeof(__pyx_k_StopIteration), 0, 0, 1, 1}, {&__pyx_n_s_TypeError, __pyx_k_TypeError, sizeof(__pyx_k_TypeError), 0, 0, 1, 1}, {&__pyx_n_s_capacity, __pyx_k_capacity, sizeof(__pyx_k_capacity), 0, 0, 1, 1}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_csize, __pyx_k_csize, sizeof(__pyx_k_csize), 0, 0, 1, 1}, {&__pyx_n_s_default, __pyx_k_default, sizeof(__pyx_k_default), 0, 0, 1, 1}, {&__pyx_n_s_fsencode, __pyx_k_fsencode, sizeof(__pyx_k_fsencode), 0, 0, 1, 1}, {&__pyx_n_s_getstate, __pyx_k_getstate, sizeof(__pyx_k_getstate), 0, 0, 1, 1}, {&__pyx_kp_s_hashindex_delete_failed, __pyx_k_hashindex_delete_failed, sizeof(__pyx_k_hashindex_delete_failed), 0, 0, 1, 0}, {&__pyx_kp_s_hashindex_init_failed, __pyx_k_hashindex_init_failed, sizeof(__pyx_k_hashindex_init_failed), 0, 0, 1, 0}, {&__pyx_kp_s_hashindex_read_failed, __pyx_k_hashindex_read_failed, sizeof(__pyx_k_hashindex_read_failed), 0, 0, 1, 0}, {&__pyx_kp_s_hashindex_set_failed, __pyx_k_hashindex_set_failed, sizeof(__pyx_k_hashindex_set_failed), 0, 0, 1, 0}, {&__pyx_kp_s_hashindex_write_failed, __pyx_k_hashindex_write_failed, sizeof(__pyx_k_hashindex_write_failed), 0, 0, 1, 0}, {&__pyx_n_s_import, __pyx_k_import, sizeof(__pyx_k_import), 0, 0, 1, 1}, {&__pyx_kp_s_invalid_reference_count, __pyx_k_invalid_reference_count, sizeof(__pyx_k_invalid_reference_count), 0, 0, 1, 0}, {&__pyx_n_s_key, __pyx_k_key, sizeof(__pyx_k_key), 0, 0, 1, 1}, {&__pyx_n_s_key_size, __pyx_k_key_size, sizeof(__pyx_k_key_size), 0, 0, 1, 1}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_marker, __pyx_k_marker, sizeof(__pyx_k_marker), 0, 0, 1, 1}, {&__pyx_kp_s_maximum_number_of_segments_reach, __pyx_k_maximum_number_of_segments_reach, sizeof(__pyx_k_maximum_number_of_segments_reach), 0, 0, 1, 0}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_kp_s_no_default___reduce___due_to_non, __pyx_k_no_default___reduce___due_to_non, sizeof(__pyx_k_no_default___reduce___due_to_non), 0, 0, 1, 0}, {&__pyx_n_s_object, __pyx_k_object, sizeof(__pyx_k_object), 0, 0, 1, 1}, {&__pyx_n_s_os, __pyx_k_os, sizeof(__pyx_k_os), 0, 0, 1, 1}, {&__pyx_n_s_path, __pyx_k_path, sizeof(__pyx_k_path), 0, 0, 1, 1}, {&__pyx_n_s_pyx_vtable, __pyx_k_pyx_vtable, sizeof(__pyx_k_pyx_vtable), 0, 0, 1, 1}, {&__pyx_n_s_read, __pyx_k_read, sizeof(__pyx_k_read), 0, 0, 1, 1}, {&__pyx_n_s_reduce, __pyx_k_reduce, sizeof(__pyx_k_reduce), 0, 0, 1, 1}, {&__pyx_n_s_reduce_cython, __pyx_k_reduce_cython, sizeof(__pyx_k_reduce_cython), 0, 0, 1, 1}, {&__pyx_n_s_reduce_ex, __pyx_k_reduce_ex, sizeof(__pyx_k_reduce_ex), 0, 0, 1, 1}, {&__pyx_n_s_refs, __pyx_k_refs, sizeof(__pyx_k_refs), 0, 0, 1, 1}, {&__pyx_n_s_setstate, __pyx_k_setstate, sizeof(__pyx_k_setstate), 0, 0, 1, 1}, {&__pyx_n_s_setstate_cython, __pyx_k_setstate_cython, sizeof(__pyx_k_setstate_cython), 0, 0, 1, 1}, {&__pyx_n_s_size, __pyx_k_size, sizeof(__pyx_k_size), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {&__pyx_n_s_value, __pyx_k_value, sizeof(__pyx_k_value), 0, 0, 1, 1}, {&__pyx_n_s_value_size, __pyx_k_value_size, sizeof(__pyx_k_value_size), 0, 0, 1, 1}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { __pyx_builtin_object = __Pyx_GetBuiltinName(__pyx_n_s_object); if (!__pyx_builtin_object) __PYX_ERR(0, 29, __pyx_L1_error) __pyx_builtin_KeyError = __Pyx_GetBuiltinName(__pyx_n_s_KeyError); if (!__pyx_builtin_KeyError) __PYX_ERR(0, 104, __pyx_L1_error) __pyx_builtin_TypeError = __Pyx_GetBuiltinName(__pyx_n_s_TypeError); if (!__pyx_builtin_TypeError) __PYX_ERR(1, 2, __pyx_L1_error) __pyx_builtin_IndexError = __Pyx_GetBuiltinName(__pyx_n_s_IndexError); if (!__pyx_builtin_IndexError) __PYX_ERR(0, 161, __pyx_L1_error) __pyx_builtin_StopIteration = __Pyx_GetBuiltinName(__pyx_n_s_StopIteration); if (!__pyx_builtin_StopIteration) __PYX_ERR(0, 183, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "borg/hashindex.pyx":67 * self.index = hashindex_read(path) * if not self.index: * raise Exception('hashindex_read failed') # <<<<<<<<<<<<<< * else: * self.index = hashindex_init(capacity, self.key_size, self.value_size) */ __pyx_tuple_ = PyTuple_Pack(1, __pyx_kp_s_hashindex_read_failed); if (unlikely(!__pyx_tuple_)) __PYX_ERR(0, 67, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple_); __Pyx_GIVEREF(__pyx_tuple_); /* "borg/hashindex.pyx":71 * self.index = hashindex_init(capacity, self.key_size, self.value_size) * if not self.index: * raise Exception('hashindex_init failed') # <<<<<<<<<<<<<< * * def __dealloc__(self): */ __pyx_tuple__2 = PyTuple_Pack(1, __pyx_kp_s_hashindex_init_failed); if (unlikely(!__pyx_tuple__2)) __PYX_ERR(0, 71, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__2); __Pyx_GIVEREF(__pyx_tuple__2); /* "borg/hashindex.pyx":84 * path = os.fsencode(path) * if not hashindex_write(self.index, path): * raise Exception('hashindex_write failed') # <<<<<<<<<<<<<< * * def clear(self): */ __pyx_tuple__3 = PyTuple_Pack(1, __pyx_kp_s_hashindex_write_failed); if (unlikely(!__pyx_tuple__3)) __PYX_ERR(0, 84, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__3); __Pyx_GIVEREF(__pyx_tuple__3); /* "borg/hashindex.pyx":90 * self.index = hashindex_init(0, self.key_size, self.value_size) * if not self.index: * raise Exception('hashindex_init failed') # <<<<<<<<<<<<<< * * def setdefault(self, key, value): */ __pyx_tuple__2 = PyTuple_Pack(1, __pyx_kp_s_hashindex_init_failed); if (unlikely(!__pyx_tuple__2)) __PYX_ERR(0, 90, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__2); __Pyx_GIVEREF(__pyx_tuple__2); /* "borg/hashindex.pyx":99 * assert len(key) == self.key_size * if not hashindex_delete(self.index, key): * raise Exception('hashindex_delete failed') # <<<<<<<<<<<<<< * * def get(self, key, default=None): */ __pyx_tuple__4 = PyTuple_Pack(1, __pyx_kp_s_hashindex_delete_failed); if (unlikely(!__pyx_tuple__4)) __PYX_ERR(0, 99, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__4); __Pyx_GIVEREF(__pyx_tuple__4); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple__6 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__6)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__6); __Pyx_GIVEREF(__pyx_tuple__6); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__7 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__7)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__7); __Pyx_GIVEREF(__pyx_tuple__7); /* "borg/hashindex.pyx":142 * data[1] = _htole32(value[1]) * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') # <<<<<<<<<<<<<< * * def __contains__(self, key): */ __pyx_tuple__8 = PyTuple_Pack(1, __pyx_kp_s_hashindex_set_failed); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(0, 142, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__8); __Pyx_GIVEREF(__pyx_tuple__8); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple__9 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__9)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__9); __Pyx_GIVEREF(__pyx_tuple__9); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__10 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__10)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__10); __Pyx_GIVEREF(__pyx_tuple__10); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple__11 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__11)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__11); __Pyx_GIVEREF(__pyx_tuple__11); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__12 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__12)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__12); __Pyx_GIVEREF(__pyx_tuple__12); /* "borg/hashindex.pyx":231 * data[2] = _htole32(value[2]) * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') # <<<<<<<<<<<<<< * * def __contains__(self, key): */ __pyx_tuple__8 = PyTuple_Pack(1, __pyx_kp_s_hashindex_set_failed); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(0, 231, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__8); __Pyx_GIVEREF(__pyx_tuple__8); /* "borg/hashindex.pyx":321 * else: * if not hashindex_set(self.index, key, data): * raise Exception('hashindex_set failed') # <<<<<<<<<<<<<< * * def merge(self, ChunkIndex other): */ __pyx_tuple__8 = PyTuple_Pack(1, __pyx_kp_s_hashindex_set_failed); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(0, 321, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__8); __Pyx_GIVEREF(__pyx_tuple__8); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple__13 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__13)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__13); __Pyx_GIVEREF(__pyx_tuple__13); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__14 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__14)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__14); __Pyx_GIVEREF(__pyx_tuple__14); /* "(tree fragment)":2 * def __reduce_cython__(self): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") */ __pyx_tuple__15 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__15)) __PYX_ERR(1, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__15); __Pyx_GIVEREF(__pyx_tuple__15); /* "(tree fragment)":4 * raise TypeError("no default __reduce__ due to non-trivial __cinit__") * def __setstate_cython__(self, __pyx_state): * raise TypeError("no default __reduce__ due to non-trivial __cinit__") # <<<<<<<<<<<<<< */ __pyx_tuple__16 = PyTuple_Pack(1, __pyx_kp_s_no_default___reduce___due_to_non); if (unlikely(!__pyx_tuple__16)) __PYX_ERR(1, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__16); __Pyx_GIVEREF(__pyx_tuple__16); __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(0, 1, __pyx_L1_error); __pyx_int_0 = PyInt_FromLong(0); if (unlikely(!__pyx_int_0)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_8 = PyInt_FromLong(8); if (unlikely(!__pyx_int_8)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_12 = PyInt_FromLong(12); if (unlikely(!__pyx_int_12)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_32 = PyInt_FromLong(32); if (unlikely(!__pyx_int_32)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_4294967295 = PyInt_FromString((char *)"4294967295", 0, 0); if (unlikely(!__pyx_int_4294967295)) __PYX_ERR(0, 1, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __pyx_v_4borg_9hashindex__NoDefault = Py_None; Py_INCREF(Py_None); __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ if (PyType_Ready(&__pyx_type_4borg_9hashindex_IndexBase) < 0) __PYX_ERR(0, 54, __pyx_L1_error) __pyx_type_4borg_9hashindex_IndexBase.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_9hashindex_IndexBase.tp_dictoffset && __pyx_type_4borg_9hashindex_IndexBase.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_9hashindex_IndexBase.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_9hashindex_IndexBase) < 0) __PYX_ERR(0, 54, __pyx_L1_error) __pyx_ptype_4borg_9hashindex_IndexBase = &__pyx_type_4borg_9hashindex_IndexBase; __pyx_type_4borg_9hashindex_NSIndex.tp_base = __pyx_ptype_4borg_9hashindex_IndexBase; if (PyType_Ready(&__pyx_type_4borg_9hashindex_NSIndex) < 0) __PYX_ERR(0, 121, __pyx_L1_error) __pyx_type_4borg_9hashindex_NSIndex.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_9hashindex_NSIndex.tp_dictoffset && __pyx_type_4borg_9hashindex_NSIndex.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_9hashindex_NSIndex.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (PyObject_SetAttr(__pyx_m, __pyx_n_s_NSIndex, (PyObject *)&__pyx_type_4borg_9hashindex_NSIndex) < 0) __PYX_ERR(0, 121, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_9hashindex_NSIndex) < 0) __PYX_ERR(0, 121, __pyx_L1_error) __pyx_ptype_4borg_9hashindex_NSIndex = &__pyx_type_4borg_9hashindex_NSIndex; if (PyType_Ready(&__pyx_type_4borg_9hashindex_NSKeyIterator) < 0) __PYX_ERR(0, 166, __pyx_L1_error) __pyx_type_4borg_9hashindex_NSKeyIterator.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_9hashindex_NSKeyIterator.tp_dictoffset && __pyx_type_4borg_9hashindex_NSKeyIterator.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_9hashindex_NSKeyIterator.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (PyObject_SetAttr(__pyx_m, __pyx_n_s_NSKeyIterator, (PyObject *)&__pyx_type_4borg_9hashindex_NSKeyIterator) < 0) __PYX_ERR(0, 166, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_9hashindex_NSKeyIterator) < 0) __PYX_ERR(0, 166, __pyx_L1_error) __pyx_ptype_4borg_9hashindex_NSKeyIterator = &__pyx_type_4borg_9hashindex_NSKeyIterator; __pyx_vtabptr_4borg_9hashindex_ChunkIndex = &__pyx_vtable_4borg_9hashindex_ChunkIndex; __pyx_vtable_4borg_9hashindex_ChunkIndex._add = (PyObject *(*)(struct __pyx_obj_4borg_9hashindex_ChunkIndex *, void *, uint32_t *))__pyx_f_4borg_9hashindex_10ChunkIndex__add; __pyx_type_4borg_9hashindex_ChunkIndex.tp_base = __pyx_ptype_4borg_9hashindex_IndexBase; if (PyType_Ready(&__pyx_type_4borg_9hashindex_ChunkIndex) < 0) __PYX_ERR(0, 194, __pyx_L1_error) __pyx_type_4borg_9hashindex_ChunkIndex.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_9hashindex_ChunkIndex.tp_dictoffset && __pyx_type_4borg_9hashindex_ChunkIndex.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_9hashindex_ChunkIndex.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (__Pyx_SetVtable(__pyx_type_4borg_9hashindex_ChunkIndex.tp_dict, __pyx_vtabptr_4borg_9hashindex_ChunkIndex) < 0) __PYX_ERR(0, 194, __pyx_L1_error) if (PyObject_SetAttr(__pyx_m, __pyx_n_s_ChunkIndex, (PyObject *)&__pyx_type_4borg_9hashindex_ChunkIndex) < 0) __PYX_ERR(0, 194, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_9hashindex_ChunkIndex) < 0) __PYX_ERR(0, 194, __pyx_L1_error) __pyx_ptype_4borg_9hashindex_ChunkIndex = &__pyx_type_4borg_9hashindex_ChunkIndex; if (PyType_Ready(&__pyx_type_4borg_9hashindex_ChunkKeyIterator) < 0) __PYX_ERR(0, 333, __pyx_L1_error) __pyx_type_4borg_9hashindex_ChunkKeyIterator.tp_print = 0; if ((CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP) && likely(!__pyx_type_4borg_9hashindex_ChunkKeyIterator.tp_dictoffset && __pyx_type_4borg_9hashindex_ChunkKeyIterator.tp_getattro == PyObject_GenericGetAttr)) { __pyx_type_4borg_9hashindex_ChunkKeyIterator.tp_getattro = __Pyx_PyObject_GenericGetAttr; } if (PyObject_SetAttr(__pyx_m, __pyx_n_s_ChunkKeyIterator, (PyObject *)&__pyx_type_4borg_9hashindex_ChunkKeyIterator) < 0) __PYX_ERR(0, 333, __pyx_L1_error) if (__Pyx_setup_reduce((PyObject*)&__pyx_type_4borg_9hashindex_ChunkKeyIterator) < 0) __PYX_ERR(0, 333, __pyx_L1_error) __pyx_ptype_4borg_9hashindex_ChunkKeyIterator = &__pyx_type_4borg_9hashindex_ChunkKeyIterator; __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC inithashindex(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC inithashindex(void) #else __Pyx_PyMODINIT_FUNC PyInit_hashindex(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_hashindex(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_hashindex(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; int __pyx_t_3; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'hashindex' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_hashindex(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("hashindex", __pyx_methods, 0, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(0, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(0, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(0, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(0, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__hashindex) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name, __pyx_n_s_main) < 0) __PYX_ERR(0, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(0, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.hashindex")) { if (unlikely(PyDict_SetItemString(modules, "borg.hashindex", __pyx_m) < 0)) __PYX_ERR(0, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); if (unlikely(__Pyx_modinit_type_init_code() != 0)) goto __pyx_L1_error; (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /* "borg/hashindex.pyx":2 * # -*- coding: utf-8 -*- * import os # <<<<<<<<<<<<<< * * cimport cython */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_os, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_os, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/hashindex.pyx":7 * from libc.stdint cimport uint32_t, UINT32_MAX, uint64_t * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(0, 7, __pyx_L1_error) /* "borg/hashindex.pyx":29 * * * cdef _NoDefault = object() # <<<<<<<<<<<<<< * * """ */ __pyx_t_1 = __Pyx_PyObject_CallNoArg(__pyx_builtin_object); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 29, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_XGOTREF(__pyx_v_4borg_9hashindex__NoDefault); __Pyx_DECREF_SET(__pyx_v_4borg_9hashindex__NoDefault, __pyx_t_1); __Pyx_GIVEREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/hashindex.pyx":46 * """ * * assert UINT32_MAX == 2**32-1 # <<<<<<<<<<<<<< * * # module-level constant because cdef's in classes can't have default values */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { __pyx_t_1 = __Pyx_PyInt_From_uint32_t(UINT32_MAX); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyObject_RichCompare(__pyx_t_1, __pyx_int_4294967295, Py_EQ); __Pyx_XGOTREF(__pyx_t_2); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_3 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_3 < 0)) __PYX_ERR(0, 46, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 46, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":49 * * # module-level constant because cdef's in classes can't have default values * cdef uint32_t _MAX_VALUE = 2**32-1025 # <<<<<<<<<<<<<< * * assert _MAX_VALUE % 2 == 1 */ __pyx_v_4borg_9hashindex__MAX_VALUE = 0xFFFFFBFF; /* "borg/hashindex.pyx":51 * cdef uint32_t _MAX_VALUE = 2**32-1025 * * assert _MAX_VALUE % 2 == 1 # <<<<<<<<<<<<<< * * @cython.internal */ #ifndef CYTHON_WITHOUT_ASSERTIONS if (unlikely(!Py_OptimizeFlag)) { if (unlikely(!((__Pyx_mod_long(__pyx_v_4borg_9hashindex__MAX_VALUE, 2) == 1) != 0))) { PyErr_SetNone(PyExc_AssertionError); __PYX_ERR(0, 51, __pyx_L1_error) } } #endif /* "borg/hashindex.pyx":58 * cdef int key_size * * MAX_LOAD_FACTOR = HASH_MAX_LOAD # <<<<<<<<<<<<<< * MAX_VALUE = _MAX_VALUE * */ __pyx_t_2 = PyFloat_FromDouble(HASH_MAX_LOAD); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 58, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_9hashindex_IndexBase->tp_dict, __pyx_n_s_MAX_LOAD_FACTOR, __pyx_t_2) < 0) __PYX_ERR(0, 58, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; PyType_Modified(__pyx_ptype_4borg_9hashindex_IndexBase); /* "borg/hashindex.pyx":59 * * MAX_LOAD_FACTOR = HASH_MAX_LOAD * MAX_VALUE = _MAX_VALUE # <<<<<<<<<<<<<< * * def __cinit__(self, capacity=0, path=None, key_size=32): */ __pyx_t_2 = __Pyx_PyInt_From_uint32_t(__pyx_v_4borg_9hashindex__MAX_VALUE); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_9hashindex_IndexBase->tp_dict, __pyx_n_s_MAX_VALUE, __pyx_t_2) < 0) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; PyType_Modified(__pyx_ptype_4borg_9hashindex_IndexBase); /* "borg/hashindex.pyx":78 * * @classmethod * def read(cls, path): # <<<<<<<<<<<<<< * return cls(path=path) * */ __Pyx_GetNameInClass(__pyx_t_2, (PyObject *)__pyx_ptype_4borg_9hashindex_IndexBase, __pyx_n_s_read); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 78, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); /* "borg/hashindex.pyx":77 * hashindex_free(self.index) * * @classmethod # <<<<<<<<<<<<<< * def read(cls, path): * return cls(path=path) */ __pyx_t_1 = __Pyx_Method_ClassMethod(__pyx_t_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 77, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_9hashindex_IndexBase->tp_dict, __pyx_n_s_read, __pyx_t_1) < 0) __PYX_ERR(0, 78, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; PyType_Modified(__pyx_ptype_4borg_9hashindex_IndexBase); /* "borg/hashindex.pyx":107 * return default * * def pop(self, key, default=_NoDefault): # <<<<<<<<<<<<<< * try: * value = self[key] */ __Pyx_INCREF(__pyx_v_4borg_9hashindex__NoDefault); __pyx_k__5 = __pyx_v_4borg_9hashindex__NoDefault; __Pyx_GIVEREF(__pyx_v_4borg_9hashindex__NoDefault); /* "borg/hashindex.pyx":123 * cdef class NSIndex(IndexBase): * * value_size = 8 # <<<<<<<<<<<<<< * * def __getitem__(self, key): */ if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_9hashindex_NSIndex->tp_dict, __pyx_n_s_value_size, __pyx_int_8) < 0) __PYX_ERR(0, 123, __pyx_L1_error) PyType_Modified(__pyx_ptype_4borg_9hashindex_NSIndex); /* "borg/hashindex.pyx":211 * """ * * value_size = 12 # <<<<<<<<<<<<<< * * def __getitem__(self, key): */ if (PyDict_SetItem((PyObject *)__pyx_ptype_4borg_9hashindex_ChunkIndex->tp_dict, __pyx_n_s_value_size, __pyx_int_12) < 0) __PYX_ERR(0, 211, __pyx_L1_error) PyType_Modified(__pyx_ptype_4borg_9hashindex_ChunkIndex); /* "borg/hashindex.pyx":1 * # -*- coding: utf-8 -*- # <<<<<<<<<<<<<< * import os * */ __pyx_t_1 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_1) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.hashindex", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.hashindex"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* GetModuleGlobalName */ #if CYTHON_USE_DICT_VERSIONS static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value) #else static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name) #endif { PyObject *result; #if !CYTHON_AVOID_BORROWED_REFS #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 result = _PyDict_GetItem_KnownHash(__pyx_d, name, ((PyASCIIObject *) name)->hash); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } else if (unlikely(PyErr_Occurred())) { return NULL; } #else result = PyDict_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } #endif #else result = PyObject_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } PyErr_Clear(); #endif return __Pyx_GetBuiltinName(name); } /* PyCFunctionFastCall */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, Py_ssize_t nargs) { PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); int flags = PyCFunction_GET_FLAGS(func); assert(PyCFunction_Check(func)); assert(METH_FASTCALL == (flags & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))); assert(nargs >= 0); assert(nargs == 0 || args != NULL); /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); if ((PY_VERSION_HEX < 0x030700A0) || unlikely(flags & METH_KEYWORDS)) { return (*((__Pyx_PyCFunctionFastWithKeywords)(void*)meth)) (self, args, nargs, NULL); } else { return (*((__Pyx_PyCFunctionFast)(void*)meth)) (self, args, nargs); } } #endif /* PyFunctionFastCall */ #if CYTHON_FAST_PYCALL static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject **fastlocals; Py_ssize_t i; PyObject *result; assert(globals != NULL); /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) { return NULL; } fastlocals = __Pyx_PyFrame_GetLocalsplus(f); for (i = 0; i < na; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } result = PyEval_EvalFrameEx(f,0); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; return result; } #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *closure; #if PY_MAJOR_VERSION >= 3 PyObject *kwdefs; #endif PyObject *kwtuple, **k; PyObject **d; Py_ssize_t nd; Py_ssize_t nk; PyObject *result; assert(kwargs == NULL || PyDict_Check(kwargs)); nk = kwargs ? PyDict_Size(kwargs) : 0; if (Py_EnterRecursiveCall((char*)" while calling a Python object")) { return NULL; } if ( #if PY_MAJOR_VERSION >= 3 co->co_kwonlyargcount == 0 && #endif likely(kwargs == NULL || nk == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { result = __Pyx_PyFunction_FastCallNoKw(co, args, nargs, globals); goto done; } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); result =__Pyx_PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), globals); goto done; } } if (kwargs != NULL) { Py_ssize_t pos, i; kwtuple = PyTuple_New(2 * nk); if (kwtuple == NULL) { result = NULL; goto done; } k = &PyTuple_GET_ITEM(kwtuple, 0); pos = i = 0; while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { Py_INCREF(k[i]); Py_INCREF(k[i+1]); i += 2; } nk = i / 2; } else { kwtuple = NULL; k = NULL; } closure = PyFunction_GET_CLOSURE(func); #if PY_MAJOR_VERSION >= 3 kwdefs = PyFunction_GET_KW_DEFAULTS(func); #endif if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } else { d = NULL; nd = 0; } #if PY_MAJOR_VERSION >= 3 result = PyEval_EvalCodeEx((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, kwdefs, closure); #else result = PyEval_EvalCodeEx(co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, closure); #endif Py_XDECREF(kwtuple); done: Py_LeaveRecursiveCall(); return result; } #endif #endif /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCall2Args */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2) { PyObject *args, *result = NULL; #if CYTHON_FAST_PYCALL if (PyFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyFunction_FastCall(function, args, 2); } #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyCFunction_FastCall(function, args, 2); } #endif args = PyTuple_New(2); if (unlikely(!args)) goto done; Py_INCREF(arg1); PyTuple_SET_ITEM(args, 0, arg1); Py_INCREF(arg2); PyTuple_SET_ITEM(args, 1, arg2); Py_INCREF(function); result = __Pyx_PyObject_Call(function, args, NULL); Py_DECREF(args); Py_DECREF(function); done: return result; } /* PyObjectCallMethO */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg) { PyObject *self, *result; PyCFunction cfunc; cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = cfunc(self, arg); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCallOneArg */ #if CYTHON_COMPILING_IN_CPYTHON static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_New(1); if (unlikely(!args)) return NULL; Py_INCREF(arg); PyTuple_SET_ITEM(args, 0, arg); result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, &arg, 1); } #endif if (likely(PyCFunction_Check(func))) { if (likely(PyCFunction_GET_FLAGS(func) & METH_O)) { return __Pyx_PyObject_CallMethO(func, arg); #if CYTHON_FAST_PYCCALL } else if (PyCFunction_GET_FLAGS(func) & METH_FASTCALL) { return __Pyx_PyCFunction_FastCall(func, &arg, 1); #endif } } return __Pyx__PyObject_CallOneArg(func, arg); } #else static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_Pack(1, arg); if (unlikely(!args)) return NULL; result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } #endif /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* RaiseException */ #if PY_MAJOR_VERSION < 3 static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, CYTHON_UNUSED PyObject *cause) { __Pyx_PyThreadState_declare Py_XINCREF(type); if (!value || value == Py_None) value = NULL; else Py_INCREF(value); if (!tb || tb == Py_None) tb = NULL; else { Py_INCREF(tb); if (!PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto raise_error; } } if (PyType_Check(type)) { #if CYTHON_COMPILING_IN_PYPY if (!value) { Py_INCREF(Py_None); value = Py_None; } #endif PyErr_NormalizeException(&type, &value, &tb); } else { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto raise_error; } value = type; type = (PyObject*) Py_TYPE(type); Py_INCREF(type); if (!PyType_IsSubtype((PyTypeObject *)type, (PyTypeObject *)PyExc_BaseException)) { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto raise_error; } } __Pyx_PyThreadState_assign __Pyx_ErrRestore(type, value, tb); return; raise_error: Py_XDECREF(value); Py_XDECREF(type); Py_XDECREF(tb); return; } #else static void __Pyx_Raise(PyObject *type, PyObject *value, PyObject *tb, PyObject *cause) { PyObject* owned_instance = NULL; if (tb == Py_None) { tb = 0; } else if (tb && !PyTraceBack_Check(tb)) { PyErr_SetString(PyExc_TypeError, "raise: arg 3 must be a traceback or None"); goto bad; } if (value == Py_None) value = 0; if (PyExceptionInstance_Check(type)) { if (value) { PyErr_SetString(PyExc_TypeError, "instance exception may not have a separate value"); goto bad; } value = type; type = (PyObject*) Py_TYPE(value); } else if (PyExceptionClass_Check(type)) { PyObject *instance_class = NULL; if (value && PyExceptionInstance_Check(value)) { instance_class = (PyObject*) Py_TYPE(value); if (instance_class != type) { int is_subclass = PyObject_IsSubclass(instance_class, type); if (!is_subclass) { instance_class = NULL; } else if (unlikely(is_subclass == -1)) { goto bad; } else { type = instance_class; } } } if (!instance_class) { PyObject *args; if (!value) args = PyTuple_New(0); else if (PyTuple_Check(value)) { Py_INCREF(value); args = value; } else args = PyTuple_Pack(1, value); if (!args) goto bad; owned_instance = PyObject_Call(type, args, NULL); Py_DECREF(args); if (!owned_instance) goto bad; value = owned_instance; if (!PyExceptionInstance_Check(value)) { PyErr_Format(PyExc_TypeError, "calling %R should have returned an instance of " "BaseException, not %R", type, Py_TYPE(value)); goto bad; } } } else { PyErr_SetString(PyExc_TypeError, "raise: exception class must be a subclass of BaseException"); goto bad; } if (cause) { PyObject *fixed_cause; if (cause == Py_None) { fixed_cause = NULL; } else if (PyExceptionClass_Check(cause)) { fixed_cause = PyObject_CallObject(cause, NULL); if (fixed_cause == NULL) goto bad; } else if (PyExceptionInstance_Check(cause)) { fixed_cause = cause; Py_INCREF(fixed_cause); } else { PyErr_SetString(PyExc_TypeError, "exception causes must derive from " "BaseException"); goto bad; } PyException_SetCause(value, fixed_cause); } PyErr_SetObject(type, value); if (tb) { #if CYTHON_COMPILING_IN_PYPY PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_Fetch(&tmp_type, &tmp_value, &tmp_tb); Py_INCREF(tb); PyErr_Restore(tmp_type, tmp_value, tb); Py_XDECREF(tmp_tb); #else PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* tmp_tb = tstate->curexc_traceback; if (tb != tmp_tb) { Py_INCREF(tb); tstate->curexc_traceback = tb; Py_XDECREF(tmp_tb); } #endif } bad: Py_XDECREF(owned_instance); return; } #endif /* GetItemInt */ static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) { PyObject *r; if (!j) return NULL; r = PyObject_GetItem(o, j); Py_DECREF(j); return r; } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyList_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyList_GET_SIZE(o)))) { PyObject *r = PyList_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyTuple_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyList_GET_SIZE(o); if ((!boundscheck) || (likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o))))) { PyObject *r = PyList_GET_ITEM(o, n); Py_INCREF(r); return r; } } else if (PyTuple_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyTuple_GET_SIZE(o); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, n); Py_INCREF(r); return r; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return NULL; PyErr_Clear(); } } return m->sq_item(o, i); } } #else if (is_list || PySequence_Check(o)) { return PySequence_GetItem(o, i); } #endif return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); } /* ObjectGetItem */ #if CYTHON_USE_TYPE_SLOTS static PyObject *__Pyx_PyObject_GetIndex(PyObject *obj, PyObject* index) { PyObject *runerr; Py_ssize_t key_value; PySequenceMethods *m = Py_TYPE(obj)->tp_as_sequence; if (unlikely(!(m && m->sq_item))) { PyErr_Format(PyExc_TypeError, "'%.200s' object is not subscriptable", Py_TYPE(obj)->tp_name); return NULL; } key_value = __Pyx_PyIndex_AsSsize_t(index); if (likely(key_value != -1 || !(runerr = PyErr_Occurred()))) { return __Pyx_GetItemInt_Fast(obj, key_value, 0, 1, 1); } if (PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError)) { PyErr_Clear(); PyErr_Format(PyExc_IndexError, "cannot fit '%.200s' into an index-sized integer", Py_TYPE(index)->tp_name); } return NULL; } static PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key) { PyMappingMethods *m = Py_TYPE(obj)->tp_as_mapping; if (likely(m && m->mp_subscript)) { return m->mp_subscript(obj, key); } return __Pyx_PyObject_GetIndex(obj, key); } #endif /* GetTopmostException */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate) { _PyErr_StackItem *exc_info = tstate->exc_info; while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && exc_info->previous_item != NULL) { exc_info = exc_info->previous_item; } return exc_info; } #endif /* SaveResetException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = __Pyx_PyErr_GetTopmostException(tstate); *type = exc_info->exc_type; *value = exc_info->exc_value; *tb = exc_info->exc_traceback; #else *type = tstate->exc_type; *value = tstate->exc_value; *tb = tstate->exc_traceback; #endif Py_XINCREF(*type); Py_XINCREF(*value); Py_XINCREF(*tb); } static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = type; exc_info->exc_value = value; exc_info->exc_traceback = tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = type; tstate->exc_value = value; tstate->exc_traceback = tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } #endif /* PyErrExceptionMatches */ #if CYTHON_FAST_THREAD_STATE static int __Pyx_PyErr_ExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; icurexc_type; if (exc_type == err) return 1; if (unlikely(!exc_type)) return 0; if (unlikely(PyTuple_Check(err))) return __Pyx_PyErr_ExceptionMatchesTuple(exc_type, err); return __Pyx_PyErr_GivenExceptionMatches(exc_type, err); } #endif /* GetException */ #if CYTHON_FAST_THREAD_STATE static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb) #endif { PyObject *local_type, *local_value, *local_tb; #if CYTHON_FAST_THREAD_STATE PyObject *tmp_type, *tmp_value, *tmp_tb; local_type = tstate->curexc_type; local_value = tstate->curexc_value; local_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; #else PyErr_Fetch(&local_type, &local_value, &local_tb); #endif PyErr_NormalizeException(&local_type, &local_value, &local_tb); #if CYTHON_FAST_THREAD_STATE if (unlikely(tstate->curexc_type)) #else if (unlikely(PyErr_Occurred())) #endif goto bad; #if PY_MAJOR_VERSION >= 3 if (local_tb) { if (unlikely(PyException_SetTraceback(local_value, local_tb) < 0)) goto bad; } #endif Py_XINCREF(local_tb); Py_XINCREF(local_type); Py_XINCREF(local_value); *type = local_type; *value = local_value; *tb = local_tb; #if CYTHON_FAST_THREAD_STATE #if CYTHON_USE_EXC_INFO_STACK { _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = local_type; exc_info->exc_value = local_value; exc_info->exc_traceback = local_tb; } #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = local_type; tstate->exc_value = local_value; tstate->exc_traceback = local_tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); #else PyErr_SetExcInfo(local_type, local_value, local_tb); #endif return 0; bad: *type = 0; *value = 0; *tb = 0; Py_XDECREF(local_type); Py_XDECREF(local_value); Py_XDECREF(local_tb); return -1; } /* ArgTypeTest */ static int __Pyx__ArgTypeTest(PyObject *obj, PyTypeObject *type, const char *name, int exact) { if (unlikely(!type)) { PyErr_SetString(PyExc_SystemError, "Missing type object"); return 0; } else if (exact) { #if PY_MAJOR_VERSION == 2 if ((type == &PyBaseString_Type) && likely(__Pyx_PyBaseString_CheckExact(obj))) return 1; #endif } else { if (likely(__Pyx_TypeCheck(obj, type))) return 1; } PyErr_Format(PyExc_TypeError, "Argument '%.200s' has incorrect type (expected %.200s, got %.200s)", name, type->tp_name, Py_TYPE(obj)->tp_name); return 0; } /* PyObject_GenericGetAttrNoDict */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject *__Pyx_RaiseGenericGetAttributeError(PyTypeObject *tp, PyObject *attr_name) { PyErr_Format(PyExc_AttributeError, #if PY_MAJOR_VERSION >= 3 "'%.50s' object has no attribute '%U'", tp->tp_name, attr_name); #else "'%.50s' object has no attribute '%.400s'", tp->tp_name, PyString_AS_STRING(attr_name)); #endif return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyObject_GenericGetAttrNoDict(PyObject* obj, PyObject* attr_name) { PyObject *descr; PyTypeObject *tp = Py_TYPE(obj); if (unlikely(!PyString_Check(attr_name))) { return PyObject_GenericGetAttr(obj, attr_name); } assert(!tp->tp_dictoffset); descr = _PyType_Lookup(tp, attr_name); if (unlikely(!descr)) { return __Pyx_RaiseGenericGetAttributeError(tp, attr_name); } Py_INCREF(descr); #if PY_MAJOR_VERSION < 3 if (likely(PyType_HasFeature(Py_TYPE(descr), Py_TPFLAGS_HAVE_CLASS))) #endif { descrgetfunc f = Py_TYPE(descr)->tp_descr_get; if (unlikely(f)) { PyObject *res = f(descr, obj, (PyObject *)tp); Py_DECREF(descr); return res; } } return descr; } #endif /* PyObject_GenericGetAttr */ #if CYTHON_USE_TYPE_SLOTS && CYTHON_USE_PYTYPE_LOOKUP && PY_VERSION_HEX < 0x03070000 static PyObject* __Pyx_PyObject_GenericGetAttr(PyObject* obj, PyObject* attr_name) { if (unlikely(Py_TYPE(obj)->tp_dictoffset)) { return PyObject_GenericGetAttr(obj, attr_name); } return __Pyx_PyObject_GenericGetAttrNoDict(obj, attr_name); } #endif /* SetupReduce */ static int __Pyx_setup_reduce_is_named(PyObject* meth, PyObject* name) { int ret; PyObject *name_attr; name_attr = __Pyx_PyObject_GetAttrStr(meth, __pyx_n_s_name); if (likely(name_attr)) { ret = PyObject_RichCompareBool(name_attr, name, Py_EQ); } else { ret = -1; } if (unlikely(ret < 0)) { PyErr_Clear(); ret = 0; } Py_XDECREF(name_attr); return ret; } static int __Pyx_setup_reduce(PyObject* type_obj) { int ret = 0; PyObject *object_reduce = NULL; PyObject *object_reduce_ex = NULL; PyObject *reduce = NULL; PyObject *reduce_ex = NULL; PyObject *reduce_cython = NULL; PyObject *setstate = NULL; PyObject *setstate_cython = NULL; #if CYTHON_USE_PYTYPE_LOOKUP if (_PyType_Lookup((PyTypeObject*)type_obj, __pyx_n_s_getstate)) goto GOOD; #else if (PyObject_HasAttr(type_obj, __pyx_n_s_getstate)) goto GOOD; #endif #if CYTHON_USE_PYTYPE_LOOKUP object_reduce_ex = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #else object_reduce_ex = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce_ex); if (!object_reduce_ex) goto BAD; #endif reduce_ex = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_ex); if (unlikely(!reduce_ex)) goto BAD; if (reduce_ex == object_reduce_ex) { #if CYTHON_USE_PYTYPE_LOOKUP object_reduce = _PyType_Lookup(&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #else object_reduce = __Pyx_PyObject_GetAttrStr((PyObject*)&PyBaseObject_Type, __pyx_n_s_reduce); if (!object_reduce) goto BAD; #endif reduce = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce); if (unlikely(!reduce)) goto BAD; if (reduce == object_reduce || __Pyx_setup_reduce_is_named(reduce, __pyx_n_s_reduce_cython)) { reduce_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_reduce_cython); if (unlikely(!reduce_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce, reduce_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_reduce_cython); if (unlikely(ret < 0)) goto BAD; setstate = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate); if (!setstate) PyErr_Clear(); if (!setstate || __Pyx_setup_reduce_is_named(setstate, __pyx_n_s_setstate_cython)) { setstate_cython = __Pyx_PyObject_GetAttrStr(type_obj, __pyx_n_s_setstate_cython); if (unlikely(!setstate_cython)) goto BAD; ret = PyDict_SetItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate, setstate_cython); if (unlikely(ret < 0)) goto BAD; ret = PyDict_DelItem(((PyTypeObject*)type_obj)->tp_dict, __pyx_n_s_setstate_cython); if (unlikely(ret < 0)) goto BAD; } PyType_Modified((PyTypeObject*)type_obj); } } goto GOOD; BAD: if (!PyErr_Occurred()) PyErr_Format(PyExc_RuntimeError, "Unable to initialize pickling for %s", ((PyTypeObject*)type_obj)->tp_name); ret = -1; GOOD: #if !CYTHON_USE_PYTYPE_LOOKUP Py_XDECREF(object_reduce); Py_XDECREF(object_reduce_ex); #endif Py_XDECREF(reduce); Py_XDECREF(reduce_ex); Py_XDECREF(reduce_cython); Py_XDECREF(setstate); Py_XDECREF(setstate_cython); return ret; } /* SetVTable */ static int __Pyx_SetVtable(PyObject *dict, void *vtable) { #if PY_VERSION_HEX >= 0x02070000 PyObject *ob = PyCapsule_New(vtable, 0, 0); #else PyObject *ob = PyCObject_FromVoidPtr(vtable, 0); #endif if (!ob) goto bad; if (PyDict_SetItem(dict, __pyx_n_s_pyx_vtable, ob) < 0) goto bad; Py_DECREF(ob); return 0; bad: Py_XDECREF(ob); return -1; } /* Import */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level) { PyObject *empty_list = 0; PyObject *module = 0; PyObject *global_dict = 0; PyObject *empty_dict = 0; PyObject *list; #if PY_MAJOR_VERSION < 3 PyObject *py_import; py_import = __Pyx_PyObject_GetAttrStr(__pyx_b, __pyx_n_s_import); if (!py_import) goto bad; #endif if (from_list) list = from_list; else { empty_list = PyList_New(0); if (!empty_list) goto bad; list = empty_list; } global_dict = PyModule_GetDict(__pyx_m); if (!global_dict) goto bad; empty_dict = PyDict_New(); if (!empty_dict) goto bad; { #if PY_MAJOR_VERSION >= 3 if (level == -1) { if (strchr(__Pyx_MODULE_NAME, '.')) { module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, 1); if (!module) { if (!PyErr_ExceptionMatches(PyExc_ImportError)) goto bad; PyErr_Clear(); } } level = 0; } #endif if (!module) { #if PY_MAJOR_VERSION < 3 PyObject *py_level = PyInt_FromLong(level); if (!py_level) goto bad; module = PyObject_CallFunctionObjArgs(py_import, name, global_dict, empty_dict, list, py_level, (PyObject *)NULL); Py_DECREF(py_level); #else module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, level); #endif } } bad: #if PY_MAJOR_VERSION < 3 Py_XDECREF(py_import); #endif Py_XDECREF(empty_list); Py_XDECREF(empty_dict); return module; } /* PyObjectCallNoArg */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallNoArg(PyObject *func) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, NULL, 0); } #endif #ifdef __Pyx_CyFunction_USED if (likely(PyCFunction_Check(func) || __Pyx_CyFunction_Check(func))) #else if (likely(PyCFunction_Check(func))) #endif { if (likely(PyCFunction_GET_FLAGS(func) & METH_NOARGS)) { return __Pyx_PyObject_CallMethO(func, NULL); } } return __Pyx_PyObject_Call(func, __pyx_empty_tuple, NULL); } #endif /* None */ static CYTHON_INLINE long __Pyx_mod_long(long a, long b) { long r = a % b; r += ((r != 0) & ((r ^ b) < 0)) * b; return r; } /* ClassMethod */ static PyObject* __Pyx_Method_ClassMethod(PyObject *method) { #if CYTHON_COMPILING_IN_PYPY && PYPY_VERSION_NUM <= 0x05080000 if (PyObject_TypeCheck(method, &PyWrapperDescr_Type)) { return PyClassMethod_New(method); } #else #if CYTHON_COMPILING_IN_PYSTON || CYTHON_COMPILING_IN_PYPY if (PyMethodDescr_Check(method)) #else static PyTypeObject *methoddescr_type = NULL; if (methoddescr_type == NULL) { PyObject *meth = PyObject_GetAttrString((PyObject*)&PyList_Type, "append"); if (!meth) return NULL; methoddescr_type = Py_TYPE(meth); Py_DECREF(meth); } if (__Pyx_TypeCheck(method, methoddescr_type)) #endif { PyMethodDescrObject *descr = (PyMethodDescrObject *)method; #if PY_VERSION_HEX < 0x03020000 PyTypeObject *d_type = descr->d_type; #else PyTypeObject *d_type = descr->d_common.d_type; #endif return PyDescr_NewClassMethod(d_type, descr->d_method); } #endif else if (PyMethod_Check(method)) { return PyClassMethod_New(PyMethod_GET_FUNCTION(method)); } else if (PyCFunction_Check(method)) { return PyClassMethod_New(method); } #ifdef __Pyx_CyFunction_USED else if (__Pyx_CyFunction_Check(method)) { return PyClassMethod_New(method); } #endif PyErr_SetString(PyExc_TypeError, "Class-level classmethod() can only be called on " "a method_descriptor or instance method."); return NULL; } /* GetNameInClass */ static PyObject *__Pyx_GetGlobalNameAfterAttributeLookup(PyObject *name) { PyObject *result; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign if (unlikely(!__Pyx_PyErr_ExceptionMatches(PyExc_AttributeError))) return NULL; __Pyx_PyErr_Clear(); __Pyx_GetModuleGlobalNameUncached(result, name); return result; } static PyObject *__Pyx__GetNameInClass(PyObject *nmspace, PyObject *name) { PyObject *result; result = __Pyx_PyObject_GetAttrStr(nmspace, name); if (!result) { result = __Pyx_GetGlobalNameAfterAttributeLookup(name); } return result; } /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_uint32_t(uint32_t value) { const uint32_t neg_one = (uint32_t) ((uint32_t) 0 - (uint32_t) 1), const_zero = (uint32_t) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(uint32_t) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(uint32_t) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(uint32_t) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(uint32_t), little, !is_unsigned); } } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(int) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(int) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(int) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(int), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_uint64_t(uint64_t value) { const uint64_t neg_one = (uint64_t) ((uint64_t) 0 - (uint64_t) 1), const_zero = (uint64_t) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(uint64_t) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(uint64_t) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(uint64_t) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(uint64_t) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(uint64_t) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(uint64_t), little, !is_unsigned); } } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* CIntFromPy */ static CYTHON_INLINE uint32_t __Pyx_PyInt_As_uint32_t(PyObject *x) { const uint32_t neg_one = (uint32_t) ((uint32_t) 0 - (uint32_t) 1), const_zero = (uint32_t) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(uint32_t) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(uint32_t, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (uint32_t) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (uint32_t) 0; case 1: __PYX_VERIFY_RETURN_INT(uint32_t, digit, digits[0]) case 2: if (8 * sizeof(uint32_t) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) >= 2 * PyLong_SHIFT) { return (uint32_t) (((((uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0])); } } break; case 3: if (8 * sizeof(uint32_t) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) >= 3 * PyLong_SHIFT) { return (uint32_t) (((((((uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0])); } } break; case 4: if (8 * sizeof(uint32_t) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) >= 4 * PyLong_SHIFT) { return (uint32_t) (((((((((uint32_t)digits[3]) << PyLong_SHIFT) | (uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (uint32_t) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(uint32_t) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (uint32_t) 0; case -1: __PYX_VERIFY_RETURN_INT(uint32_t, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(uint32_t, digit, +digits[0]) case -2: if (8 * sizeof(uint32_t) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 2 * PyLong_SHIFT) { return (uint32_t) (((uint32_t)-1)*(((((uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case 2: if (8 * sizeof(uint32_t) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 2 * PyLong_SHIFT) { return (uint32_t) ((((((uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case -3: if (8 * sizeof(uint32_t) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 3 * PyLong_SHIFT) { return (uint32_t) (((uint32_t)-1)*(((((((uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case 3: if (8 * sizeof(uint32_t) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 3 * PyLong_SHIFT) { return (uint32_t) ((((((((uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case -4: if (8 * sizeof(uint32_t) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 4 * PyLong_SHIFT) { return (uint32_t) (((uint32_t)-1)*(((((((((uint32_t)digits[3]) << PyLong_SHIFT) | (uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; case 4: if (8 * sizeof(uint32_t) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(uint32_t, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(uint32_t) - 1 > 4 * PyLong_SHIFT) { return (uint32_t) ((((((((((uint32_t)digits[3]) << PyLong_SHIFT) | (uint32_t)digits[2]) << PyLong_SHIFT) | (uint32_t)digits[1]) << PyLong_SHIFT) | (uint32_t)digits[0]))); } } break; } #endif if (sizeof(uint32_t) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(uint32_t) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(uint32_t, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else uint32_t val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (uint32_t) -1; } } else { uint32_t val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (uint32_t) -1; val = __Pyx_PyInt_As_uint32_t(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to uint32_t"); return (uint32_t) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to uint32_t"); return (uint32_t) -1; } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; ip) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/hashindex.pyx0000644000175000017500000002764413431513133017351 0ustar useruser00000000000000# -*- coding: utf-8 -*- import os cimport cython from libc.stdint cimport uint32_t, UINT32_MAX, uint64_t API_VERSION = '1.0_01' cdef extern from "_hashindex.c": ctypedef struct HashIndex: pass HashIndex *hashindex_read(char *path) HashIndex *hashindex_init(int capacity, int key_size, int value_size) void hashindex_free(HashIndex *index) int hashindex_get_size(HashIndex *index) int hashindex_write(HashIndex *index, char *path) void *hashindex_get(HashIndex *index, void *key) void *hashindex_next_key(HashIndex *index, void *key) int hashindex_delete(HashIndex *index, void *key) int hashindex_set(HashIndex *index, void *key, void *value) uint32_t _htole32(uint32_t v) uint32_t _le32toh(uint32_t v) double HASH_MAX_LOAD cdef _NoDefault = object() """ The HashIndex is *not* a general purpose data structure. The value size must be at least 4 bytes, and these first bytes are used for in-band signalling in the data structure itself. The constant MAX_VALUE defines the valid range for these 4 bytes when interpreted as an uint32_t from 0 to MAX_VALUE (inclusive). The following reserved values beyond MAX_VALUE are currently in use (byte order is LE):: 0xffffffff marks empty entries in the hashtable 0xfffffffe marks deleted entries in the hashtable None of the publicly available classes in this module will accept nor return a reserved value; AssertionError is raised instead. """ assert UINT32_MAX == 2**32-1 # module-level constant because cdef's in classes can't have default values cdef uint32_t _MAX_VALUE = 2**32-1025 assert _MAX_VALUE % 2 == 1 @cython.internal cdef class IndexBase: cdef HashIndex *index cdef int key_size MAX_LOAD_FACTOR = HASH_MAX_LOAD MAX_VALUE = _MAX_VALUE def __cinit__(self, capacity=0, path=None, key_size=32): self.key_size = key_size if path: path = os.fsencode(path) self.index = hashindex_read(path) if not self.index: raise Exception('hashindex_read failed') else: self.index = hashindex_init(capacity, self.key_size, self.value_size) if not self.index: raise Exception('hashindex_init failed') def __dealloc__(self): if self.index: hashindex_free(self.index) @classmethod def read(cls, path): return cls(path=path) def write(self, path): path = os.fsencode(path) if not hashindex_write(self.index, path): raise Exception('hashindex_write failed') def clear(self): hashindex_free(self.index) self.index = hashindex_init(0, self.key_size, self.value_size) if not self.index: raise Exception('hashindex_init failed') def setdefault(self, key, value): if not key in self: self[key] = value def __delitem__(self, key): assert len(key) == self.key_size if not hashindex_delete(self.index, key): raise Exception('hashindex_delete failed') def get(self, key, default=None): try: return self[key] except KeyError: return default def pop(self, key, default=_NoDefault): try: value = self[key] del self[key] return value except KeyError: if default != _NoDefault: return default raise def __len__(self): return hashindex_get_size(self.index) cdef class NSIndex(IndexBase): value_size = 8 def __getitem__(self, key): assert len(key) == self.key_size data = hashindex_get(self.index, key) if not data: raise KeyError(key) cdef uint32_t segment = _le32toh(data[0]) assert segment <= _MAX_VALUE, "maximum number of segments reached" return segment, _le32toh(data[1]) def __setitem__(self, key, value): assert len(key) == self.key_size cdef uint32_t[2] data cdef uint32_t segment = value[0] assert segment <= _MAX_VALUE, "maximum number of segments reached" data[0] = _htole32(segment) data[1] = _htole32(value[1]) if not hashindex_set(self.index, key, data): raise Exception('hashindex_set failed') def __contains__(self, key): cdef uint32_t segment assert len(key) == self.key_size data = hashindex_get(self.index, key) if data != NULL: segment = _le32toh(data[0]) assert segment <= _MAX_VALUE, "maximum number of segments reached" return data != NULL def iteritems(self, marker=None): cdef const void *key iter = NSKeyIterator(self.key_size) iter.idx = self iter.index = self.index if marker: key = hashindex_get(self.index, marker) if marker is None: raise IndexError iter.key = key - self.key_size return iter cdef class NSKeyIterator: cdef NSIndex idx cdef HashIndex *index cdef const void *key cdef int key_size cdef int exhausted def __cinit__(self, key_size): self.key = NULL self.key_size = key_size self.exhausted = 0 def __iter__(self): return self def __next__(self): if self.exhausted: raise StopIteration self.key = hashindex_next_key(self.index, self.key) if not self.key: self.exhausted = 1 raise StopIteration cdef uint32_t *value = (self.key + self.key_size) cdef uint32_t segment = _le32toh(value[0]) assert segment <= _MAX_VALUE, "maximum number of segments reached" return (self.key)[:self.key_size], (segment, _le32toh(value[1])) cdef class ChunkIndex(IndexBase): """ Mapping of 32 byte keys to (refcount, size, csize), which are all 32-bit unsigned. The reference count cannot overflow. If an overflow would occur, the refcount is fixed to MAX_VALUE and will neither increase nor decrease by incref(), decref() or add(). Prior signed 32-bit overflow is handled correctly for most cases: All values from UINT32_MAX (2**32-1, inclusive) to MAX_VALUE (exclusive) are reserved and either cause silent data loss (-1, -2) or will raise an AssertionError when accessed. Other values are handled correctly. Note that previously the refcount could also reach 0 by *increasing* it. Assigning refcounts in this reserved range is an invalid operation and raises AssertionError. """ value_size = 12 def __getitem__(self, key): assert len(key) == self.key_size data = hashindex_get(self.index, key) if not data: raise KeyError(key) cdef uint32_t refcount = _le32toh(data[0]) assert refcount <= _MAX_VALUE, "invalid reference count" return refcount, _le32toh(data[1]), _le32toh(data[2]) def __setitem__(self, key, value): assert len(key) == self.key_size cdef uint32_t[3] data cdef uint32_t refcount = value[0] assert refcount <= _MAX_VALUE, "invalid reference count" data[0] = _htole32(refcount) data[1] = _htole32(value[1]) data[2] = _htole32(value[2]) if not hashindex_set(self.index, key, data): raise Exception('hashindex_set failed') def __contains__(self, key): assert len(key) == self.key_size data = hashindex_get(self.index, key) if data != NULL: assert _le32toh(data[0]) <= _MAX_VALUE, "invalid reference count" return data != NULL def incref(self, key): """Increase refcount for 'key', return (refcount, size, csize)""" assert len(key) == self.key_size data = hashindex_get(self.index, key) if not data: raise KeyError(key) cdef uint32_t refcount = _le32toh(data[0]) assert refcount <= _MAX_VALUE, "invalid reference count" if refcount != _MAX_VALUE: refcount += 1 data[0] = _htole32(refcount) return refcount, _le32toh(data[1]), _le32toh(data[2]) def decref(self, key): """Decrease refcount for 'key', return (refcount, size, csize)""" assert len(key) == self.key_size data = hashindex_get(self.index, key) if not data: raise KeyError(key) cdef uint32_t refcount = _le32toh(data[0]) # Never decrease a reference count of zero assert 0 < refcount <= _MAX_VALUE, "invalid reference count" if refcount != _MAX_VALUE: refcount -= 1 data[0] = _htole32(refcount) return refcount, _le32toh(data[1]), _le32toh(data[2]) def iteritems(self, marker=None): cdef const void *key iter = ChunkKeyIterator(self.key_size) iter.idx = self iter.index = self.index if marker: key = hashindex_get(self.index, marker) if marker is None: raise IndexError iter.key = key - self.key_size return iter def summarize(self): cdef uint64_t size = 0, csize = 0, unique_size = 0, unique_csize = 0, chunks = 0, unique_chunks = 0 cdef uint32_t *values cdef uint32_t refcount cdef void *key = NULL while True: key = hashindex_next_key(self.index, key) if not key: break unique_chunks += 1 values = (key + self.key_size) refcount = _le32toh(values[0]) assert refcount <= _MAX_VALUE, "invalid reference count" chunks += refcount unique_size += _le32toh(values[1]) unique_csize += _le32toh(values[2]) size += _le32toh(values[1]) * _le32toh(values[0]) csize += _le32toh(values[2]) * _le32toh(values[0]) return size, csize, unique_size, unique_csize, unique_chunks, chunks def add(self, key, refs, size, csize): assert len(key) == self.key_size cdef uint32_t[3] data data[0] = _htole32(refs) data[1] = _htole32(size) data[2] = _htole32(csize) self._add( key, data) cdef _add(self, void *key, uint32_t *data): cdef uint64_t refcount1, refcount2, result64 values = hashindex_get(self.index, key) if values: refcount1 = _le32toh(values[0]) refcount2 = _le32toh(data[0]) assert refcount1 <= _MAX_VALUE, "invalid reference count" assert refcount2 <= _MAX_VALUE, "invalid reference count" result64 = refcount1 + refcount2 values[0] = _htole32(min(result64, _MAX_VALUE)) else: if not hashindex_set(self.index, key, data): raise Exception('hashindex_set failed') def merge(self, ChunkIndex other): cdef void *key = NULL while True: key = hashindex_next_key(other.index, key) if not key: break self._add(key, (key + self.key_size)) cdef class ChunkKeyIterator: cdef ChunkIndex idx cdef HashIndex *index cdef const void *key cdef int key_size cdef int exhausted def __cinit__(self, key_size): self.key = NULL self.key_size = key_size self.exhausted = 0 def __iter__(self): return self def __next__(self): if self.exhausted: raise StopIteration self.key = hashindex_next_key(self.index, self.key) if not self.key: self.exhausted = 1 raise StopIteration cdef uint32_t *value = (self.key + self.key_size) cdef uint32_t refcount = _le32toh(value[0]) assert refcount <= _MAX_VALUE, "invalid reference count" return (self.key)[:self.key_size], (refcount, _le32toh(value[1]), _le32toh(value[2])) borgbackup-1.0.13/borg/helpers.py0000644000175000017500000016207213431513133016643 0ustar useruser00000000000000import argparse from binascii import hexlify from collections import namedtuple import contextlib import enum from functools import wraps import grp import os import stat import textwrap import pwd import re from shutil import get_terminal_size import sys import platform import signal import threading import time import unicodedata import io import errno import logging from .logger import create_logger logger = create_logger() from datetime import datetime, timezone, timedelta from fnmatch import translate from operator import attrgetter from . import __version__ as borg_version from . import __version_tuple__ as borg_version_tuple from . import hashindex from . import chunker from . import crypto from . import shellpattern import msgpack import msgpack.fallback import socket # never use datetime.isoformat(), it is evil. always use one of these: # datetime.strftime(ISO_FORMAT) # output always includes .microseconds # datetime.strftime(ISO_FORMAT_NO_USECS) # output never includes microseconds ISO_FORMAT_NO_USECS = '%Y-%m-%dT%H:%M:%S' ISO_FORMAT = ISO_FORMAT_NO_USECS + '.%f' # 20 MiB minus 41 bytes for a Repository header (because the "size" field in the Repository includes # the header, and the total size was set to 20 MiB). MAX_DATA_SIZE = 20971479 # MAX_OBJECT_SIZE = <20 MiB (MAX_DATA_SIZE) + 41 bytes for a Repository PUT header, which consists of # a 1 byte tag ID, 4 byte CRC, 4 byte size and 32 bytes for the ID. MAX_OBJECT_SIZE = MAX_DATA_SIZE + 41 # see LoggedIO.put_header_fmt.size assertion in repository module assert MAX_OBJECT_SIZE == 20971520 == 20 * 1024 * 1024 # borg.remote read() buffer size BUFSIZE = 10 * 1024 * 1024 # to use a safe, limited unpacker, we need to set a upper limit to the archive count in the manifest. # this does not mean that you can always really reach that number, because it also needs to be less than # MAX_DATA_SIZE or it will trigger the check for that. MAX_ARCHIVES = 400000 # repo.list() / .scan() result count limit the borg client uses LIST_SCAN_LIMIT = 10000 # return codes returned by borg command # when borg is killed by signal N, rc = 128 + N EXIT_SUCCESS = 0 # everything done, no problems EXIT_WARNING = 1 # reached normal end of operation, but there were issues EXIT_ERROR = 2 # terminated abruptly, did not reach end of operation ''' The global exit_code variable is used so that modules other than archiver can increase the program exit code if a warning or error occured during their operation. This is different from archiver.exit_code, which is only accessible from the archiver object. ''' exit_code = EXIT_SUCCESS def set_ec(ec): ''' Sets the exit code of the program, if an exit code higher or equal than this is set, this does nothing. This makes EXIT_ERROR override EXIT_WARNING, etc.. ec: exit code to set ''' global exit_code exit_code = max(exit_code, ec) return exit_code class Error(Exception): """Error base class""" # if we raise such an Error and it is only catched by the uppermost # exception handler (that exits short after with the given exit_code), # it is always a (fatal and abrupt) EXIT_ERROR, never just a warning. exit_code = EXIT_ERROR # show a traceback? traceback = False def __init__(self, *args): super().__init__(*args) self.args = args def get_message(self): return type(self).__doc__.format(*self.args) __str__ = get_message class ErrorWithTraceback(Error): """like Error, but show a traceback also""" traceback = True class IntegrityError(ErrorWithTraceback): """Data integrity error: {}""" class DecompressionError(IntegrityError): """Decompression error: {}""" class ExtensionModuleError(Error): """The Borg binary extension modules do not seem to be properly installed""" class NoManifestError(Error): """Repository has no manifest.""" class PlaceholderError(Error): """Formatting Error: "{}".format({}): {}({})""" class MandatoryFeatureUnsupported(Error): """Unsupported repository feature(s) {}. A newer version of borg is required to access this repository.""" class PythonLibcTooOld(Error): """FATAL: this Python was compiled for a too old (g)libc and misses required functionality.""" def check_python(): required_funcs = {os.stat, os.utime, os.chown} if not os.supports_follow_symlinks.issuperset(required_funcs): raise PythonLibcTooOld def check_extension_modules(): from . import platform, compress if hashindex.API_VERSION != '1.0_01': raise ExtensionModuleError if chunker.API_VERSION != '1.0_01': raise ExtensionModuleError if compress.API_VERSION != '1.0_01': raise ExtensionModuleError if crypto.API_VERSION != '1.0_01': raise ExtensionModuleError if platform.API_VERSION != '1.0_01': raise ExtensionModuleError def get_limited_unpacker(kind): """return a limited Unpacker because we should not trust msgpack data received from remote""" args = dict(use_list=False, # return tuples, not lists max_bin_len=0, # not used max_ext_len=0, # not used max_buffer_size=3 * max(BUFSIZE, MAX_OBJECT_SIZE), max_str_len=MAX_OBJECT_SIZE, # a chunk or other repo object ) if kind == 'server': args.update(dict(max_array_len=100, # misc. cmd tuples max_map_len=100, # misc. cmd dicts )) elif kind == 'client': args.update(dict(max_array_len=LIST_SCAN_LIMIT, # result list from repo.list() / .scan() max_map_len=100, # misc. result dicts )) elif kind == 'manifest': args.update(dict(use_list=True, # default value max_array_len=100, # ITEM_KEYS ~= 22 max_map_len=MAX_ARCHIVES, # list of archives max_str_len=255, # archive name object_hook=StableDict, unicode_errors='surrogateescape', )) elif kind == 'key': args.update(dict(use_list=True, # default value max_array_len=0, # not used max_map_len=10, # EncryptedKey dict max_str_len=4000, # inner key data object_hook=StableDict, unicode_errors='surrogateescape', )) else: raise ValueError('kind must be "server", "client", "manifest" or "key"') return msgpack.Unpacker(**args) class Manifest: @enum.unique class Operation(enum.Enum): # The comments here only roughly describe the scope of each feature. In the end, additions need to be # based on potential problems older clients could produce when accessing newer repositories and the # tradeofs of locking version out or still allowing access. As all older versions and their exact # behaviours are known when introducing new features sometimes this might not match the general descriptions # below. # The READ operation describes which features are needed to safely list and extract the archives in the # repository. READ = 'read' # The CHECK operation is for all operations that need either to understand every detail # of the repository (for consistency checks and repairs) or are seldom used functions that just # should use the most restrictive feature set because more fine grained compatibility tracking is # not needed. CHECK = 'check' # The WRITE operation is for adding archives. Features here ensure that older clients don't add archives # in an old format, or is used to lock out clients that for other reasons can no longer safely add new # archives. WRITE = 'write' # The DELETE operation is for all operations (like archive deletion) that need a 100% correct reference # count and the need to be able to find all (directly and indirectly) referenced chunks of a given archive. DELETE = 'delete' NO_OPERATION_CHECK = tuple() SUPPORTED_REPO_FEATURES = frozenset([]) MANIFEST_ID = b'\0' * 32 def __init__(self, key, repository, item_keys=None): from .archive import ITEM_KEYS self.archives = {} self.config = {} self.key = key self.repository = repository self.item_keys = frozenset(item_keys) if item_keys is not None else ITEM_KEYS self.tam_verified = False self.timestamp = None @classmethod def load(cls, repository, operations, key=None, force_tam_not_required=False): from .key import key_factory, tam_required_file, tam_required from .repository import Repository from .archive import ITEM_KEYS try: cdata = repository.get(cls.MANIFEST_ID) except Repository.ObjectNotFound: raise NoManifestError if not key: key = key_factory(repository, cdata) manifest = cls(key, repository) data = key.decrypt(None, cdata) m, manifest.tam_verified = key.unpack_and_verify_manifest(data, force_tam_not_required=force_tam_not_required) manifest.id = key.id_hash(data) if m.get(b'version') not in (1, 2): raise ValueError('Invalid manifest version') manifest.archives = dict((k.decode('utf-8'), v) for k, v in m[b'archives'].items()) manifest.timestamp = m.get(b'timestamp') if manifest.timestamp: manifest.timestamp = manifest.timestamp.decode('ascii') manifest.config = m[b'config'] # valid item keys are whatever is known in the repo or every key we know manifest.item_keys = frozenset(m.get(b'item_keys', [])) | ITEM_KEYS if manifest.tam_verified: manifest_required = manifest.config.get(b'tam_required', False) security_required = tam_required(repository) if manifest_required and not security_required: logger.debug('Manifest is TAM verified and says TAM is required, updating security database...') file = tam_required_file(repository) open(file, 'w').close() if not manifest_required and security_required: logger.debug('Manifest is TAM verified and says TAM is *not* required, updating security database...') os.unlink(tam_required_file(repository)) manifest.check_repository_compatibility(operations) return manifest, key def check_repository_compatibility(self, operations): for operation in operations: assert isinstance(operation, self.Operation) feature_flags = self.config.get(b'feature_flags', None) if feature_flags is None: return if operation.value.encode() not in feature_flags: continue requirements = feature_flags[operation.value.encode()] if b'mandatory' in requirements: unsupported = set(requirements[b'mandatory']) - self.SUPPORTED_REPO_FEATURES if unsupported: raise MandatoryFeatureUnsupported([f.decode() for f in unsupported]) def get_all_mandatory_features(self): result = {} feature_flags = self.config.get(b'feature_flags', None) if feature_flags is None: return result for operation, requirements in feature_flags.items(): if b'mandatory' in requirements: result[operation.decode()] = set([feature.decode() for feature in requirements[b'mandatory']]) return result def write(self): if self.key.tam_required: self.config[b'tam_required'] = True # self.timestamp needs to be strictly monotonically increasing. Clocks often are not set correctly if self.timestamp is None: self.timestamp = datetime.utcnow().strftime(ISO_FORMAT) else: prev_ts = parse_timestamp(self.timestamp, tzinfo=None) incremented = (prev_ts + timedelta(microseconds=1)).strftime(ISO_FORMAT) self.timestamp = max(incremented, datetime.utcnow().strftime(ISO_FORMAT)) # include checks for limits as enforced by limited unpacker (used by load()) assert len(self.archives) <= MAX_ARCHIVES assert all(len(name) <= 255 for name in self.archives) assert len(self.item_keys) <= 100 m = { 'version': 1, 'archives': StableDict((name, StableDict(archive)) for name, archive in self.archives.items()), 'timestamp': self.timestamp, 'config': StableDict(self.config), 'item_keys': tuple(sorted(self.item_keys)), } self.tam_verified = True data = self.key.pack_and_authenticate_metadata(m) self.id = self.key.id_hash(data) self.repository.put(self.MANIFEST_ID, self.key.encrypt(data, none_compression=True)) def list_archive_infos(self, sort_by=None, reverse=False): # inexpensive Archive.list_archives replacement if we just need .name, .id, .ts ArchiveInfo = namedtuple('ArchiveInfo', 'name id ts') archives = [] for name, values in self.archives.items(): ts = parse_timestamp(values[b'time'].decode('utf-8')) id = values[b'id'] archives.append(ArchiveInfo(name=name, id=id, ts=ts)) if sort_by is not None: archives = sorted(archives, key=attrgetter(sort_by), reverse=reverse) return archives def interval(s): """Convert a string representing a valid interval to a number of hours.""" multiplier = {'H': 1, 'd': 24, 'w': 24 * 7, 'm': 24 * 31, 'y': 24 * 365} if s.endswith(tuple(multiplier.keys())): number = s[:-1] suffix = s[-1] else: # range suffixes in ascending multiplier order ranges = [k for k, v in sorted(multiplier.items(), key=lambda t: t[1])] raise argparse.ArgumentTypeError( 'Unexpected interval time unit "%s": expected one of %r' % (s[-1], ranges)) try: hours = int(number) * multiplier[suffix] except ValueError: hours = -1 if hours <= 0: raise argparse.ArgumentTypeError( 'Unexpected interval number "%s": expected an integer greater than 0' % number) return hours def prune_within(archives, hours): target = datetime.now(timezone.utc) - timedelta(seconds=hours * 3600) return [a for a in archives if a.ts > target] def prune_split(archives, pattern, n, skip=[]): last = None keep = [] if n == 0: return keep for a in sorted(archives, key=attrgetter('ts'), reverse=True): period = to_localtime(a.ts).strftime(pattern) if period != last: last = period if a not in skip: keep.append(a) if len(keep) == n: break return keep class Statistics: def __init__(self): self.osize = self.csize = self.usize = self.nfiles = 0 self.last_progress = 0 # timestamp when last progress was shown def update(self, size, csize, unique): self.osize += size self.csize += csize if unique: self.usize += csize summary = """\ Original size Compressed size Deduplicated size {label:15} {stats.osize_fmt:>20s} {stats.csize_fmt:>20s} {stats.usize_fmt:>20s}""" def __str__(self): return self.summary.format(stats=self, label='This archive:') def __repr__(self): return "<{cls} object at {hash:#x} ({self.osize}, {self.csize}, {self.usize})>".format(cls=type(self).__name__, hash=id(self), self=self) @property def osize_fmt(self): return format_file_size(self.osize) @property def usize_fmt(self): return format_file_size(self.usize) @property def csize_fmt(self): return format_file_size(self.csize) def show_progress(self, item=None, final=False, stream=None, dt=None): now = time.monotonic() if dt is None or now - self.last_progress > dt: self.last_progress = now columns, lines = get_terminal_size() if not final: msg = '{0.osize_fmt} O {0.csize_fmt} C {0.usize_fmt} D {0.nfiles} N '.format(self) path = remove_surrogates(item[b'path']) if item else '' space = columns - len(msg) if space < 12: msg = '' space = columns - len(msg) if space >= 8: if space < len('...') + len(path): path = '%s...%s' % (path[:(space // 2) - len('...')], path[-space // 2:]) msg += "{0:<{space}}".format(path, space=space) else: msg = ' ' * columns print(msg, file=stream or sys.stderr, end="\r", flush=True) def get_keys_dir(): """Determine where to repository keys and cache""" xdg_config = os.environ.get('XDG_CONFIG_HOME', os.path.join(os.path.expanduser('~'), '.config')) keys_dir = os.environ.get('BORG_KEYS_DIR', os.path.join(xdg_config, 'borg', 'keys')) if not os.path.exists(keys_dir): os.makedirs(keys_dir) os.chmod(keys_dir, stat.S_IRWXU) return keys_dir def get_security_dir(repository_id=None): """Determine where to store local security information.""" xdg_config = os.environ.get('XDG_CONFIG_HOME', os.path.join(os.path.expanduser('~'), '.config')) security_dir = os.environ.get('BORG_SECURITY_DIR', os.path.join(xdg_config, 'borg', 'security')) if repository_id: security_dir = os.path.join(security_dir, repository_id) if not os.path.exists(security_dir): os.makedirs(security_dir) os.chmod(security_dir, stat.S_IRWXU) return security_dir def get_cache_dir(): """Determine where to repository keys and cache""" xdg_cache = os.environ.get('XDG_CACHE_HOME', os.path.join(os.path.expanduser('~'), '.cache')) cache_dir = os.environ.get('BORG_CACHE_DIR', os.path.join(xdg_cache, 'borg')) if not os.path.exists(cache_dir): os.makedirs(cache_dir) os.chmod(cache_dir, stat.S_IRWXU) with open(os.path.join(cache_dir, 'CACHEDIR.TAG'), 'w') as fd: fd.write(textwrap.dedent(""" Signature: 8a477f597d28d172789f06886806bc55 # This file is a cache directory tag created by Borg. # For information about cache directory tags, see: # http://www.brynosaurus.com/cachedir/ """).lstrip()) return cache_dir def to_localtime(ts): """Convert datetime object from UTC to local time zone""" return datetime(*time.localtime((ts - datetime(1970, 1, 1, tzinfo=timezone.utc)).total_seconds())[:6]) def parse_timestamp(timestamp, tzinfo=timezone.utc): """Parse a ISO 8601 timestamp string""" fmt = ISO_FORMAT if '.' in timestamp else ISO_FORMAT_NO_USECS dt = datetime.strptime(timestamp, fmt) if tzinfo is not None: dt = dt.replace(tzinfo=tzinfo) return dt def load_excludes(fh): """Load and parse exclude patterns from file object. Lines empty or starting with '#' after stripping whitespace on both line ends are ignored. """ patterns = (line for line in (i.strip() for i in fh) if not line.startswith('#')) return [parse_pattern(pattern) for pattern in patterns if pattern] def update_excludes(args): """Merge exclude patterns from files with those on command line.""" if hasattr(args, 'exclude_files') and args.exclude_files: if not hasattr(args, 'excludes') or args.excludes is None: args.excludes = [] for file in args.exclude_files: args.excludes += load_excludes(file) file.close() class PatternMatcher: def __init__(self, fallback=None): self._items = [] # Value to return from match function when none of the patterns match. self.fallback = fallback def add(self, patterns, value): """Add list of patterns to internal list. The given value is returned from the match function when one of the given patterns matches. """ self._items.extend((i, value) for i in patterns) def match(self, path): for (pattern, value) in self._items: if pattern.match(path): return value return self.fallback def normalized(func): """ Decorator for the Pattern match methods, returning a wrapper that normalizes OSX paths to match the normalized pattern on OSX, and returning the original method on other platforms""" @wraps(func) def normalize_wrapper(self, path): return func(self, unicodedata.normalize("NFD", path)) if sys.platform in ('darwin',): # HFS+ converts paths to a canonical form, so users shouldn't be # required to enter an exact match return normalize_wrapper else: # Windows and Unix filesystems allow different forms, so users # always have to enter an exact match return func class PatternBase: """Shared logic for inclusion/exclusion patterns. """ PREFIX = NotImplemented def __init__(self, pattern): self.pattern_orig = pattern self.match_count = 0 if sys.platform in ('darwin',): pattern = unicodedata.normalize("NFD", pattern) self._prepare(pattern) @normalized def match(self, path): matches = self._match(path) if matches: self.match_count += 1 return matches def __repr__(self): return '%s(%s)' % (type(self), self.pattern) def __str__(self): return self.pattern_orig def _prepare(self, pattern): raise NotImplementedError def _match(self, path): raise NotImplementedError # For PathPrefixPattern, FnmatchPattern and ShellPattern, we require that the pattern either match the whole path # or an initial segment of the path up to but not including a path separator. To unify the two cases, we add a path # separator to the end of the path before matching. class PathPrefixPattern(PatternBase): """Literal files or directories listed on the command line for some operations (e.g. extract, but not create). If a directory is specified, all paths that start with that path match as well. A trailing slash makes no difference. """ PREFIX = "pp" def _prepare(self, pattern): self.pattern = os.path.normpath(pattern).rstrip(os.path.sep) + os.path.sep def _match(self, path): return (path + os.path.sep).startswith(self.pattern) class FnmatchPattern(PatternBase): """Shell glob patterns to exclude. A trailing slash means to exclude the contents of a directory, but not the directory itself. """ PREFIX = "fm" def _prepare(self, pattern): if pattern.endswith(os.path.sep): pattern = os.path.normpath(pattern).rstrip(os.path.sep) + os.path.sep + '*' + os.path.sep else: pattern = os.path.normpath(pattern) + os.path.sep + '*' self.pattern = pattern # fnmatch and re.match both cache compiled regular expressions. # Nevertheless, this is about 10 times faster. self.regex = re.compile(translate(self.pattern)) def _match(self, path): return (self.regex.match(path + os.path.sep) is not None) class ShellPattern(PatternBase): """Shell glob patterns to exclude. A trailing slash means to exclude the contents of a directory, but not the directory itself. """ PREFIX = "sh" def _prepare(self, pattern): sep = os.path.sep if pattern.endswith(sep): pattern = os.path.normpath(pattern).rstrip(sep) + sep + "**" + sep + "*" + sep else: pattern = os.path.normpath(pattern) + sep + "**" + sep + "*" self.pattern = pattern self.regex = re.compile(shellpattern.translate(self.pattern)) def _match(self, path): return (self.regex.match(path + os.path.sep) is not None) class RegexPattern(PatternBase): """Regular expression to exclude. """ PREFIX = "re" def _prepare(self, pattern): self.pattern = pattern self.regex = re.compile(pattern) def _match(self, path): # Normalize path separators if os.path.sep != '/': path = path.replace(os.path.sep, '/') return (self.regex.search(path) is not None) _PATTERN_STYLES = set([ FnmatchPattern, PathPrefixPattern, RegexPattern, ShellPattern, ]) _PATTERN_STYLE_BY_PREFIX = dict((i.PREFIX, i) for i in _PATTERN_STYLES) def parse_pattern(pattern, fallback=FnmatchPattern): """Read pattern from string and return an instance of the appropriate implementation class. """ if len(pattern) > 2 and pattern[2] == ":" and pattern[:2].isalnum(): (style, pattern) = (pattern[:2], pattern[3:]) cls = _PATTERN_STYLE_BY_PREFIX.get(style, None) if cls is None: raise ValueError("Unknown pattern style: {}".format(style)) else: cls = fallback return cls(pattern) def timestamp(s): """Convert a --timestamp=s argument to a datetime object""" try: # is it pointing to a file / directory? ts = safe_s(os.stat(s).st_mtime) return datetime.utcfromtimestamp(ts) except OSError: # didn't work, try parsing as timestamp. UTC, no TZ, no microsecs support. for format in ('%Y-%m-%dT%H:%M:%SZ', '%Y-%m-%dT%H:%M:%S+00:00', '%Y-%m-%dT%H:%M:%S', '%Y-%m-%d %H:%M:%S', '%Y-%m-%dT%H:%M', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%Y-%j', ): try: return datetime.strptime(s, format) except ValueError: continue raise ValueError def ChunkerParams(s): chunk_min, chunk_max, chunk_mask, window_size = s.split(',') if int(chunk_max) > 23: raise ValueError('max. chunk size exponent must not be more than 23 (2^23 = 8MiB max. chunk size)') return int(chunk_min), int(chunk_max), int(chunk_mask), int(window_size) def CompressionSpec(s): values = s.split(',') count = len(values) if count < 1: raise ValueError # --compression algo[,level] name = values[0] if name in ('none', 'lz4', ): return dict(name=name) if name in ('zlib', 'lzma', ): if count < 2: level = 6 # default compression level in py stdlib elif count == 2: level = int(values[1]) if not 0 <= level <= 9: raise ValueError else: raise ValueError return dict(name=name, level=level) raise ValueError def PrefixSpec(s): return replace_placeholders(s) def dir_is_cachedir(path): """Determines whether the specified path is a cache directory (and therefore should potentially be excluded from the backup) according to the CACHEDIR.TAG protocol (http://www.brynosaurus.com/cachedir/spec.html). """ tag_contents = b'Signature: 8a477f597d28d172789f06886806bc55' tag_path = os.path.join(path, 'CACHEDIR.TAG') try: if os.path.exists(tag_path): with open(tag_path, 'rb') as tag_file: tag_data = tag_file.read(len(tag_contents)) if tag_data == tag_contents: return True except OSError: pass return False def dir_is_tagged(path, exclude_caches, exclude_if_present): """Determines whether the specified path is excluded by being a cache directory or containing user-specified tag files. Returns a list of the paths of the tag files (either CACHEDIR.TAG or the matching user-specified files). """ tag_paths = [] if exclude_caches and dir_is_cachedir(path): tag_paths.append(os.path.join(path, 'CACHEDIR.TAG')) if exclude_if_present is not None: for tag in exclude_if_present: tag_path = os.path.join(path, tag) if os.path.isfile(tag_path): tag_paths.append(tag_path) return tag_paths def format_line(format, data): try: return format.format(**data) except Exception as e: raise PlaceholderError(format, data, e.__class__.__name__, str(e)) def replace_placeholders(text): """Replace placeholders in text with their values.""" current_time = datetime.now() data = { 'pid': os.getpid(), 'fqdn': socket.getfqdn(), 'hostname': socket.gethostname(), 'now': current_time.now(), 'utcnow': current_time.utcnow(), 'user': uid2user(os.getuid(), os.getuid()), 'borgversion': borg_version, 'borgmajor': '%d' % borg_version_tuple[:1], 'borgminor': '%d.%d' % borg_version_tuple[:2], 'borgpatch': '%d.%d.%d' % borg_version_tuple[:3], } return format_line(text, data) # Not too rarely, we get crappy timestamps from the fs, that overflow some computations. # As they are crap anyway (valid filesystem timestamps always refer to the past up to # the present, but never to the future), nothing is lost if we just clamp them to the # maximum value we can support. # As long as people are using borg on 32bit platforms to access borg archives, we must # keep this value True. But we can expect that we can stop supporting 32bit platforms # well before coming close to the year 2038, so this will never be a practical problem. SUPPORT_32BIT_PLATFORMS = True # set this to False before y2038. if SUPPORT_32BIT_PLATFORMS: # second timestamps will fit into a signed int32 (platform time_t limit). # nanosecond timestamps thus will naturally fit into a signed int64. # subtract last 48h to avoid any issues that could be caused by tz calculations. # this is in the year 2038, so it is also less than y9999 (which is a datetime internal limit). # msgpack can pack up to uint64. MAX_S = 2**31-1 - 48*3600 MAX_NS = MAX_S * 1000000000 else: # nanosecond timestamps will fit into a signed int64. # subtract last 48h to avoid any issues that could be caused by tz calculations. # this is in the year 2262, so it is also less than y9999 (which is a datetime internal limit). # round down to 1e9 multiple, so MAX_NS corresponds precisely to a integer MAX_S. # msgpack can pack up to uint64. MAX_NS = (2**63-1 - 48*3600*1000000000) // 1000000000 * 1000000000 MAX_S = MAX_NS // 1000000000 def safe_s(ts): if 0 <= ts <= MAX_S: return ts elif ts < 0: return 0 else: return MAX_S def safe_ns(ts): if 0 <= ts <= MAX_NS: return ts elif ts < 0: return 0 else: return MAX_NS def safe_timestamp(item_timestamp_ns): t_ns = safe_ns(bigint_to_int(item_timestamp_ns)) return datetime.fromtimestamp(t_ns / 1e9) def format_time(t): """use ISO-8601 date and time format """ return t.strftime('%a, %Y-%m-%d %H:%M:%S') def format_timedelta(td): """Format timedelta in a human friendly format """ # Since td.total_seconds() requires python 2.7 ts = (td.microseconds + (td.seconds + td.days * 24 * 3600) * 10 ** 6) / float(10 ** 6) s = ts % 60 m = int(ts / 60) % 60 h = int(ts / 3600) % 24 txt = '%.2f seconds' % s if m: txt = '%d minutes %s' % (m, txt) if h: txt = '%d hours %s' % (h, txt) if td.days: txt = '%d days %s' % (td.days, txt) return txt def format_file_size(v, precision=2): """Format file size into a human friendly format """ return sizeof_fmt_decimal(v, suffix='B', sep=' ', precision=precision) def sizeof_fmt(num, suffix='B', units=None, power=None, sep='', precision=2): for unit in units[:-1]: if abs(round(num, precision)) < power: if isinstance(num, int): return "{}{}{}{}".format(num, sep, unit, suffix) else: return "{:3.{}f}{}{}{}".format(num, precision, sep, unit, suffix) num /= float(power) return "{:.{}f}{}{}{}".format(num, precision, sep, units[-1], suffix) def sizeof_fmt_iec(num, suffix='B', sep='', precision=2): return sizeof_fmt(num, suffix=suffix, sep=sep, precision=precision, units=['', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi', 'Yi'], power=1024) def sizeof_fmt_decimal(num, suffix='B', sep='', precision=2): return sizeof_fmt(num, suffix=suffix, sep=sep, precision=precision, units=['', 'k', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y'], power=1000) def format_archive(archive): return '%-36s %s' % (archive.name, format_time(to_localtime(archive.ts))) def memoize(function): cache = {} def decorated_function(*args): try: return cache[args] except KeyError: val = function(*args) cache[args] = val return val return decorated_function class Buffer: """ Provides a managed, resizable buffer. """ class MemoryLimitExceeded(Error, OSError): """Requested buffer size {} is above the limit of {}.""" def __init__(self, allocator, size=4096, limit=None): """ Initialize the buffer: use allocator(size) call to allocate a buffer. Optionally, set the upper for the buffer size. """ assert callable(allocator), 'must give alloc(size) function as first param' assert limit is None or size <= limit, 'initial size must be <= limit' self.allocator = allocator self.limit = limit self.resize(size, init=True) def __len__(self): return len(self.buffer) def resize(self, size, init=False): """ resize the buffer - to avoid frequent reallocation, we usually always grow (if needed). giving init=True it is possible to first-time initialize or shrink the buffer. if a buffer size beyond the limit is requested, raise Buffer.MemoryLimitExceeded (OSError). """ size = int(size) if self.limit is not None and size > self.limit: raise Buffer.MemoryLimitExceeded(size, self.limit) if init or len(self) < size: self.buffer = self.allocator(size) def get(self, size=None, init=False): """ return a buffer of at least the requested size (None: any current size). init=True can be given to trigger shrinking of the buffer to the given size. """ if size is not None: self.resize(size, init) return self.buffer @memoize def uid2user(uid, default=None): try: return pwd.getpwuid(uid).pw_name except KeyError: return default @memoize def user2uid(user, default=None): try: return user and pwd.getpwnam(user).pw_uid except KeyError: return default @memoize def gid2group(gid, default=None): try: return grp.getgrgid(gid).gr_name except KeyError: return default @memoize def group2gid(group, default=None): try: return group and grp.getgrnam(group).gr_gid except KeyError: return default def posix_acl_use_stored_uid_gid(acl): """Replace the user/group field with the stored uid/gid """ entries = [] for entry in safe_decode(acl).split('\n'): if entry: fields = entry.split(':') if len(fields) == 4: entries.append(':'.join([fields[0], fields[3], fields[2]])) else: entries.append(entry) return safe_encode('\n'.join(entries)) def safe_decode(s, coding='utf-8', errors='surrogateescape'): """decode bytes to str, with round-tripping "invalid" bytes""" return s.decode(coding, errors) def safe_encode(s, coding='utf-8', errors='surrogateescape'): """encode str to bytes, with round-tripping "invalid" bytes""" return s.encode(coding, errors) def bin_to_hex(binary): return hexlify(binary).decode('ascii') def parse_stringified_list(s): l = re.split(" *, *", s) return [item for item in l if item != ''] class Location: """Object representing a repository / archive location """ proto = user = _host = port = path = archive = None # user must not contain "@", ":" or "/". # Quoting adduser error message: # "To avoid problems, the username should consist only of letters, digits, # underscores, periods, at signs and dashes, and not start with a dash # (as defined by IEEE Std 1003.1-2001)." # We use "@" as separator between username and hostname, so we must # disallow it within the pure username part. optional_user_re = r""" (?:(?P[^@:/]+)@)? """ # path must not contain :: (it ends at :: or string end), but may contain single colons. # to avoid ambiguities with other regexes, it must also not start with ":" nor with "//" nor with "ssh://". scp_path_re = r""" (?!(:|//|ssh://)) # not starting with ":" or // or ssh:// (?P([^:]|(:(?!:)))+) # any chars, but no "::" """ # file_path must not contain :: (it ends at :: or string end), but may contain single colons. # it must start with a / and that slash is part of the path. file_path_re = r""" (?P(([^/]*)/([^:]|(:(?!:)))+)) # start opt. servername, then /, then any chars, but no "::" """ # abs_path must not contain :: (it ends at :: or string end), but may contain single colons. # it must start with a / and that slash is part of the path. abs_path_re = r""" (?P(/([^:]|(:(?!:)))+)) # start with /, then any chars, but no "::" """ # optional ::archive_name at the end, archive name must not contain "/". # borg mount's FUSE filesystem creates one level of directories from # the archive names and of course "/" is not valid in a directory name. optional_archive_re = r""" (?: :: # "::" as separator (?P[^/]+) # archive name must not contain "/" )?$""" # must match until the end # regexes for misc. kinds of supported location specifiers: ssh_re = re.compile(r""" (?Pssh):// # ssh:// """ + optional_user_re + r""" # user@ (optional) (?P([^:/]+|\[[0-9a-fA-F:.]+\]))(?::(?P\d+))? # host or host:port or [ipv6] or [ipv6]:port """ + abs_path_re + optional_archive_re, re.VERBOSE) # path or path::archive file_re = re.compile(r""" (?Pfile):// # file:// """ + file_path_re + optional_archive_re, re.VERBOSE) # servername/path, path or path::archive # note: scp_re is also use for local pathes scp_re = re.compile(r""" ( """ + optional_user_re + r""" # user@ (optional) (?P([^:/]+|\[[0-9a-fA-F:.]+\])): # host: (don't match / or [ipv6] in host to disambiguate from file:) )? # user@host: part is optional """ + scp_path_re + optional_archive_re, re.VERBOSE) # path with optional archive # get the repo from BORG_REPO env and the optional archive from param. # if the syntax requires giving REPOSITORY (see "borg mount"), # use "::" to let it use the env var. # if REPOSITORY argument is optional, it'll automatically use the env. env_re = re.compile(r""" # the repo part is fetched from BORG_REPO (?:::$) # just "::" is ok (when a pos. arg is required, no archive) | # or """ + optional_archive_re, re.VERBOSE) # archive name (optional, may be empty) def __init__(self, text=''): self.orig = text if not self.parse(self.orig): raise ValueError('Location: parse failed: %s' % self.orig) def parse(self, text): text = replace_placeholders(text) valid = self._parse(text) if valid: return True m = self.env_re.match(text) if not m: return False repo = os.environ.get('BORG_REPO') if repo is None: return False valid = self._parse(repo) if not valid: return False self.archive = m.group('archive') return True def _parse(self, text): def normpath_special(p): # avoid that normpath strips away our relative path hack and even makes p absolute relative = p.startswith('/./') p = os.path.normpath(p) return ('/.' + p) if relative else p m = self.ssh_re.match(text) if m: self.proto = m.group('proto') self.user = m.group('user') self._host = m.group('host') self.port = m.group('port') and int(m.group('port')) or None self.path = normpath_special(m.group('path')) self.archive = m.group('archive') return True m = self.file_re.match(text) if m: self.proto = m.group('proto') self.path = normpath_special(m.group('path')) self.archive = m.group('archive') return True m = self.scp_re.match(text) if m: self.user = m.group('user') self._host = m.group('host') self.path = normpath_special(m.group('path')) self.archive = m.group('archive') self.proto = self._host and 'ssh' or 'file' return True return False def __str__(self): items = [ 'proto=%r' % self.proto, 'user=%r' % self.user, 'host=%r' % self.host, 'port=%r' % self.port, 'path=%r' % self.path, 'archive=%r' % self.archive, ] return ', '.join(items) def to_key_filename(self): name = re.sub('[^\w]', '_', self.path).strip('_') if self.proto != 'file': name = re.sub('[^\w]', '_', self.host) + '__' + name if len(name) > 100: # Limit file names to some reasonable length. Most file systems # limit them to 255 [unit of choice]; due to variations in unicode # handling we truncate to 100 *characters*. name = name[:100] return os.path.join(get_keys_dir(), name) def __repr__(self): return "Location(%s)" % self @property def host(self): # strip square brackets used for IPv6 addrs if self._host is not None: return self._host.lstrip('[').rstrip(']') def canonical_path(self): if self.proto == 'file': return self.path else: if self.path and self.path.startswith('~'): path = '/' + self.path # /~/x = path x relative to home dir elif self.path and not self.path.startswith('/'): path = '/./' + self.path # /./x = path x relative to cwd else: path = self.path return 'ssh://{}{}{}{}'.format('{}@'.format(self.user) if self.user else '', self._host, # needed for ipv6 addrs ':{}'.format(self.port) if self.port else '', path) def location_validator(archive=None): def validator(text): try: loc = Location(text) except ValueError: raise argparse.ArgumentTypeError('Invalid location format: "%s"' % text) from None if archive is True and not loc.archive: raise argparse.ArgumentTypeError('"%s": No archive specified' % text) elif archive is False and loc.archive: raise argparse.ArgumentTypeError('"%s" No archive can be specified' % text) return loc return validator def archivename_validator(): def validator(text): if '/' in text or '::' in text or not text: raise argparse.ArgumentTypeError('Invalid repository name: "%s"' % text) return text return validator def decode_dict(d, keys, encoding='utf-8', errors='surrogateescape'): for key in keys: if isinstance(d.get(key), bytes): d[key] = d[key].decode(encoding, errors) return d def remove_surrogates(s, errors='replace'): """Replace surrogates generated by fsdecode with '?' """ return s.encode('utf-8', errors).decode('utf-8') _safe_re = re.compile(r'^((\.\.)?/+)+') def make_path_safe(path): """Make path safe by making it relative and local """ return _safe_re.sub('', path) or '.' def daemonize(): """Detach process from controlling terminal and run in background Returns: old and new get_process_id tuples """ from .locking import get_id as get_process_id old_id = get_process_id() pid = os.fork() if pid: os._exit(0) os.setsid() pid = os.fork() if pid: os._exit(0) os.chdir('/') os.close(0) os.close(1) os.close(2) fd = os.open('/dev/null', os.O_RDWR) os.dup2(fd, 0) os.dup2(fd, 1) os.dup2(fd, 2) new_id = get_process_id() return old_id, new_id class StableDict(dict): """A dict subclass with stable items() ordering""" def items(self): return sorted(super().items()) def bigint_to_int(mtime): """Convert bytearray to int """ if isinstance(mtime, bytes): return int.from_bytes(mtime, 'little', signed=True) return mtime def int_to_bigint(value): """Convert integers larger than 64 bits to bytearray Smaller integers are left alone """ if value.bit_length() > 63: return value.to_bytes((value.bit_length() + 9) // 8, 'little', signed=True) return value def is_slow_msgpack(): return msgpack.Packer is msgpack.fallback.Packer FALSISH = ('No', 'NO', 'no', 'N', 'n', '0', ) TRUISH = ('Yes', 'YES', 'yes', 'Y', 'y', '1', ) DEFAULTISH = ('Default', 'DEFAULT', 'default', 'D', 'd', '', ) def yes(msg=None, false_msg=None, true_msg=None, default_msg=None, retry_msg=None, invalid_msg=None, env_msg='{} (from {})', falsish=FALSISH, truish=TRUISH, defaultish=DEFAULTISH, default=False, retry=True, env_var_override=None, ofile=None, input=input): """Output (usually a question) and let user input an answer. Qualifies the answer according to falsish, truish and defaultish as True, False or . If it didn't qualify and retry is False (no retries wanted), return the default [which defaults to False]. If retry is True let user retry answering until answer is qualified. If env_var_override is given and this var is present in the environment, do not ask the user, but just use the env var contents as answer as if it was typed in. Otherwise read input from stdin and proceed as normal. If EOF is received instead an input or an invalid input without retry possibility, return default. :param msg: introducing message to output on ofile, no \n is added [None] :param retry_msg: retry message to output on ofile, no \n is added [None] :param false_msg: message to output before returning False [None] :param true_msg: message to output before returning True [None] :param default_msg: message to output before returning a [None] :param invalid_msg: message to output after a invalid answer was given [None] :param env_msg: message to output when using input from env_var_override ['{} (from {})'], needs to have 2 placeholders for answer and env var name :param falsish: sequence of answers qualifying as False :param truish: sequence of answers qualifying as True :param defaultish: sequence of answers qualifying as :param default: default return value (defaultish answer was given or no-answer condition) [False] :param retry: if True and input is incorrect, retry. Otherwise return default. [True] :param env_var_override: environment variable name [None] :param ofile: output stream [sys.stderr] :param input: input function [input from builtins] :return: boolean answer value, True or False """ # note: we do not assign sys.stderr as default above, so it is # really evaluated NOW, not at function definition time. if ofile is None: ofile = sys.stderr if default not in (True, False): raise ValueError("invalid default value, must be True or False") if msg: print(msg, file=ofile, end='', flush=True) while True: answer = None if env_var_override: answer = os.environ.get(env_var_override) if answer is not None and env_msg: print(env_msg.format(answer, env_var_override), file=ofile) if answer is None: try: answer = input() except EOFError: # avoid defaultish[0], defaultish could be empty answer = truish[0] if default else falsish[0] if answer in defaultish: if default_msg: print(default_msg, file=ofile) return default if answer in truish: if true_msg: print(true_msg, file=ofile) return True if answer in falsish: if false_msg: print(false_msg, file=ofile) return False # if we get here, the answer was invalid if invalid_msg: print(invalid_msg, file=ofile) if not retry: return default if retry_msg: print(retry_msg, file=ofile, end='', flush=True) # in case we used an environment variable and it gave an invalid answer, do not use it again: env_var_override = None class ProgressIndicatorPercent: def __init__(self, total, step=5, start=0, same_line=False, msg="%3.0f%%", file=None): """ Percentage-based progress indicator :param total: total amount of items :param step: step size in percent :param start: at which percent value to start :param same_line: if True, emit output always on same line :param msg: output message, must contain one %f placeholder for the percentage :param file: output file, default: sys.stderr """ self.counter = 0 # 0 .. (total-1) self.total = total self.trigger_at = start # output next percentage value when reaching (at least) this self.step = step if file is None: file = sys.stderr self.file = file self.msg = msg self.same_line = same_line def progress(self, current=None): if current is not None: self.counter = current pct = self.counter * 100 / self.total self.counter += 1 if pct >= self.trigger_at: self.trigger_at += self.step return pct def show(self, current=None): pct = self.progress(current) if pct is not None: return self.output(pct) def output(self, percent): print(self.msg % percent, file=self.file, end='\r' if self.same_line else '\n', flush=True) def finish(self): if self.same_line: print(" " * len(self.msg % 100.0), file=self.file, end='\r') class ProgressIndicatorEndless: def __init__(self, step=10, file=None): """ Progress indicator (long row of dots) :param step: every Nth call, call the func :param file: output file, default: sys.stderr """ self.counter = 0 # call counter self.triggered = 0 # increases 1 per trigger event self.step = step # trigger every calls if file is None: file = sys.stderr self.file = file def progress(self): self.counter += 1 trigger = self.counter % self.step == 0 if trigger: self.triggered += 1 return trigger def show(self): trigger = self.progress() if trigger: return self.output(self.triggered) def output(self, triggered): print('.', end='', file=self.file, flush=True) def finish(self): print(file=self.file) def sysinfo(): python_implementation = platform.python_implementation() python_version = platform.python_version() # platform.uname() does a shell call internally to get processor info, # creating #3732 issue, so rather use os.uname(). try: uname = os.uname() except AttributeError: uname = None if sys.platform.startswith('linux'): try: linux_distribution = platform.linux_distribution() except: # platform.linux_distribution() is deprecated since py 3.5 and removed in 3.7. linux_distribution = ('Unknown Linux', '', '') else: linux_distribution = None info = [] if uname is not None: info.append('Platform: %s' % (' '.join(uname), )) if linux_distribution is not None: info.append('Linux: %s %s %s' % linux_distribution) info.append('Borg: %s Python: %s %s' % (borg_version, python_implementation, python_version)) info.append('PID: %d CWD: %s' % (os.getpid(), os.getcwd())) info.append('sys.argv: %r' % sys.argv) info.append('SSH_ORIGINAL_COMMAND: %r' % os.environ.get('SSH_ORIGINAL_COMMAND')) info.append('') return '\n'.join(info) def log_multi(*msgs, level=logging.INFO): """ log multiple lines of text, each line by a separate logging call for cosmetic reasons each positional argument may be a single or multiple lines (separated by newlines) of text. """ lines = [] for msg in msgs: lines.extend(msg.splitlines()) for line in lines: logger.log(level, line) class ErrorIgnoringTextIOWrapper(io.TextIOWrapper): def read(self, n): if not self.closed: try: return super().read(n) except BrokenPipeError: try: super().close() except OSError: pass return '' def write(self, s): if not self.closed: try: return super().write(s) except BrokenPipeError: try: super().close() except OSError: pass return len(s) class SignalException(BaseException): """base class for all signal-based exceptions""" class SigHup(SignalException): """raised on SIGHUP signal""" class SigTerm(SignalException): """raised on SIGTERM signal""" @contextlib.contextmanager def signal_handler(sig, handler): """ when entering context, set up signal handler for signal . when leaving context, restore original signal handler. can bei either a str when giving a signal.SIGXXX attribute name (it won't crash if the attribute name does not exist as some names are platform specific) or a int, when giving a signal number. is any handler value as accepted by the signal.signal(sig, handler). """ if isinstance(sig, str): sig = getattr(signal, sig, None) if sig is not None: orig_handler = signal.signal(sig, handler) try: yield finally: if sig is not None: signal.signal(sig, orig_handler) def raising_signal_handler(exc_cls): def handler(sig_no, frame): # setting SIG_IGN avoids that an incoming second signal of this # kind would raise a 2nd exception while we still process the # exception handler for exc_cls for the 1st signal. signal.signal(sig_no, signal.SIG_IGN) raise exc_cls return handler def prepare_subprocess_env(system, env=None): """ Prepare the environment for a subprocess we are going to create. :param system: True for preparing to invoke system-installed binaries, False for stuff inside the pyinstaller environment (like borg, python). :param env: optionally give a environment dict here. if not given, default to os.environ. :return: a modified copy of the environment """ env = dict(env if env is not None else os.environ) if system: # a pyinstaller binary's bootloader modifies LD_LIBRARY_PATH=/tmp/_MEIXXXXXX, # but we do not want that system binaries (like ssh or other) pick up # (non-matching) libraries from there. # thus we install the original LDLP, before pyinstaller has modified it: lp_key = 'LD_LIBRARY_PATH' lp_orig = env.get(lp_key + '_ORIG') # pyinstaller >= 20160820 / v3.2.1 has this if lp_orig is not None: env[lp_key] = lp_orig else: # We get here in 2 cases: # 1. when not running a pyinstaller-made binary. # in this case, we must not kill LDLP. # 2. when running a pyinstaller-made binary and there was no LDLP # in the original env (in this case, the pyinstaller bootloader # does *not* put ..._ORIG into the env either). # in this case, we must kill LDLP. # The directory used by pyinstaller is created by mkdtemp("_MEIXXXXXX"), # we can use that to differentiate between the cases. lp = env.get(lp_key) if lp is not None and re.search(r'/_MEI......', lp): env.pop(lp_key) # security: do not give secrets to subprocess env.pop('BORG_PASSPHRASE', None) # for information, give borg version to the subprocess env['BORG_VERSION'] = borg_version return env borgbackup-1.0.13/borg/key.py0000644000175000017500000005604113431513133015767 0ustar useruser00000000000000from binascii import hexlify, a2b_base64, b2a_base64 import configparser import getpass import os import shlex import subprocess import sys import textwrap from hmac import HMAC, compare_digest from hashlib import sha256, sha512, pbkdf2_hmac import msgpack from .helpers import StableDict, IntegrityError, get_keys_dir, get_security_dir, Error, yes, bin_to_hex from .logger import create_logger logger = create_logger() from .crypto import AES, bytes_to_long, long_to_bytes, bytes_to_int, num_aes_blocks from .crypto import hkdf_hmac_sha512 from .compress import Compressor, CNONE from .helpers import get_limited_unpacker from .helpers import prepare_subprocess_env PREFIX = b'\0' * 8 class PassphraseWrong(Error): """passphrase supplied in BORG_PASSPHRASE, by BORG_PASSCOMMAND or via BORG_PASSPHRASE_FD is incorrect.""" class PasscommandFailure(Error): """passcommand supplied in BORG_PASSCOMMAND failed: {}""" class PasswordRetriesExceeded(Error): """exceeded the maximum password retries""" class UnsupportedPayloadError(Error): """Unsupported payload type {}. A newer version is required to access this repository.""" class UnsupportedManifestError(Error): """Unsupported manifest envelope. A newer version is required to access this repository.""" class KeyfileNotFoundError(Error): """No key file for repository {} found in {}.""" class RepoKeyNotFoundError(Error): """No key entry found in the config of repository {}.""" class TAMRequiredError(IntegrityError): __doc__ = textwrap.dedent(""" Manifest is unauthenticated, but it is required for this repository. This either means that you are under attack, or that you modified this repository with a Borg version older than 1.0.9 after TAM authentication was enabled. In the latter case, use "borg upgrade --tam --force '{}'" to re-authenticate the manifest. """).strip() traceback = False class TAMInvalid(IntegrityError): __doc__ = IntegrityError.__doc__ traceback = False def __init__(self): # Error message becomes: "Data integrity error: Manifest authentication did not verify" super().__init__('Manifest authentication did not verify') class TAMUnsupportedSuiteError(IntegrityError): """Could not verify manifest: Unsupported suite {!r}; a newer version is needed.""" traceback = False def key_creator(repository, args): if args.encryption == 'keyfile': return KeyfileKey.create(repository, args) elif args.encryption == 'repokey': return RepoKey.create(repository, args) else: return PlaintextKey.create(repository, args) def key_factory(repository, manifest_data): key_type = manifest_data[0] if key_type == KeyfileKey.TYPE: return KeyfileKey.detect(repository, manifest_data) elif key_type == RepoKey.TYPE: return RepoKey.detect(repository, manifest_data) elif key_type == PassphraseKey.TYPE: # we just dispatch to repokey mode and assume the passphrase was migrated to a repokey. # see also comment in PassphraseKey class. return RepoKey.detect(repository, manifest_data) elif key_type == PlaintextKey.TYPE: return PlaintextKey.detect(repository, manifest_data) else: raise UnsupportedPayloadError(key_type) def tam_required_file(repository): security_dir = get_security_dir(bin_to_hex(repository.id)) return os.path.join(security_dir, 'tam_required') def tam_required(repository): file = tam_required_file(repository) return os.path.isfile(file) class KeyBase: TYPE = None # override in subclasses def __init__(self, repository): self.TYPE_STR = bytes([self.TYPE]) self.repository = repository self.target = None # key location file path / repo obj self.compressor = Compressor('none') self.tam_required = True def id_hash(self, data): """Return HMAC hash using the "id" HMAC key """ def encrypt(self, data, none_compression=False): pass def decrypt(self, id, data): pass def _tam_key(self, salt, context): return hkdf_hmac_sha512( ikm=self.id_key + self.enc_key + self.enc_hmac_key, salt=salt, info=b'borg-metadata-authentication-' + context, output_length=64 ) def pack_and_authenticate_metadata(self, metadata_dict, context=b'manifest'): metadata_dict = StableDict(metadata_dict) tam = metadata_dict['tam'] = StableDict({ 'type': 'HKDF_HMAC_SHA512', 'hmac': bytes(64), 'salt': os.urandom(64), }) packed = msgpack.packb(metadata_dict, unicode_errors='surrogateescape') tam_key = self._tam_key(tam['salt'], context) tam['hmac'] = HMAC(tam_key, packed, sha512).digest() return msgpack.packb(metadata_dict, unicode_errors='surrogateescape') def unpack_and_verify_manifest(self, data, force_tam_not_required=False): """Unpack msgpacked *data* and return (object, did_verify).""" if data.startswith(b'\xc1' * 4): # This is a manifest from the future, we can't read it. raise UnsupportedManifestError() tam_required = self.tam_required if force_tam_not_required and tam_required: logger.warning('Manifest authentication DISABLED.') tam_required = False data = bytearray(data) unpacker = get_limited_unpacker('manifest') unpacker.feed(data) unpacked = unpacker.unpack() if b'tam' not in unpacked: if tam_required: raise TAMRequiredError(self.repository._location.canonical_path()) else: logger.debug('TAM not found and not required') return unpacked, False tam = unpacked.pop(b'tam', None) if not isinstance(tam, dict): raise TAMInvalid() tam_type = tam.get(b'type', b'').decode('ascii', 'replace') if tam_type != 'HKDF_HMAC_SHA512': if tam_required: raise TAMUnsupportedSuiteError(repr(tam_type)) else: logger.debug('Ignoring TAM made with unsupported suite, since TAM is not required: %r', tam_type) return unpacked, False tam_hmac = tam.get(b'hmac') tam_salt = tam.get(b'salt') if not isinstance(tam_salt, bytes) or not isinstance(tam_hmac, bytes): raise TAMInvalid() offset = data.index(tam_hmac) data[offset:offset + 64] = bytes(64) tam_key = self._tam_key(tam_salt, context=b'manifest') calculated_hmac = HMAC(tam_key, data, sha512).digest() if not compare_digest(calculated_hmac, tam_hmac): raise TAMInvalid() logger.debug('TAM-verified manifest') return unpacked, True class PlaintextKey(KeyBase): TYPE = 0x02 chunk_seed = 0 def __init__(self, repository): super().__init__(repository) self.tam_required = False @classmethod def create(cls, repository, args): logger.info('Encryption NOT enabled.\nUse the "--encryption=repokey|keyfile" to enable encryption.') return cls(repository) @classmethod def detect(cls, repository, manifest_data): return cls(repository) def id_hash(self, data): return sha256(data).digest() def encrypt(self, data, none_compression=False): if none_compression: compressed = CNONE().compress(data) else: compressed = self.compressor.compress(data) return b''.join([self.TYPE_STR, compressed]) def decrypt(self, id, data): if data[0] != self.TYPE: id_str = bin_to_hex(id) if id is not None else '(unknown)' raise IntegrityError('Chunk %s: Invalid encryption envelope' % id_str) data = self.compressor.decompress(memoryview(data)[1:]) if id and sha256(data).digest() != id: raise IntegrityError('Chunk %s: id verification failed' % bin_to_hex(id)) return data def _tam_key(self, salt, context): return salt + context class AESKeyBase(KeyBase): """Common base class shared by KeyfileKey and PassphraseKey Chunks are encrypted using 256bit AES in Counter Mode (CTR) Payload layout: TYPE(1) + HMAC(32) + NONCE(8) + CIPHERTEXT To reduce payload size only 8 bytes of the 16 bytes nonce is saved in the payload, the first 8 bytes are always zeros. This does not affect security but limits the maximum repository capacity to only 295 exabytes! """ PAYLOAD_OVERHEAD = 1 + 32 + 8 # TYPE + HMAC + NONCE def id_hash(self, data): """Return HMAC hash using the "id" HMAC key """ return HMAC(self.id_key, data, sha256).digest() def encrypt(self, data, none_compression=False): if none_compression: data = CNONE().compress(data) else: data = self.compressor.compress(data) self.enc_cipher.reset() data = b''.join((self.enc_cipher.iv[8:], self.enc_cipher.encrypt(data))) hmac = HMAC(self.enc_hmac_key, data, sha256).digest() return b''.join((self.TYPE_STR, hmac, data)) def decrypt(self, id, data): if not (data[0] == self.TYPE or data[0] == PassphraseKey.TYPE and isinstance(self, RepoKey)): id_str = bin_to_hex(id) if id is not None else '(unknown)' raise IntegrityError('Chunk %s: Invalid encryption envelope' % id_str) hmac_given = memoryview(data)[1:33] hmac_computed = memoryview(HMAC(self.enc_hmac_key, memoryview(data)[33:], sha256).digest()) if not compare_digest(hmac_computed, hmac_given): id_str = bin_to_hex(id) if id is not None else '(unknown)' raise IntegrityError('Chunk %s: Encryption envelope checksum mismatch' % id_str) self.dec_cipher.reset(iv=PREFIX + data[33:41]) data = self.compressor.decompress(self.dec_cipher.decrypt(data[41:])) if id: hmac_given = id hmac_computed = HMAC(self.id_key, data, sha256).digest() if not compare_digest(hmac_computed, hmac_given): raise IntegrityError('Chunk %s: Chunk id verification failed' % bin_to_hex(id)) return data def extract_nonce(self, payload): if not (payload[0] == self.TYPE or payload[0] == PassphraseKey.TYPE and isinstance(self, RepoKey)): raise IntegrityError('Manifest: Invalid encryption envelope') nonce = bytes_to_long(payload[33:41]) return nonce def init_from_random_data(self, data): self.enc_key = data[0:32] self.enc_hmac_key = data[32:64] self.id_key = data[64:96] self.chunk_seed = bytes_to_int(data[96:100]) # Convert to signed int32 if self.chunk_seed & 0x80000000: self.chunk_seed = self.chunk_seed - 0xffffffff - 1 def init_ciphers(self, enc_iv=b''): self.enc_cipher = AES(is_encrypt=True, key=self.enc_key, iv=enc_iv) self.dec_cipher = AES(is_encrypt=False, key=self.enc_key) class Passphrase(str): @classmethod def _env_passphrase(cls, env_var, default=None): passphrase = os.environ.get(env_var, default) if passphrase is not None: return cls(passphrase) @classmethod def env_passphrase(cls, default=None): passphrase = cls._env_passphrase('BORG_PASSPHRASE', default) if passphrase is not None: return passphrase passphrase = cls.env_passcommand() if passphrase is not None: return passphrase passphrase = cls.fd_passphrase() if passphrase is not None: return passphrase @classmethod def env_passcommand(cls, default=None): passcommand = os.environ.get('BORG_PASSCOMMAND', None) if passcommand is not None: # passcommand is a system command (not inside pyinstaller env) env = prepare_subprocess_env(system=True) try: passphrase = subprocess.check_output(shlex.split(passcommand), universal_newlines=True, env=env) except (subprocess.CalledProcessError, FileNotFoundError) as e: raise PasscommandFailure(e) return cls(passphrase.rstrip('\n')) @classmethod def fd_passphrase(cls): try: fd = int(os.environ.get('BORG_PASSPHRASE_FD')) except (ValueError, TypeError): return None with os.fdopen(fd, mode='r') as f: passphrase = f.read() return cls(passphrase.rstrip('\n')) @classmethod def getpass(cls, prompt): return cls(getpass.getpass(prompt)) @classmethod def verification(cls, passphrase): msg = 'Do you want your passphrase to be displayed for verification? [yN]: ' if yes(msg, retry_msg=msg, invalid_msg='Invalid answer, try again.', retry=True, env_var_override='BORG_DISPLAY_PASSPHRASE'): print('Your passphrase (between double-quotes): "%s"' % passphrase, file=sys.stderr) print('Make sure the passphrase displayed above is exactly what you wanted.', file=sys.stderr) try: passphrase.encode('ascii') except UnicodeEncodeError: print('Your passphrase (UTF-8 encoding in hex): %s' % bin_to_hex(passphrase.encode('utf-8')), file=sys.stderr) print('As you have a non-ASCII passphrase, it is recommended to keep the UTF-8 encoding in hex together with the passphrase at a safe place.', file=sys.stderr) @classmethod def new(cls, allow_empty=False): passphrase = cls.env_passphrase() if passphrase is not None: return passphrase for retry in range(1, 11): passphrase = cls.getpass('Enter new passphrase: ') if allow_empty or passphrase: passphrase2 = cls.getpass('Enter same passphrase again: ') if passphrase == passphrase2: cls.verification(passphrase) logger.info('Remember your passphrase. Your data will be inaccessible without it.') return passphrase else: print('Passphrases do not match', file=sys.stderr) else: print('Passphrase must not be blank', file=sys.stderr) else: raise PasswordRetriesExceeded def __repr__(self): return '' def kdf(self, salt, iterations, length): return pbkdf2_hmac('sha256', self.encode('utf-8'), salt, iterations, length) class PassphraseKey(AESKeyBase): # This mode was killed in borg 1.0, see: https://github.com/borgbackup/borg/issues/97 # Reasons: # - you can never ever change your passphrase for existing repos. # - you can never ever use a different iterations count for existing repos. # "Killed" means: # - there is no automatic dispatch to this class via type byte # - --encryption=passphrase is an invalid argument now # This class is kept for a while to support migration from passphrase to repokey mode. TYPE = 0x01 iterations = 100000 # must not be changed ever! @classmethod def create(cls, repository, args): key = cls(repository) logger.warning('WARNING: "passphrase" mode is unsupported since borg 1.0.') passphrase = Passphrase.new(allow_empty=False) key.init(repository, passphrase) return key @classmethod def detect(cls, repository, manifest_data): prompt = 'Enter passphrase for %s: ' % repository._location.orig key = cls(repository) passphrase = Passphrase.env_passphrase() if passphrase is None: passphrase = Passphrase.getpass(prompt) for retry in range(1, 3): key.init(repository, passphrase) try: key.decrypt(None, manifest_data) num_blocks = num_aes_blocks(len(manifest_data) - 41) key.init_ciphers(PREFIX + long_to_bytes(key.extract_nonce(manifest_data) + num_blocks)) key._passphrase = passphrase return key except IntegrityError: passphrase = Passphrase.getpass(prompt) else: raise PasswordRetriesExceeded def change_passphrase(self): class ImmutablePassphraseError(Error): """The passphrase for this encryption key type can't be changed.""" raise ImmutablePassphraseError def init(self, repository, passphrase): self.init_from_random_data(passphrase.kdf(repository.id, self.iterations, 100)) self.init_ciphers() self.tam_required = False class KeyfileKeyBase(AESKeyBase): @classmethod def detect(cls, repository, manifest_data): key = cls(repository) target = key.find_key() prompt = 'Enter passphrase for key %s: ' % target passphrase = Passphrase.env_passphrase() if passphrase is None: passphrase = Passphrase() if not key.load(target, passphrase): for retry in range(0, 3): passphrase = Passphrase.getpass(prompt) if key.load(target, passphrase): break else: raise PasswordRetriesExceeded else: if not key.load(target, passphrase): raise PassphraseWrong num_blocks = num_aes_blocks(len(manifest_data) - 41) key.init_ciphers(PREFIX + long_to_bytes(key.extract_nonce(manifest_data) + num_blocks)) key._passphrase = passphrase return key def find_key(self): raise NotImplementedError def load(self, target, passphrase): raise NotImplementedError def _load(self, key_data, passphrase): cdata = a2b_base64(key_data) data = self.decrypt_key_file(cdata, passphrase) if data: key = msgpack.unpackb(data) if key[b'version'] != 1: raise IntegrityError('Invalid key file header') self.repository_id = key[b'repository_id'] self.enc_key = key[b'enc_key'] self.enc_hmac_key = key[b'enc_hmac_key'] self.id_key = key[b'id_key'] self.chunk_seed = key[b'chunk_seed'] self.tam_required = key.get(b'tam_required', tam_required(self.repository)) return True return False def decrypt_key_file(self, data, passphrase): unpacker = get_limited_unpacker('key') unpacker.feed(data) d = unpacker.unpack() assert d[b'version'] == 1 assert d[b'algorithm'] == b'sha256' key = passphrase.kdf(d[b'salt'], d[b'iterations'], 32) data = AES(is_encrypt=False, key=key).decrypt(d[b'data']) if HMAC(key, data, sha256).digest() == d[b'hash']: return data def encrypt_key_file(self, data, passphrase): salt = os.urandom(32) iterations = 100000 key = passphrase.kdf(salt, iterations, 32) hash = HMAC(key, data, sha256).digest() cdata = AES(is_encrypt=True, key=key).encrypt(data) d = { 'version': 1, 'salt': salt, 'iterations': iterations, 'algorithm': 'sha256', 'hash': hash, 'data': cdata, } return msgpack.packb(d) def _save(self, passphrase): key = { 'version': 1, 'repository_id': self.repository_id, 'enc_key': self.enc_key, 'enc_hmac_key': self.enc_hmac_key, 'id_key': self.id_key, 'chunk_seed': self.chunk_seed, 'tam_required': self.tam_required, } data = self.encrypt_key_file(msgpack.packb(key), passphrase) key_data = '\n'.join(textwrap.wrap(b2a_base64(data).decode('ascii'))) return key_data def change_passphrase(self, passphrase=None): if passphrase is None: passphrase = Passphrase.new(allow_empty=True) self.save(self.target, passphrase) @classmethod def create(cls, repository, args): passphrase = Passphrase.new(allow_empty=True) key = cls(repository) key.repository_id = repository.id key.init_from_random_data(os.urandom(100)) key.init_ciphers() target = key.get_new_target(args) key.save(target, passphrase) logger.info('Key in "%s" created.' % target) logger.info('Keep this key safe. Your data will be inaccessible without it.') return key def save(self, target, passphrase): raise NotImplementedError def get_new_target(self, args): raise NotImplementedError class KeyfileKey(KeyfileKeyBase): TYPE = 0x00 FILE_ID = 'BORG_KEY' def find_key(self): file_id = self.FILE_ID.encode() first_line = file_id + b' ' + hexlify(self.repository.id) keys_dir = get_keys_dir() for name in os.listdir(keys_dir): filename = os.path.join(keys_dir, name) # we do the magic / id check in binary mode to avoid stumbling over # decoding errors if somebody has binary files in the keys dir for some reason. with open(filename, 'rb') as fd: if fd.read(len(first_line)) == first_line: return filename raise KeyfileNotFoundError(self.repository._location.canonical_path(), get_keys_dir()) def get_new_target(self, args): filename = args.location.to_key_filename() path = filename i = 1 while os.path.exists(path): i += 1 path = filename + '.%d' % i return path def load(self, target, passphrase): with open(target, 'r') as fd: key_data = ''.join(fd.readlines()[1:]) success = self._load(key_data, passphrase) if success: self.target = target return success def save(self, target, passphrase): key_data = self._save(passphrase) with open(target, 'w') as fd: fd.write('%s %s\n' % (self.FILE_ID, bin_to_hex(self.repository_id))) fd.write(key_data) fd.write('\n') self.target = target class RepoKey(KeyfileKeyBase): TYPE = 0x03 def find_key(self): loc = self.repository._location.canonical_path() try: self.repository.load_key() return loc except configparser.NoOptionError: raise RepoKeyNotFoundError(loc) from None def get_new_target(self, args): return self.repository def load(self, target, passphrase): # what we get in target is just a repo location, but we already have the repo obj: target = self.repository key_data = target.load_key() key_data = key_data.decode('utf-8') # remote repo: msgpack issue #99, getting bytes success = self._load(key_data, passphrase) if success: self.target = target return success def save(self, target, passphrase): key_data = self._save(passphrase) key_data = key_data.encode('utf-8') # remote repo: msgpack issue #99, giving bytes target.save_key(key_data) self.target = target borgbackup-1.0.13/borg/keymanager.py0000644000175000017500000002002413431513133017312 0ustar useruser00000000000000from binascii import unhexlify, a2b_base64, b2a_base64 import binascii import textwrap from hashlib import sha256 import pkgutil from .key import KeyfileKey, RepoKey, PassphraseKey, KeyfileNotFoundError, PlaintextKey from .helpers import Manifest, NoManifestError, Error, yes, bin_to_hex from .repository import Repository class UnencryptedRepo(Error): """Keymanagement not available for unencrypted repositories.""" class UnknownKeyType(Error): """Keytype {0} is unknown.""" class RepoIdMismatch(Error): """This key backup seems to be for a different backup repository, aborting.""" class NotABorgKeyFile(Error): """This file is not a borg key backup, aborting.""" def sha256_truncated(data, num): h = sha256() h.update(data) return h.hexdigest()[:num] KEYBLOB_LOCAL = 'local' KEYBLOB_REPO = 'repo' class KeyManager: def __init__(self, repository): self.repository = repository self.keyblob = None self.keyblob_storage = None try: cdata = self.repository.get(Manifest.MANIFEST_ID) except Repository.ObjectNotFound: raise NoManifestError key_type = cdata[0] if key_type == KeyfileKey.TYPE: self.keyblob_storage = KEYBLOB_LOCAL elif key_type == RepoKey.TYPE or key_type == PassphraseKey.TYPE: self.keyblob_storage = KEYBLOB_REPO elif key_type == PlaintextKey.TYPE: raise UnencryptedRepo() else: raise UnknownKeyType(key_type) def load_keyblob(self): if self.keyblob_storage == KEYBLOB_LOCAL: k = KeyfileKey(self.repository) target = k.find_key() with open(target, 'r') as fd: self.keyblob = ''.join(fd.readlines()[1:]) elif self.keyblob_storage == KEYBLOB_REPO: self.keyblob = self.repository.load_key().decode() def store_keyblob(self, args): if self.keyblob_storage == KEYBLOB_LOCAL: k = KeyfileKey(self.repository) try: target = k.find_key() except KeyfileNotFoundError: target = k.get_new_target(args) self.store_keyfile(target) elif self.keyblob_storage == KEYBLOB_REPO: self.repository.save_key(self.keyblob.encode('utf-8')) def get_keyfile_data(self): data = '%s %s\n' % (KeyfileKey.FILE_ID, bin_to_hex(self.repository.id)) data += self.keyblob if not self.keyblob.endswith('\n'): data += '\n' return data def store_keyfile(self, target): with open(target, 'w') as fd: fd.write(self.get_keyfile_data()) def export(self, path): self.store_keyfile(path) def export_qr(self, path): with open(path, 'wb') as fd: key_data = self.get_keyfile_data() html = pkgutil.get_data('borg', 'paperkey.html') html = html.replace(b'', key_data.encode() + b'') fd.write(html) def export_paperkey(self, path): def grouped(s): ret = '' i = 0 for ch in s: if i and i % 6 == 0: ret += ' ' ret += ch i += 1 return ret export = 'To restore key use borg key import --paper /path/to/repo\n\n' binary = a2b_base64(self.keyblob) export += 'BORG PAPER KEY v1\n' lines = (len(binary) + 17) // 18 repoid = bin_to_hex(self.repository.id)[:18] complete_checksum = sha256_truncated(binary, 12) export += 'id: {0:d} / {1} / {2} - {3}\n'.format(lines, grouped(repoid), grouped(complete_checksum), sha256_truncated((str(lines) + '/' + repoid + '/' + complete_checksum).encode('ascii'), 2)) idx = 0 while len(binary): idx += 1 binline = binary[:18] checksum = sha256_truncated(idx.to_bytes(2, byteorder='big') + binline, 2) export += '{0:2d}: {1} - {2}\n'.format(idx, grouped(bin_to_hex(binline)), checksum) binary = binary[18:] if path: with open(path, 'w') as fd: fd.write(export) else: print(export) def import_keyfile(self, args): file_id = KeyfileKey.FILE_ID first_line = file_id + ' ' + bin_to_hex(self.repository.id) + '\n' with open(args.path, 'r') as fd: file_first_line = fd.read(len(first_line)) if file_first_line != first_line: if not file_first_line.startswith(file_id): raise NotABorgKeyFile() else: raise RepoIdMismatch() self.keyblob = fd.read() self.store_keyblob(args) def import_paperkey(self, args): try: # imported here because it has global side effects import readline except ImportError: print('Note: No line editing available due to missing readline support') repoid = bin_to_hex(self.repository.id)[:18] try: while True: # used for repeating on overall checksum mismatch # id line input while True: idline = input('id: ').replace(' ', '') if idline == '': if yes('Abort import? [yN]:'): raise EOFError() try: (data, checksum) = idline.split('-') except ValueError: print("each line must contain exactly one '-', try again") continue try: (id_lines, id_repoid, id_complete_checksum) = data.split('/') except ValueError: print("the id line must contain exactly three '/', try again") continue if sha256_truncated(data.lower().encode('ascii'), 2) != checksum: print('line checksum did not match, try same line again') continue try: lines = int(id_lines) except ValueError: print('internal error while parsing length') break if repoid != id_repoid: raise RepoIdMismatch() result = b'' idx = 1 # body line input while True: inline = input('{0:2d}: '.format(idx)) inline = inline.replace(' ', '') if inline == '': if yes('Abort import? [yN]:'): raise EOFError() try: (data, checksum) = inline.split('-') except ValueError: print("each line must contain exactly one '-', try again") continue try: part = unhexlify(data) except binascii.Error: print("only characters 0-9 and a-f and '-' are valid, try again") continue if sha256_truncated(idx.to_bytes(2, byteorder='big') + part, 2) != checksum: print('line checksum did not match, try line {0} again'.format(idx)) continue result += part if idx == lines: break idx += 1 if sha256_truncated(result, 12) != id_complete_checksum: print('The overall checksum did not match, retry or enter a blank line to abort.') continue self.keyblob = '\n'.join(textwrap.wrap(b2a_base64(result).decode('ascii'))) + '\n' self.store_keyblob(args) break except EOFError: print('\n - aborted') return borgbackup-1.0.13/borg/locking.py0000644000175000017500000002657413431513133016635 0ustar useruser00000000000000import json import os import socket import time from borg.helpers import Error, ErrorWithTraceback ADD, REMOVE = 'add', 'remove' SHARED, EXCLUSIVE = 'shared', 'exclusive' # for performance reasons, only determine the hostname once. _hostname = socket.gethostname() def get_id(): """ Return identification tuple (hostname, pid, thread_id) for 'us'. This always returns the current pid, which might be different from before, e.g. if daemonize() was used. Note: Currently thread_id is *always* zero. """ thread_id = 0 pid = os.getpid() return _hostname, pid, thread_id class TimeoutTimer: """ A timer for timeout checks (can also deal with "never timeout"). It can also compute and optionally execute a reasonable sleep time (e.g. to avoid polling too often or to support thread/process rescheduling). """ def __init__(self, timeout=None, sleep=None): """ Initialize a timer. :param timeout: time out interval [s] or None (never timeout, wait forever) [default] :param sleep: sleep interval [s] (>= 0: do sleep call, <0: don't call sleep) or None (autocompute: use 10% of timeout [but not more than 60s], or 1s for "never timeout" mode) """ if timeout is not None and timeout < 0: raise ValueError("timeout must be >= 0") self.timeout_interval = timeout if sleep is None: if timeout is None: sleep = 1.0 else: sleep = min(60.0, timeout / 10.0) self.sleep_interval = sleep self.start_time = None self.end_time = None def __repr__(self): return "<%s: start=%r end=%r timeout=%r sleep=%r>" % ( self.__class__.__name__, self.start_time, self.end_time, self.timeout_interval, self.sleep_interval) def start(self): self.start_time = time.time() if self.timeout_interval is not None: self.end_time = self.start_time + self.timeout_interval return self def sleep(self): if self.sleep_interval >= 0: time.sleep(self.sleep_interval) def timed_out(self): return self.end_time is not None and time.time() >= self.end_time def timed_out_or_sleep(self): if self.timed_out(): return True else: self.sleep() return False class LockError(Error): """Failed to acquire the lock {}.""" class LockErrorT(ErrorWithTraceback): """Failed to acquire the lock {}.""" class LockTimeout(LockError): """Failed to create/acquire the lock {} (timeout).""" class LockFailed(LockErrorT): """Failed to create/acquire the lock {} ({}).""" class NotLocked(LockErrorT): """Failed to release the lock {} (was not locked).""" class NotMyLock(LockErrorT): """Failed to release the lock {} (was/is locked, but not by me).""" class ExclusiveLock: """An exclusive Lock based on mkdir fs operation being atomic. If possible, try to use the contextmanager here like:: with ExclusiveLock(...) as lock: ... This makes sure the lock is released again if the block is left, no matter how (e.g. if an exception occurred). """ def __init__(self, path, timeout=None, sleep=None, id=None): self.timeout = timeout self.sleep = sleep self.path = os.path.abspath(path) self.id = id or get_id() self.unique_name = os.path.join(self.path, "%s.%d-%x" % self.id) def __enter__(self): return self.acquire() def __exit__(self, *exc): self.release() def __repr__(self): return "<%s: %r>" % (self.__class__.__name__, self.unique_name) def acquire(self, timeout=None, sleep=None): if timeout is None: timeout = self.timeout if sleep is None: sleep = self.sleep timer = TimeoutTimer(timeout, sleep).start() while True: try: os.mkdir(self.path) except FileExistsError: # already locked if self.by_me(): return self if timer.timed_out_or_sleep(): raise LockTimeout(self.path) except OSError as err: raise LockFailed(self.path, str(err)) from None else: with open(self.unique_name, "wb"): pass return self def release(self): if not self.is_locked(): raise NotLocked(self.path) if not self.by_me(): raise NotMyLock(self.path) os.unlink(self.unique_name) os.rmdir(self.path) def is_locked(self): return os.path.exists(self.path) def by_me(self): return os.path.exists(self.unique_name) def break_lock(self): if self.is_locked(): for name in os.listdir(self.path): os.unlink(os.path.join(self.path, name)) os.rmdir(self.path) def migrate_lock(self, old_id, new_id): """migrate the lock ownership from old_id to new_id""" assert self.id == old_id new_unique_name = os.path.join(self.path, "%s.%d-%x" % new_id) if self.is_locked() and self.by_me(): with open(new_unique_name, "wb"): pass os.unlink(self.unique_name) self.id, self.unique_name = new_id, new_unique_name class LockRoster: """ A Lock Roster to track shared/exclusive lockers. Note: you usually should call the methods with an exclusive lock held, to avoid conflicting access by multiple threads/processes/machines. """ def __init__(self, path, id=None): self.path = path self.id = id or get_id() def load(self): try: with open(self.path) as f: data = json.load(f) except (FileNotFoundError, ValueError): # no or corrupt/empty roster file? data = {} return data def save(self, data): with open(self.path, "w") as f: json.dump(data, f) def remove(self): try: os.unlink(self.path) except FileNotFoundError: pass def get(self, key): roster = self.load() return set(tuple(e) for e in roster.get(key, [])) def empty(self, *keys): return all(not self.get(key) for key in keys) def modify(self, key, op): roster = self.load() try: elements = set(tuple(e) for e in roster[key]) except KeyError: elements = set() if op == ADD: elements.add(self.id) elif op == REMOVE: elements.remove(self.id) else: raise ValueError('Unknown LockRoster op %r' % op) roster[key] = list(list(e) for e in elements) self.save(roster) def migrate_lock(self, key, old_id, new_id): """migrate the lock ownership from old_id to new_id""" assert self.id == old_id try: self.modify(key, REMOVE) except KeyError: # entry was not there, so no need to add a new one, but still update our id self.id = new_id else: # old entry removed, update our id and add a updated entry self.id = new_id self.modify(key, ADD) class Lock: """ A Lock for a resource that can be accessed in a shared or exclusive way. Typically, write access to a resource needs an exclusive lock (1 writer, noone is allowed reading) and read access to a resource needs a shared lock (multiple readers are allowed). If possible, try to use the contextmanager here like:: with Lock(...) as lock: ... This makes sure the lock is released again if the block is left, no matter how (e.g. if an exception occurred). """ def __init__(self, path, exclusive=False, sleep=None, timeout=None, id=None): self.path = path self.is_exclusive = exclusive self.sleep = sleep self.timeout = timeout self.id = id or get_id() # globally keeping track of shared and exclusive lockers: self._roster = LockRoster(path + '.roster', id=id) # an exclusive lock, used for: # - holding while doing roster queries / updates # - holding while the Lock instance itself is exclusive self._lock = ExclusiveLock(path + '.exclusive', id=id, timeout=timeout) def __enter__(self): return self.acquire() def __exit__(self, *exc): self.release() def __repr__(self): return "<%s: %r>" % (self.__class__.__name__, self.id) def acquire(self, exclusive=None, remove=None, sleep=None): if exclusive is None: exclusive = self.is_exclusive sleep = sleep or self.sleep or 0.2 if exclusive: self._wait_for_readers_finishing(remove, sleep) self._roster.modify(EXCLUSIVE, ADD) else: with self._lock: if remove is not None: self._roster.modify(remove, REMOVE) self._roster.modify(SHARED, ADD) self.is_exclusive = exclusive return self def _wait_for_readers_finishing(self, remove, sleep): timer = TimeoutTimer(self.timeout, sleep).start() while True: self._lock.acquire() try: if remove is not None: self._roster.modify(remove, REMOVE) if len(self._roster.get(SHARED)) == 0: return # we are the only one and we keep the lock! # restore the roster state as before (undo the roster change): if remove is not None: self._roster.modify(remove, ADD) except: # avoid orphan lock when an exception happens here, e.g. Ctrl-C! self._lock.release() raise else: self._lock.release() if timer.timed_out_or_sleep(): raise LockTimeout(self.path) def release(self): if self.is_exclusive: self._roster.modify(EXCLUSIVE, REMOVE) if self._roster.empty(EXCLUSIVE, SHARED): self._roster.remove() self._lock.release() else: with self._lock: self._roster.modify(SHARED, REMOVE) if self._roster.empty(EXCLUSIVE, SHARED): self._roster.remove() def upgrade(self): # WARNING: if multiple read-lockers want to upgrade, it will deadlock because they # all will wait until the other read locks go away - and that won't happen. if not self.is_exclusive: self.acquire(exclusive=True, remove=SHARED) def downgrade(self): if self.is_exclusive: self.acquire(exclusive=False, remove=EXCLUSIVE) def got_exclusive_lock(self): return self.is_exclusive and self._lock.is_locked() and self._lock.by_me() def break_lock(self): self._roster.remove() self._lock.break_lock() def migrate_lock(self, old_id, new_id): assert self.id == old_id self.id = new_id if self.is_exclusive: self._lock.migrate_lock(old_id, new_id) self._roster.migrate_lock(EXCLUSIVE, old_id, new_id) else: with self._lock: self._lock.migrate_lock(old_id, new_id) self._roster.migrate_lock(SHARED, old_id, new_id) borgbackup-1.0.13/borg/logger.py0000644000175000017500000001451413431513133016455 0ustar useruser00000000000000"""logging facilities The way to use this is as follows: * each module declares its own logger, using: from .logger import create_logger logger = create_logger() * then each module uses logger.info/warning/debug/etc according to the level it believes is appropriate: logger.debug('debugging info for developers or power users') logger.info('normal, informational output') logger.warning('warn about a non-fatal error or sth else') logger.error('a fatal error') ... and so on. see the `logging documentation `_ for more information * console interaction happens on stderr, that includes interactive reporting functions like `help`, `info` and `list` * ...except ``input()`` is special, because we can't control the stream it is using, unfortunately. we assume that it won't clutter stdout, because interaction would be broken then anyways * what is output on INFO level is additionally controlled by commandline flags """ import inspect import logging import logging.config import logging.handlers # needed for handlers defined there being configurable in logging.conf file import os import warnings configured = False # use something like this to ignore warnings: # warnings.filterwarnings('ignore', r'... regex for warning message to ignore ...') def _log_warning(message, category, filename, lineno, file=None, line=None): # for warnings, we just want to use the logging system, not stderr or other files msg = "{0}:{1}: {2}: {3}".format(filename, lineno, category.__name__, message) logger = create_logger(__name__) # Note: the warning will look like coming from here, # but msg contains info about where it really comes from logger.warning(msg) def setup_logging(stream=None, conf_fname=None, env_var='BORG_LOGGING_CONF', level='info', is_serve=False): """setup logging module according to the arguments provided if conf_fname is given (or the config file name can be determined via the env_var, if given): load this logging configuration. otherwise, set up a stream handler logger on stderr (by default, if no stream is provided). if is_serve == True, we configure a special log format as expected by the borg client log message interceptor. """ global configured err_msg = None if env_var: conf_fname = os.environ.get(env_var, conf_fname) if conf_fname: try: conf_fname = os.path.abspath(conf_fname) # we open the conf file here to be able to give a reasonable # error message in case of failure (if we give the filename to # fileConfig(), it silently ignores unreadable files and gives # unhelpful error msgs like "No section: 'formatters'"): with open(conf_fname) as f: logging.config.fileConfig(f) configured = True logger = logging.getLogger(__name__) logger.debug('using logging configuration read from "{0}"'.format(conf_fname)) warnings.showwarning = _log_warning return None except Exception as err: # XXX be more precise err_msg = str(err) # if we did not / not successfully load a logging configuration, fallback to this: logger = logging.getLogger('') handler = logging.StreamHandler(stream) if is_serve: fmt = '$LOG %(levelname)s Remote: %(message)s' else: fmt = '%(message)s' handler.setFormatter(logging.Formatter(fmt)) logger.addHandler(handler) logger.setLevel(level.upper()) configured = True logger = logging.getLogger(__name__) if err_msg: logger.warning('setup_logging for "{0}" failed with "{1}".'.format(conf_fname, err_msg)) logger.debug('using builtin fallback logging configuration') warnings.showwarning = _log_warning return handler def find_parent_module(): """find the name of a the first module calling this module if we cannot find it, we return the current module's name (__name__) instead. """ try: frame = inspect.currentframe().f_back module = inspect.getmodule(frame) while module is None or module.__name__ == __name__: frame = frame.f_back module = inspect.getmodule(frame) return module.__name__ except AttributeError: # somehow we failed to find our module # return the logger module name by default return __name__ def create_logger(name=None): """lazily create a Logger object with the proper path, which is returned by find_parent_module() by default, or is provided via the commandline this is really a shortcut for: logger = logging.getLogger(__name__) we use it to avoid errors and provide a more standard API. We must create the logger lazily, because this is usually called from module level (and thus executed at import time - BEFORE setup_logging() was called). By doing it lazily we can do the setup first, we just have to be careful not to call any logger methods before the setup_logging() call. If you try, you'll get an exception. """ class LazyLogger: def __init__(self, name=None): self.__name = name or find_parent_module() self.__real_logger = None @property def __logger(self): if self.__real_logger is None: if not configured: raise Exception("tried to call a logger before setup_logging() was called") self.__real_logger = logging.getLogger(self.__name) return self.__real_logger def setLevel(self, *args, **kw): return self.__logger.setLevel(*args, **kw) def log(self, *args, **kw): return self.__logger.log(*args, **kw) def exception(self, *args, **kw): return self.__logger.exception(*args, **kw) def debug(self, *args, **kw): return self.__logger.debug(*args, **kw) def info(self, *args, **kw): return self.__logger.info(*args, **kw) def warning(self, *args, **kw): return self.__logger.warning(*args, **kw) def error(self, *args, **kw): return self.__logger.error(*args, **kw) def critical(self, *args, **kw): return self.__logger.critical(*args, **kw) return LazyLogger(name) borgbackup-1.0.13/borg/lrucache.py0000644000175000017500000000223513431513133016761 0ustar useruser00000000000000class LRUCache: def __init__(self, capacity, dispose): self._cache = {} self._lru = [] self._capacity = capacity self._dispose = dispose def __setitem__(self, key, value): assert key not in self._cache, ( "Unexpected attempt to replace a cached item," " without first deleting the old item.") self._lru.append(key) while len(self._lru) > self._capacity: del self[self._lru[0]] self._cache[key] = value def __getitem__(self, key): value = self._cache[key] # raise KeyError if not found self._lru.remove(key) self._lru.append(key) return value def __delitem__(self, key): value = self._cache.pop(key) # raise KeyError if not found self._dispose(value) self._lru.remove(key) def __contains__(self, key): return key in self._cache def clear(self): for value in self._cache.values(): self._dispose(value) self._cache.clear() # useful for testing def items(self): return self._cache.items() def __len__(self): return len(self._cache) borgbackup-1.0.13/borg/paperkey.html0000644000175000017500000020075313431513133017334 0ustar useruser00000000000000 BorgBackup Printable Key Template

To create a printable key, either paste the contents of your keyfile or a key export in the text field below, or select a key export file.

To create a key export use 

borg key export /path/to/repository exportfile.txt

If you are using keyfile mode, keyfiles are usually stored in $HOME/.config/borg/keys/

You can edit the parts with light blue border in the print preview below by click into them.

Key security: This print template will never send anything to remote servers. But keep in mind, that printing might involve computers that can store the printed image, for example with cloud printing services, or networked printers.

QR error correction:
QR code size
Text size
Text columns
BorgBackup Printable Key Backup
To restore either scan the QR code below, decode it and import it using 
borg key import /path/to/repo scannedfile
Or run 
borg key import --paper /path/to/repo
and type in the text below.

Notes:
borgbackup-1.0.13/borg/platform.py0000644000175000017500000000236613431513133017024 0ustar useruser00000000000000import errno import os import subprocess import sys from .helpers import prepare_subprocess_env # POSIX-only, from borg 1.1 platform.base def sync_dir(path): fd = os.open(path, os.O_RDONLY) try: os.fsync(fd) except OSError as os_error: # Some network filesystems don't support this and fail with EINVAL. # Other error codes (e.g. EIO) shouldn't be silenced. if os_error.errno != errno.EINVAL: raise finally: os.close(fd) # most POSIX platforms (but not Linux), see also borg 1.1 platform.base def umount(mountpoint): env = prepare_subprocess_env(system=True) return subprocess.call(['umount', mountpoint], env=env) if sys.platform.startswith('linux'): # pragma: linux only from .platform_linux import acl_get, acl_set, umount, API_VERSION elif sys.platform.startswith('freebsd'): # pragma: freebsd only from .platform_freebsd import acl_get, acl_set, API_VERSION elif sys.platform == 'darwin': # pragma: darwin only from .platform_darwin import acl_get, acl_set, API_VERSION else: # pragma: unknown platform only API_VERSION = '1.0_01' def acl_get(path, item, st, numeric_owner=False): pass def acl_set(path, item, numeric_owner=False): pass borgbackup-1.0.13/borg/platform_darwin.c0000644000175000017500000065346113431514213020172 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__platform_darwin #define __PYX_HAVE_API__borg__platform_darwin /* Early includes */ #include "sys/acl.h" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "borg/platform_darwin.pyx", }; /*--- Type declarations ---*/ /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* GetModuleGlobalName.proto */ #if CYTHON_USE_DICT_VERSIONS #define __Pyx_GetModuleGlobalName(var, name) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } #define __Pyx_GetModuleGlobalNameUncached(var, name) {\ PY_UINT64_T __pyx_dict_version;\ PyObject *__pyx_dict_cached_value;\ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value); #else #define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name) #define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name) static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name); #endif /* PyCFunctionFastCall.proto */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs); #else #define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL) #endif /* PyFunctionFastCall.proto */ #if CYTHON_FAST_PYCALL #define __Pyx_PyFunction_FastCall(func, args, nargs)\ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL) #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #else #define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs) #endif #define __Pyx_BUILD_ASSERT_EXPR(cond)\ (sizeof(char [1 - 2*!(cond)]) - 1) #ifndef Py_MEMBER_SIZE #define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) #endif static size_t __pyx_pyframe_localsplus_offset = 0; #include "frameobject.h" #define __Pxy_PyFrame_Initialize_Offsets()\ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\ (void)(__pyx_pyframe_localsplus_offset = PyFrame_Type.tp_basicsize - Py_MEMBER_SIZE(PyFrameObject, f_localsplus))) #define __Pyx_PyFrame_GetLocalsplus(frame)\ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset)) #endif /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* PyObjectCall2Args.proto */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2); /* PyObjectCallMethO.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg); #endif /* PyObjectCallOneArg.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg); /* GetItemInt.proto */ #define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\ __Pyx_GetItemInt_Generic(o, to_py_func(i)))) #define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); #define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j); static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, int wraparound, int boundscheck); /* IncludeStringH.proto */ #include /* BytesEquals.proto */ static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2, int equals); /* UnicodeEquals.proto */ static CYTHON_INLINE int __Pyx_PyUnicode_Equals(PyObject* s1, PyObject* s2, int equals); /* StrEquals.proto */ #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Equals __Pyx_PyUnicode_Equals #else #define __Pyx_PyString_Equals __Pyx_PyBytes_Equals #endif /* SetItemInt.proto */ #define __Pyx_SetItemInt(o, i, v, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_SetItemInt_Fast(o, (Py_ssize_t)i, v, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list assignment index out of range"), -1) :\ __Pyx_SetItemInt_Generic(o, to_py_func(i), v))) static int __Pyx_SetItemInt_Generic(PyObject *o, PyObject *j, PyObject *v); static CYTHON_INLINE int __Pyx_SetItemInt_Fast(PyObject *o, Py_ssize_t i, PyObject *v, int is_list, int wraparound, int boundscheck); /* ListAppend.proto */ #if CYTHON_USE_PYLIST_INTERNALS && CYTHON_ASSUME_SAFE_MACROS static CYTHON_INLINE int __Pyx_PyList_Append(PyObject* list, PyObject* x) { PyListObject* L = (PyListObject*) list; Py_ssize_t len = Py_SIZE(list); if (likely(L->allocated > len) & likely(len > (L->allocated >> 1))) { Py_INCREF(x); PyList_SET_ITEM(list, len, x); Py_SIZE(list) = len+1; return 0; } return PyList_Append(list, x); } #else #define __Pyx_PyList_Append(L,x) PyList_Append(L,x) #endif /* StringJoin.proto */ #if PY_MAJOR_VERSION < 3 #define __Pyx_PyString_Join __Pyx_PyBytes_Join #define __Pyx_PyBaseString_Join(s, v) (PyUnicode_CheckExact(s) ? PyUnicode_Join(s, v) : __Pyx_PyBytes_Join(s, v)) #else #define __Pyx_PyString_Join PyUnicode_Join #define __Pyx_PyBaseString_Join PyUnicode_Join #endif #if CYTHON_COMPILING_IN_CPYTHON #if PY_MAJOR_VERSION < 3 #define __Pyx_PyBytes_Join _PyString_Join #else #define __Pyx_PyBytes_Join _PyBytes_Join #endif #else static CYTHON_INLINE PyObject* __Pyx_PyBytes_Join(PyObject* sep, PyObject* values); #endif /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* GetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_GetException(type, value, tb) __Pyx__GetException(__pyx_tstate, type, value, tb) static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb); #endif /* SwapException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSwap(type, value, tb) __Pyx__ExceptionSwap(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb); #endif /* GetTopmostException.proto */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate); #endif /* SaveResetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSave(type, value, tb) __Pyx__ExceptionSave(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #define __Pyx_ExceptionReset(type, value, tb) __Pyx__ExceptionReset(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); #else #define __Pyx_ExceptionSave(type, value, tb) PyErr_GetExcInfo(type, value, tb) #define __Pyx_ExceptionReset(type, value, tb) PyErr_SetExcInfo(type, value, tb) #endif /* ObjectGetItem.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key); #else #define __Pyx_PyObject_GetItem(obj, key) PyObject_GetItem(obj, key) #endif /* PyErrExceptionMatches.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_ExceptionMatches(err) __Pyx_PyErr_ExceptionMatchesInState(__pyx_tstate, err) static CYTHON_INLINE int __Pyx_PyErr_ExceptionMatchesInState(PyThreadState* tstate, PyObject* err); #else #define __Pyx_PyErr_ExceptionMatches(err) PyErr_ExceptionMatches(err) #endif /* Import.proto */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level); /* ImportFrom.proto */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name); /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); /* Module declarations from 'borg.platform_darwin' */ #define __Pyx_MODULE_NAME "borg.platform_darwin" extern int __pyx_module_is_main_borg__platform_darwin; int __pyx_module_is_main_borg__platform_darwin = 0; /* Implementation of 'borg.platform_darwin' */ static PyObject *__pyx_builtin_KeyError; static const char __pyx_k_[] = "\n"; static const char __pyx_k__2[] = ":"; static const char __pyx_k__3[] = ""; static const char __pyx_k_os[] = "os"; static const char __pyx_k_st[] = "st"; static const char __pyx_k_acl[] = "acl"; static const char __pyx_k_item[] = "item"; static const char __pyx_k_join[] = "join"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_name[] = "__name__"; static const char __pyx_k_path[] = "path"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_text[] = "text"; static const char __pyx_k_user[] = "user"; static const char __pyx_k_entry[] = "entry"; static const char __pyx_k_group[] = "group"; static const char __pyx_k_split[] = "split"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_fields[] = "fields"; static const char __pyx_k_import[] = "__import__"; static const char __pyx_k_acl_get[] = "acl_get"; static const char __pyx_k_acl_set[] = "acl_set"; static const char __pyx_k_entries[] = "entries"; static const char __pyx_k_helpers[] = "helpers"; static const char __pyx_k_KeyError[] = "KeyError"; static const char __pyx_k_fsencode[] = "fsencode"; static const char __pyx_k_user2uid[] = "user2uid"; static const char __pyx_k_group2gid[] = "group2gid"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_safe_decode[] = "safe_decode"; static const char __pyx_k_safe_encode[] = "safe_encode"; static const char __pyx_k_acl_extended[] = "acl_extended"; static const char __pyx_k_numeric_owner[] = "numeric_owner"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_borg_platform_darwin[] = "borg.platform_darwin"; static const char __pyx_k_borg_platform_darwin_pyx[] = "borg/platform_darwin.pyx"; static const char __pyx_k_remove_non_numeric_identifier[] = "_remove_non_numeric_identifier"; static const char __pyx_k_remove_numeric_id_if_possible[] = "_remove_numeric_id_if_possible"; static PyObject *__pyx_kp_s_; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_n_s_KeyError; static PyObject *__pyx_kp_s__2; static PyObject *__pyx_kp_s__3; static PyObject *__pyx_n_s_acl; static PyObject *__pyx_n_b_acl_extended; static PyObject *__pyx_n_s_acl_get; static PyObject *__pyx_n_s_acl_set; static PyObject *__pyx_n_s_borg_platform_darwin; static PyObject *__pyx_kp_s_borg_platform_darwin_pyx; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_entries; static PyObject *__pyx_n_s_entry; static PyObject *__pyx_n_s_fields; static PyObject *__pyx_n_s_fsencode; static PyObject *__pyx_n_s_group; static PyObject *__pyx_n_s_group2gid; static PyObject *__pyx_n_s_helpers; static PyObject *__pyx_n_s_import; static PyObject *__pyx_n_s_item; static PyObject *__pyx_n_s_join; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_name; static PyObject *__pyx_n_s_numeric_owner; static PyObject *__pyx_n_s_os; static PyObject *__pyx_n_s_path; static PyObject *__pyx_n_s_remove_non_numeric_identifier; static PyObject *__pyx_n_s_remove_numeric_id_if_possible; static PyObject *__pyx_n_s_safe_decode; static PyObject *__pyx_n_s_safe_encode; static PyObject *__pyx_n_s_split; static PyObject *__pyx_n_s_st; static PyObject *__pyx_n_s_test; static PyObject *__pyx_n_s_text; static PyObject *__pyx_n_s_user; static PyObject *__pyx_n_s_user2uid; static PyObject *__pyx_pf_4borg_15platform_darwin__remove_numeric_id_if_possible(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_acl); /* proto */ static PyObject *__pyx_pf_4borg_15platform_darwin_2_remove_non_numeric_identifier(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_acl); /* proto */ static PyObject *__pyx_pf_4borg_15platform_darwin_4acl_get(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, CYTHON_UNUSED PyObject *__pyx_v_st, PyObject *__pyx_v_numeric_owner); /* proto */ static PyObject *__pyx_pf_4borg_15platform_darwin_6acl_set(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_numeric_owner); /* proto */ static PyObject *__pyx_tuple__4; static PyObject *__pyx_tuple__6; static PyObject *__pyx_tuple__8; static PyObject *__pyx_tuple__10; static PyObject *__pyx_codeobj__5; static PyObject *__pyx_codeobj__7; static PyObject *__pyx_codeobj__9; static PyObject *__pyx_codeobj__11; /* Late includes */ /* "borg/platform_darwin.pyx":19 * * * def _remove_numeric_id_if_possible(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_15platform_darwin_1_remove_numeric_id_if_possible(PyObject *__pyx_self, PyObject *__pyx_v_acl); /*proto*/ static char __pyx_doc_4borg_15platform_darwin__remove_numeric_id_if_possible[] = "Replace the user/group field with the local uid/gid if possible\n "; static PyMethodDef __pyx_mdef_4borg_15platform_darwin_1_remove_numeric_id_if_possible = {"_remove_numeric_id_if_possible", (PyCFunction)__pyx_pw_4borg_15platform_darwin_1_remove_numeric_id_if_possible, METH_O, __pyx_doc_4borg_15platform_darwin__remove_numeric_id_if_possible}; static PyObject *__pyx_pw_4borg_15platform_darwin_1_remove_numeric_id_if_possible(PyObject *__pyx_self, PyObject *__pyx_v_acl) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("_remove_numeric_id_if_possible (wrapper)", 0); __pyx_r = __pyx_pf_4borg_15platform_darwin__remove_numeric_id_if_possible(__pyx_self, ((PyObject *)__pyx_v_acl)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_15platform_darwin__remove_numeric_id_if_possible(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_acl) { PyObject *__pyx_v_entries = NULL; PyObject *__pyx_v_entry = NULL; PyObject *__pyx_v_fields = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; Py_ssize_t __pyx_t_5; PyObject *(*__pyx_t_6)(PyObject *); int __pyx_t_7; PyObject *__pyx_t_8 = NULL; int __pyx_t_9; int __pyx_t_10; __Pyx_RefNannySetupContext("_remove_numeric_id_if_possible", 0); /* "borg/platform_darwin.pyx":22 * """Replace the user/group field with the local uid/gid if possible * """ * entries = [] # <<<<<<<<<<<<<< * for entry in safe_decode(acl).split('\n'): * if entry: */ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 22, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_entries = ((PyObject*)__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":23 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * fields = entry.split(':') */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_4, __pyx_v_acl) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_acl); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_split); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_kp_s_) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_kp_s_); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (likely(PyList_CheckExact(__pyx_t_1)) || PyTuple_CheckExact(__pyx_t_1)) { __pyx_t_3 = __pyx_t_1; __Pyx_INCREF(__pyx_t_3); __pyx_t_5 = 0; __pyx_t_6 = NULL; } else { __pyx_t_5 = -1; __pyx_t_3 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = Py_TYPE(__pyx_t_3)->tp_iternext; if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 23, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; for (;;) { if (likely(!__pyx_t_6)) { if (likely(PyList_CheckExact(__pyx_t_3))) { if (__pyx_t_5 >= PyList_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyList_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 23, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } else { if (__pyx_t_5 >= PyTuple_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyTuple_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 23, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 23, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } } else { __pyx_t_1 = __pyx_t_6(__pyx_t_3); if (unlikely(!__pyx_t_1)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 23, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_1); } __Pyx_XDECREF_SET(__pyx_v_entry, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":24 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * fields = entry.split(':') * if fields[0] == 'user': */ __pyx_t_7 = __Pyx_PyObject_IsTrue(__pyx_v_entry); if (unlikely(__pyx_t_7 < 0)) __PYX_ERR(0, 24, __pyx_L1_error) if (__pyx_t_7) { /* "borg/platform_darwin.pyx":25 * for entry in safe_decode(acl).split('\n'): * if entry: * fields = entry.split(':') # <<<<<<<<<<<<<< * if fields[0] == 'user': * if user2uid(fields[2]) is not None: */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_split); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 25, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_kp_s__2) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s__2); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 25, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF_SET(__pyx_v_fields, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":26 * if entry: * fields = entry.split(':') * if fields[0] == 'user': # <<<<<<<<<<<<<< * if user2uid(fields[2]) is not None: * fields[1] = fields[3] = '' */ __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_fields, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 26, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_7 = (__Pyx_PyString_Equals(__pyx_t_1, __pyx_n_s_user, Py_EQ)); if (unlikely(__pyx_t_7 < 0)) __PYX_ERR(0, 26, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (__pyx_t_7) { /* "borg/platform_darwin.pyx":27 * fields = entry.split(':') * if fields[0] == 'user': * if user2uid(fields[2]) is not None: # <<<<<<<<<<<<<< * fields[1] = fields[3] = '' * elif fields[0] == 'group': */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_user2uid); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 27, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = __Pyx_GetItemInt(__pyx_v_fields, 2, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 27, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_8, __pyx_t_4) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_t_4); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 27, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_7 = (__pyx_t_1 != Py_None); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_9 = (__pyx_t_7 != 0); if (__pyx_t_9) { /* "borg/platform_darwin.pyx":28 * if fields[0] == 'user': * if user2uid(fields[2]) is not None: * fields[1] = fields[3] = '' # <<<<<<<<<<<<<< * elif fields[0] == 'group': * if group2gid(fields[2]) is not None: */ if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 1, __pyx_kp_s__3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 28, __pyx_L1_error) if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 3, __pyx_kp_s__3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 28, __pyx_L1_error) /* "borg/platform_darwin.pyx":27 * fields = entry.split(':') * if fields[0] == 'user': * if user2uid(fields[2]) is not None: # <<<<<<<<<<<<<< * fields[1] = fields[3] = '' * elif fields[0] == 'group': */ } /* "borg/platform_darwin.pyx":26 * if entry: * fields = entry.split(':') * if fields[0] == 'user': # <<<<<<<<<<<<<< * if user2uid(fields[2]) is not None: * fields[1] = fields[3] = '' */ goto __pyx_L6; } /* "borg/platform_darwin.pyx":29 * if user2uid(fields[2]) is not None: * fields[1] = fields[3] = '' * elif fields[0] == 'group': # <<<<<<<<<<<<<< * if group2gid(fields[2]) is not None: * fields[1] = fields[3] = '' */ __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_fields, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 29, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_9 = (__Pyx_PyString_Equals(__pyx_t_1, __pyx_n_s_group, Py_EQ)); if (unlikely(__pyx_t_9 < 0)) __PYX_ERR(0, 29, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (__pyx_t_9) { /* "borg/platform_darwin.pyx":30 * fields[1] = fields[3] = '' * elif fields[0] == 'group': * if group2gid(fields[2]) is not None: # <<<<<<<<<<<<<< * fields[1] = fields[3] = '' * entries.append(':'.join(fields)) */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_group2gid); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 30, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = __Pyx_GetItemInt(__pyx_v_fields, 2, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 30, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_8 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_8 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_8)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_8); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_8) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_8, __pyx_t_4) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_t_4); __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 30, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_9 = (__pyx_t_1 != Py_None); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_7 = (__pyx_t_9 != 0); if (__pyx_t_7) { /* "borg/platform_darwin.pyx":31 * elif fields[0] == 'group': * if group2gid(fields[2]) is not None: * fields[1] = fields[3] = '' # <<<<<<<<<<<<<< * entries.append(':'.join(fields)) * return safe_encode('\n'.join(entries)) */ if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 1, __pyx_kp_s__3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 31, __pyx_L1_error) if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 3, __pyx_kp_s__3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 31, __pyx_L1_error) /* "borg/platform_darwin.pyx":30 * fields[1] = fields[3] = '' * elif fields[0] == 'group': * if group2gid(fields[2]) is not None: # <<<<<<<<<<<<<< * fields[1] = fields[3] = '' * entries.append(':'.join(fields)) */ } /* "borg/platform_darwin.pyx":29 * if user2uid(fields[2]) is not None: * fields[1] = fields[3] = '' * elif fields[0] == 'group': # <<<<<<<<<<<<<< * if group2gid(fields[2]) is not None: * fields[1] = fields[3] = '' */ } __pyx_L6:; /* "borg/platform_darwin.pyx":32 * if group2gid(fields[2]) is not None: * fields[1] = fields[3] = '' * entries.append(':'.join(fields)) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ __pyx_t_1 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_v_fields); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 32, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_10 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_1); if (unlikely(__pyx_t_10 == ((int)-1))) __PYX_ERR(0, 32, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":24 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * fields = entry.split(':') * if fields[0] == 'user': */ } /* "borg/platform_darwin.pyx":23 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * fields = entry.split(':') */ } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_darwin.pyx":33 * fields[1] = fields[3] = '' * entries.append(':'.join(fields)) * return safe_encode('\n'.join(entries)) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 33, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyString_Join(__pyx_kp_s_, __pyx_v_entries); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 33, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_4, __pyx_t_2) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_t_2); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 33, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* "borg/platform_darwin.pyx":19 * * * def _remove_numeric_id_if_possible(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_8); __Pyx_AddTraceback("borg.platform_darwin._remove_numeric_id_if_possible", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_entries); __Pyx_XDECREF(__pyx_v_entry); __Pyx_XDECREF(__pyx_v_fields); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_darwin.pyx":36 * * * def _remove_non_numeric_identifier(acl): # <<<<<<<<<<<<<< * """Remove user and group names from the acl * """ */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_15platform_darwin_3_remove_non_numeric_identifier(PyObject *__pyx_self, PyObject *__pyx_v_acl); /*proto*/ static char __pyx_doc_4borg_15platform_darwin_2_remove_non_numeric_identifier[] = "Remove user and group names from the acl\n "; static PyMethodDef __pyx_mdef_4borg_15platform_darwin_3_remove_non_numeric_identifier = {"_remove_non_numeric_identifier", (PyCFunction)__pyx_pw_4borg_15platform_darwin_3_remove_non_numeric_identifier, METH_O, __pyx_doc_4borg_15platform_darwin_2_remove_non_numeric_identifier}; static PyObject *__pyx_pw_4borg_15platform_darwin_3_remove_non_numeric_identifier(PyObject *__pyx_self, PyObject *__pyx_v_acl) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("_remove_non_numeric_identifier (wrapper)", 0); __pyx_r = __pyx_pf_4borg_15platform_darwin_2_remove_non_numeric_identifier(__pyx_self, ((PyObject *)__pyx_v_acl)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_15platform_darwin_2_remove_non_numeric_identifier(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_acl) { PyObject *__pyx_v_entries = NULL; PyObject *__pyx_v_entry = NULL; PyObject *__pyx_v_fields = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; Py_ssize_t __pyx_t_5; PyObject *(*__pyx_t_6)(PyObject *); int __pyx_t_7; int __pyx_t_8; int __pyx_t_9; __Pyx_RefNannySetupContext("_remove_non_numeric_identifier", 0); /* "borg/platform_darwin.pyx":39 * """Remove user and group names from the acl * """ * entries = [] # <<<<<<<<<<<<<< * for entry in safe_decode(acl).split('\n'): * if entry: */ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_entries = ((PyObject*)__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":40 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * fields = entry.split(':') */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_4, __pyx_v_acl) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_acl); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_split); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_kp_s_) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_kp_s_); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (likely(PyList_CheckExact(__pyx_t_1)) || PyTuple_CheckExact(__pyx_t_1)) { __pyx_t_3 = __pyx_t_1; __Pyx_INCREF(__pyx_t_3); __pyx_t_5 = 0; __pyx_t_6 = NULL; } else { __pyx_t_5 = -1; __pyx_t_3 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = Py_TYPE(__pyx_t_3)->tp_iternext; if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 40, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; for (;;) { if (likely(!__pyx_t_6)) { if (likely(PyList_CheckExact(__pyx_t_3))) { if (__pyx_t_5 >= PyList_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyList_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 40, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } else { if (__pyx_t_5 >= PyTuple_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyTuple_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 40, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } } else { __pyx_t_1 = __pyx_t_6(__pyx_t_3); if (unlikely(!__pyx_t_1)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 40, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_1); } __Pyx_XDECREF_SET(__pyx_v_entry, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":41 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * fields = entry.split(':') * if fields[0] in ('user', 'group'): */ __pyx_t_7 = __Pyx_PyObject_IsTrue(__pyx_v_entry); if (unlikely(__pyx_t_7 < 0)) __PYX_ERR(0, 41, __pyx_L1_error) if (__pyx_t_7) { /* "borg/platform_darwin.pyx":42 * for entry in safe_decode(acl).split('\n'): * if entry: * fields = entry.split(':') # <<<<<<<<<<<<<< * if fields[0] in ('user', 'group'): * fields[2] = '' */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_split); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_kp_s__2) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s__2); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF_SET(__pyx_v_fields, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":43 * if entry: * fields = entry.split(':') * if fields[0] in ('user', 'group'): # <<<<<<<<<<<<<< * fields[2] = '' * entries.append(':'.join(fields)) */ __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_fields, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 43, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_8 = (__Pyx_PyString_Equals(__pyx_t_1, __pyx_n_s_user, Py_EQ)); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 43, __pyx_L1_error) if (!__pyx_t_8) { } else { __pyx_t_7 = __pyx_t_8; goto __pyx_L7_bool_binop_done; } __pyx_t_8 = (__Pyx_PyString_Equals(__pyx_t_1, __pyx_n_s_group, Py_EQ)); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 43, __pyx_L1_error) __pyx_t_7 = __pyx_t_8; __pyx_L7_bool_binop_done:; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_8 = (__pyx_t_7 != 0); if (__pyx_t_8) { /* "borg/platform_darwin.pyx":44 * fields = entry.split(':') * if fields[0] in ('user', 'group'): * fields[2] = '' # <<<<<<<<<<<<<< * entries.append(':'.join(fields)) * else: */ if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 2, __pyx_kp_s__3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 44, __pyx_L1_error) /* "borg/platform_darwin.pyx":45 * if fields[0] in ('user', 'group'): * fields[2] = '' * entries.append(':'.join(fields)) # <<<<<<<<<<<<<< * else: * entries.append(entry) */ __pyx_t_1 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_v_fields); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_9 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_1); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":43 * if entry: * fields = entry.split(':') * if fields[0] in ('user', 'group'): # <<<<<<<<<<<<<< * fields[2] = '' * entries.append(':'.join(fields)) */ goto __pyx_L6; } /* "borg/platform_darwin.pyx":47 * entries.append(':'.join(fields)) * else: * entries.append(entry) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ /*else*/ { __pyx_t_9 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_v_entry); if (unlikely(__pyx_t_9 == ((int)-1))) __PYX_ERR(0, 47, __pyx_L1_error) } __pyx_L6:; /* "borg/platform_darwin.pyx":41 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * fields = entry.split(':') * if fields[0] in ('user', 'group'): */ } /* "borg/platform_darwin.pyx":40 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * fields = entry.split(':') */ } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_darwin.pyx":48 * else: * entries.append(entry) * return safe_encode('\n'.join(entries)) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyString_Join(__pyx_kp_s_, __pyx_v_entries); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_4, __pyx_t_2) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_t_2); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 48, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* "borg/platform_darwin.pyx":36 * * * def _remove_non_numeric_identifier(acl): # <<<<<<<<<<<<<< * """Remove user and group names from the acl * """ */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.platform_darwin._remove_non_numeric_identifier", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_entries); __Pyx_XDECREF(__pyx_v_entry); __Pyx_XDECREF(__pyx_v_fields); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_darwin.pyx":51 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * cdef char *text = NULL */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_15platform_darwin_5acl_get(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_15platform_darwin_5acl_get = {"acl_get", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_15platform_darwin_5acl_get, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_15platform_darwin_5acl_get(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_path = 0; PyObject *__pyx_v_item = 0; CYTHON_UNUSED PyObject *__pyx_v_st = 0; PyObject *__pyx_v_numeric_owner = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_get (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_path,&__pyx_n_s_item,&__pyx_n_s_st,&__pyx_n_s_numeric_owner,0}; PyObject* values[4] = {0,0,0,0}; values[3] = ((PyObject *)Py_False); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_item)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, 1); __PYX_ERR(0, 51, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_st)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, 2); __PYX_ERR(0, 51, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_numeric_owner); if (value) { values[3] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "acl_get") < 0)) __PYX_ERR(0, 51, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_path = values[0]; __pyx_v_item = values[1]; __pyx_v_st = values[2]; __pyx_v_numeric_owner = values[3]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 51, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.platform_darwin.acl_get", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_15platform_darwin_4acl_get(__pyx_self, __pyx_v_path, __pyx_v_item, __pyx_v_st, __pyx_v_numeric_owner); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_15platform_darwin_4acl_get(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, CYTHON_UNUSED PyObject *__pyx_v_st, PyObject *__pyx_v_numeric_owner) { acl_t __pyx_v_acl; char *__pyx_v_text; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; char const *__pyx_t_4; int __pyx_t_5; PyObject *__pyx_t_6 = NULL; int __pyx_t_7; int __pyx_t_8; char const *__pyx_t_9; PyObject *__pyx_t_10 = NULL; PyObject *__pyx_t_11 = NULL; PyObject *__pyx_t_12 = NULL; PyObject *__pyx_t_13 = NULL; PyObject *__pyx_t_14 = NULL; PyObject *__pyx_t_15 = NULL; __Pyx_RefNannySetupContext("acl_get", 0); /* "borg/platform_darwin.pyx":52 * * def acl_get(path, item, st, numeric_owner=False): * cdef acl_t acl = NULL # <<<<<<<<<<<<<< * cdef char *text = NULL * try: */ __pyx_v_acl = NULL; /* "borg/platform_darwin.pyx":53 * def acl_get(path, item, st, numeric_owner=False): * cdef acl_t acl = NULL * cdef char *text = NULL # <<<<<<<<<<<<<< * try: * acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) */ __pyx_v_text = NULL; /* "borg/platform_darwin.pyx":54 * cdef acl_t acl = NULL * cdef char *text = NULL * try: # <<<<<<<<<<<<<< * acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) * if acl == NULL: */ /*try:*/ { /* "borg/platform_darwin.pyx":55 * cdef char *text = NULL * try: * acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) # <<<<<<<<<<<<<< * if acl == NULL: * return */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_os); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 55, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 55, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_path); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 55, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(__pyx_t_1 == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 55, __pyx_L4_error) } __pyx_t_4 = __Pyx_PyBytes_AsString(__pyx_t_1); if (unlikely((!__pyx_t_4) && PyErr_Occurred())) __PYX_ERR(0, 55, __pyx_L4_error) __pyx_v_acl = acl_get_link_np(__pyx_t_4, ACL_TYPE_EXTENDED); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":56 * try: * acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) * if acl == NULL: # <<<<<<<<<<<<<< * return * text = acl_to_text(acl, NULL) */ __pyx_t_5 = ((__pyx_v_acl == NULL) != 0); if (__pyx_t_5) { /* "borg/platform_darwin.pyx":57 * acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) * if acl == NULL: * return # <<<<<<<<<<<<<< * text = acl_to_text(acl, NULL) * if text == NULL: */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L3_return; /* "borg/platform_darwin.pyx":56 * try: * acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) * if acl == NULL: # <<<<<<<<<<<<<< * return * text = acl_to_text(acl, NULL) */ } /* "borg/platform_darwin.pyx":58 * if acl == NULL: * return * text = acl_to_text(acl, NULL) # <<<<<<<<<<<<<< * if text == NULL: * return */ __pyx_v_text = acl_to_text(__pyx_v_acl, NULL); /* "borg/platform_darwin.pyx":59 * return * text = acl_to_text(acl, NULL) * if text == NULL: # <<<<<<<<<<<<<< * return * if numeric_owner: */ __pyx_t_5 = ((__pyx_v_text == NULL) != 0); if (__pyx_t_5) { /* "borg/platform_darwin.pyx":60 * text = acl_to_text(acl, NULL) * if text == NULL: * return # <<<<<<<<<<<<<< * if numeric_owner: * item[b'acl_extended'] = _remove_non_numeric_identifier(text) */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L3_return; /* "borg/platform_darwin.pyx":59 * return * text = acl_to_text(acl, NULL) * if text == NULL: # <<<<<<<<<<<<<< * return * if numeric_owner: */ } /* "borg/platform_darwin.pyx":61 * if text == NULL: * return * if numeric_owner: # <<<<<<<<<<<<<< * item[b'acl_extended'] = _remove_non_numeric_identifier(text) * else: */ __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 61, __pyx_L4_error) if (__pyx_t_5) { /* "borg/platform_darwin.pyx":62 * return * if numeric_owner: * item[b'acl_extended'] = _remove_non_numeric_identifier(text) # <<<<<<<<<<<<<< * else: * item[b'acl_extended'] = text */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_remove_non_numeric_identifier); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 62, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_2 = __Pyx_PyBytes_FromString(__pyx_v_text); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 62, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_6, __pyx_t_2) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_t_2); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 62, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(PyObject_SetItem(__pyx_v_item, __pyx_n_b_acl_extended, __pyx_t_1) < 0)) __PYX_ERR(0, 62, __pyx_L4_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":61 * if text == NULL: * return * if numeric_owner: # <<<<<<<<<<<<<< * item[b'acl_extended'] = _remove_non_numeric_identifier(text) * else: */ goto __pyx_L8; } /* "borg/platform_darwin.pyx":64 * item[b'acl_extended'] = _remove_non_numeric_identifier(text) * else: * item[b'acl_extended'] = text # <<<<<<<<<<<<<< * finally: * acl_free(text) */ /*else*/ { __pyx_t_1 = __Pyx_PyBytes_FromString(__pyx_v_text); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 64, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_1); if (unlikely(PyObject_SetItem(__pyx_v_item, __pyx_n_b_acl_extended, __pyx_t_1) < 0)) __PYX_ERR(0, 64, __pyx_L4_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; } __pyx_L8:; } /* "borg/platform_darwin.pyx":66 * item[b'acl_extended'] = text * finally: * acl_free(text) # <<<<<<<<<<<<<< * acl_free(acl) * */ /*finally:*/ { /*normal exit:*/{ (void)(acl_free(__pyx_v_text)); /* "borg/platform_darwin.pyx":67 * finally: * acl_free(text) * acl_free(acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_acl)); goto __pyx_L5; } __pyx_L4_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_10 = 0; __pyx_t_11 = 0; __pyx_t_12 = 0; __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_13, &__pyx_t_14, &__pyx_t_15); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_10, &__pyx_t_11, &__pyx_t_12) < 0)) __Pyx_ErrFetch(&__pyx_t_10, &__pyx_t_11, &__pyx_t_12); __Pyx_XGOTREF(__pyx_t_10); __Pyx_XGOTREF(__pyx_t_11); __Pyx_XGOTREF(__pyx_t_12); __Pyx_XGOTREF(__pyx_t_13); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_15); __pyx_t_7 = __pyx_lineno; __pyx_t_8 = __pyx_clineno; __pyx_t_9 = __pyx_filename; { /* "borg/platform_darwin.pyx":66 * item[b'acl_extended'] = text * finally: * acl_free(text) # <<<<<<<<<<<<<< * acl_free(acl) * */ (void)(acl_free(__pyx_v_text)); /* "borg/platform_darwin.pyx":67 * finally: * acl_free(text) * acl_free(acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_acl)); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_13); __Pyx_XGIVEREF(__pyx_t_14); __Pyx_XGIVEREF(__pyx_t_15); __Pyx_ExceptionReset(__pyx_t_13, __pyx_t_14, __pyx_t_15); } __Pyx_XGIVEREF(__pyx_t_10); __Pyx_XGIVEREF(__pyx_t_11); __Pyx_XGIVEREF(__pyx_t_12); __Pyx_ErrRestore(__pyx_t_10, __pyx_t_11, __pyx_t_12); __pyx_t_10 = 0; __pyx_t_11 = 0; __pyx_t_12 = 0; __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_lineno = __pyx_t_7; __pyx_clineno = __pyx_t_8; __pyx_filename = __pyx_t_9; goto __pyx_L1_error; } __pyx_L3_return: { __pyx_t_15 = __pyx_r; __pyx_r = 0; /* "borg/platform_darwin.pyx":66 * item[b'acl_extended'] = text * finally: * acl_free(text) # <<<<<<<<<<<<<< * acl_free(acl) * */ (void)(acl_free(__pyx_v_text)); /* "borg/platform_darwin.pyx":67 * finally: * acl_free(text) * acl_free(acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_acl)); __pyx_r = __pyx_t_15; __pyx_t_15 = 0; goto __pyx_L0; } __pyx_L5:; } /* "borg/platform_darwin.pyx":51 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * cdef char *text = NULL */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.platform_darwin.acl_get", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_darwin.pyx":70 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * try: */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_15platform_darwin_7acl_set(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static PyMethodDef __pyx_mdef_4borg_15platform_darwin_7acl_set = {"acl_set", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_15platform_darwin_7acl_set, METH_VARARGS|METH_KEYWORDS, 0}; static PyObject *__pyx_pw_4borg_15platform_darwin_7acl_set(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_path = 0; PyObject *__pyx_v_item = 0; PyObject *__pyx_v_numeric_owner = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_set (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_path,&__pyx_n_s_item,&__pyx_n_s_numeric_owner,0}; PyObject* values[3] = {0,0,0}; values[2] = ((PyObject *)Py_False); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_item)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_set", 0, 2, 3, 1); __PYX_ERR(0, 70, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_numeric_owner); if (value) { values[2] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "acl_set") < 0)) __PYX_ERR(0, 70, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_path = values[0]; __pyx_v_item = values[1]; __pyx_v_numeric_owner = values[2]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("acl_set", 0, 2, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 70, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.platform_darwin.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_15platform_darwin_6acl_set(__pyx_self, __pyx_v_path, __pyx_v_item, __pyx_v_numeric_owner); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_15platform_darwin_6acl_set(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_numeric_owner) { acl_t __pyx_v_acl; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; char const *__pyx_t_6; PyObject *__pyx_t_7 = NULL; PyObject *__pyx_t_8 = NULL; PyObject *__pyx_t_9 = NULL; int __pyx_t_10; char const *__pyx_t_11; int __pyx_t_12; char const *__pyx_t_13; PyObject *__pyx_t_14 = NULL; PyObject *__pyx_t_15 = NULL; PyObject *__pyx_t_16 = NULL; __Pyx_RefNannySetupContext("acl_set", 0); /* "borg/platform_darwin.pyx":71 * * def acl_set(path, item, numeric_owner=False): * cdef acl_t acl = NULL # <<<<<<<<<<<<<< * try: * try: */ __pyx_v_acl = NULL; /* "borg/platform_darwin.pyx":72 * def acl_set(path, item, numeric_owner=False): * cdef acl_t acl = NULL * try: # <<<<<<<<<<<<<< * try: * if numeric_owner: */ /*try:*/ { /* "borg/platform_darwin.pyx":73 * cdef acl_t acl = NULL * try: * try: # <<<<<<<<<<<<<< * if numeric_owner: * acl = acl_from_text(item[b'acl_extended']) */ { __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ExceptionSave(&__pyx_t_1, &__pyx_t_2, &__pyx_t_3); __Pyx_XGOTREF(__pyx_t_1); __Pyx_XGOTREF(__pyx_t_2); __Pyx_XGOTREF(__pyx_t_3); /*try:*/ { /* "borg/platform_darwin.pyx":74 * try: * try: * if numeric_owner: # <<<<<<<<<<<<<< * acl = acl_from_text(item[b'acl_extended']) * else: */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 74, __pyx_L6_error) if (__pyx_t_4) { /* "borg/platform_darwin.pyx":75 * try: * if numeric_owner: * acl = acl_from_text(item[b'acl_extended']) # <<<<<<<<<<<<<< * else: * acl = acl_from_text(_remove_numeric_id_if_possible(item[b'acl_extended'])) */ __pyx_t_5 = __Pyx_PyObject_GetItem(__pyx_v_item, __pyx_n_b_acl_extended); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 75, __pyx_L6_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = __Pyx_PyObject_AsString(__pyx_t_5); if (unlikely((!__pyx_t_6) && PyErr_Occurred())) __PYX_ERR(0, 75, __pyx_L6_error) __pyx_v_acl = acl_from_text(__pyx_t_6); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; /* "borg/platform_darwin.pyx":74 * try: * try: * if numeric_owner: # <<<<<<<<<<<<<< * acl = acl_from_text(item[b'acl_extended']) * else: */ goto __pyx_L12; } /* "borg/platform_darwin.pyx":77 * acl = acl_from_text(item[b'acl_extended']) * else: * acl = acl_from_text(_remove_numeric_id_if_possible(item[b'acl_extended'])) # <<<<<<<<<<<<<< * except KeyError: * return */ /*else*/ { __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_remove_numeric_id_if_possible); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 77, __pyx_L6_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_8 = __Pyx_PyObject_GetItem(__pyx_v_item, __pyx_n_b_acl_extended); if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 77, __pyx_L6_error) __Pyx_GOTREF(__pyx_t_8); __pyx_t_9 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_7))) { __pyx_t_9 = PyMethod_GET_SELF(__pyx_t_7); if (likely(__pyx_t_9)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_7); __Pyx_INCREF(__pyx_t_9); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_7, function); } } __pyx_t_5 = (__pyx_t_9) ? __Pyx_PyObject_Call2Args(__pyx_t_7, __pyx_t_9, __pyx_t_8) : __Pyx_PyObject_CallOneArg(__pyx_t_7, __pyx_t_8); __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 77, __pyx_L6_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; if (unlikely(__pyx_t_5 == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 77, __pyx_L6_error) } __pyx_t_6 = __Pyx_PyBytes_AsString(__pyx_t_5); if (unlikely((!__pyx_t_6) && PyErr_Occurred())) __PYX_ERR(0, 77, __pyx_L6_error) __pyx_v_acl = acl_from_text(__pyx_t_6); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; } __pyx_L12:; /* "borg/platform_darwin.pyx":73 * cdef acl_t acl = NULL * try: * try: # <<<<<<<<<<<<<< * if numeric_owner: * acl = acl_from_text(item[b'acl_extended']) */ } __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; goto __pyx_L11_try_end; __pyx_L6_error:; __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; /* "borg/platform_darwin.pyx":78 * else: * acl = acl_from_text(_remove_numeric_id_if_possible(item[b'acl_extended'])) * except KeyError: # <<<<<<<<<<<<<< * return * if acl == NULL: */ __pyx_t_10 = __Pyx_PyErr_ExceptionMatches(__pyx_builtin_KeyError); if (__pyx_t_10) { __Pyx_AddTraceback("borg.platform_darwin.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); if (__Pyx_GetException(&__pyx_t_5, &__pyx_t_7, &__pyx_t_8) < 0) __PYX_ERR(0, 78, __pyx_L8_except_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_GOTREF(__pyx_t_7); __Pyx_GOTREF(__pyx_t_8); /* "borg/platform_darwin.pyx":79 * acl = acl_from_text(_remove_numeric_id_if_possible(item[b'acl_extended'])) * except KeyError: * return # <<<<<<<<<<<<<< * if acl == NULL: * return */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; goto __pyx_L9_except_return; } goto __pyx_L8_except_error; __pyx_L8_except_error:; /* "borg/platform_darwin.pyx":73 * cdef acl_t acl = NULL * try: * try: # <<<<<<<<<<<<<< * if numeric_owner: * acl = acl_from_text(item[b'acl_extended']) */ __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L4_error; __pyx_L9_except_return:; __Pyx_XGIVEREF(__pyx_t_1); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_3); __Pyx_ExceptionReset(__pyx_t_1, __pyx_t_2, __pyx_t_3); goto __pyx_L3_return; __pyx_L11_try_end:; } /* "borg/platform_darwin.pyx":80 * except KeyError: * return * if acl == NULL: # <<<<<<<<<<<<<< * return * if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): */ __pyx_t_4 = ((__pyx_v_acl == NULL) != 0); if (__pyx_t_4) { /* "borg/platform_darwin.pyx":81 * return * if acl == NULL: * return # <<<<<<<<<<<<<< * if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): * return */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L3_return; /* "borg/platform_darwin.pyx":80 * except KeyError: * return * if acl == NULL: # <<<<<<<<<<<<<< * return * if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): */ } /* "borg/platform_darwin.pyx":82 * if acl == NULL: * return * if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): # <<<<<<<<<<<<<< * return * finally: */ __Pyx_GetModuleGlobalName(__pyx_t_7, __pyx_n_s_os); if (unlikely(!__pyx_t_7)) __PYX_ERR(0, 82, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_7); __pyx_t_5 = __Pyx_PyObject_GetAttrStr(__pyx_t_7, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 82, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_7); __pyx_t_7 = 0; __pyx_t_7 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_5))) { __pyx_t_7 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_7)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_7); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_8 = (__pyx_t_7) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_7, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_path); __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0; if (unlikely(!__pyx_t_8)) __PYX_ERR(0, 82, __pyx_L4_error) __Pyx_GOTREF(__pyx_t_8); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; if (unlikely(__pyx_t_8 == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 82, __pyx_L4_error) } __pyx_t_11 = __Pyx_PyBytes_AsString(__pyx_t_8); if (unlikely((!__pyx_t_11) && PyErr_Occurred())) __PYX_ERR(0, 82, __pyx_L4_error) __pyx_t_4 = (acl_set_link_np(__pyx_t_11, ACL_TYPE_EXTENDED, __pyx_v_acl) != 0); __Pyx_DECREF(__pyx_t_8); __pyx_t_8 = 0; if (__pyx_t_4) { /* "borg/platform_darwin.pyx":83 * return * if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): * return # <<<<<<<<<<<<<< * finally: * acl_free(acl) */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L3_return; /* "borg/platform_darwin.pyx":82 * if acl == NULL: * return * if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): # <<<<<<<<<<<<<< * return * finally: */ } } /* "borg/platform_darwin.pyx":85 * return * finally: * acl_free(acl) # <<<<<<<<<<<<<< * */ /*finally:*/ { /*normal exit:*/{ (void)(acl_free(__pyx_v_acl)); goto __pyx_L5; } __pyx_L4_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_3 = 0; __pyx_t_2 = 0; __pyx_t_1 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __Pyx_XDECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_XDECREF(__pyx_t_7); __pyx_t_7 = 0; __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_XDECREF(__pyx_t_8); __pyx_t_8 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_14, &__pyx_t_15, &__pyx_t_16); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_3, &__pyx_t_2, &__pyx_t_1) < 0)) __Pyx_ErrFetch(&__pyx_t_3, &__pyx_t_2, &__pyx_t_1); __Pyx_XGOTREF(__pyx_t_3); __Pyx_XGOTREF(__pyx_t_2); __Pyx_XGOTREF(__pyx_t_1); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_15); __Pyx_XGOTREF(__pyx_t_16); __pyx_t_10 = __pyx_lineno; __pyx_t_12 = __pyx_clineno; __pyx_t_13 = __pyx_filename; { (void)(acl_free(__pyx_v_acl)); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_14); __Pyx_XGIVEREF(__pyx_t_15); __Pyx_XGIVEREF(__pyx_t_16); __Pyx_ExceptionReset(__pyx_t_14, __pyx_t_15, __pyx_t_16); } __Pyx_XGIVEREF(__pyx_t_3); __Pyx_XGIVEREF(__pyx_t_2); __Pyx_XGIVEREF(__pyx_t_1); __Pyx_ErrRestore(__pyx_t_3, __pyx_t_2, __pyx_t_1); __pyx_t_3 = 0; __pyx_t_2 = 0; __pyx_t_1 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_lineno = __pyx_t_10; __pyx_clineno = __pyx_t_12; __pyx_filename = __pyx_t_13; goto __pyx_L1_error; } __pyx_L3_return: { __pyx_t_16 = __pyx_r; __pyx_r = 0; (void)(acl_free(__pyx_v_acl)); __pyx_r = __pyx_t_16; __pyx_t_16 = 0; goto __pyx_L0; } __pyx_L5:; } /* "borg/platform_darwin.pyx":70 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * try: */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_7); __Pyx_XDECREF(__pyx_t_8); __Pyx_XDECREF(__pyx_t_9); __Pyx_AddTraceback("borg.platform_darwin.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_platform_darwin(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_platform_darwin}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "platform_darwin", 0, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_, __pyx_k_, sizeof(__pyx_k_), 0, 0, 1, 0}, {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_n_s_KeyError, __pyx_k_KeyError, sizeof(__pyx_k_KeyError), 0, 0, 1, 1}, {&__pyx_kp_s__2, __pyx_k__2, sizeof(__pyx_k__2), 0, 0, 1, 0}, {&__pyx_kp_s__3, __pyx_k__3, sizeof(__pyx_k__3), 0, 0, 1, 0}, {&__pyx_n_s_acl, __pyx_k_acl, sizeof(__pyx_k_acl), 0, 0, 1, 1}, {&__pyx_n_b_acl_extended, __pyx_k_acl_extended, sizeof(__pyx_k_acl_extended), 0, 0, 0, 1}, {&__pyx_n_s_acl_get, __pyx_k_acl_get, sizeof(__pyx_k_acl_get), 0, 0, 1, 1}, {&__pyx_n_s_acl_set, __pyx_k_acl_set, sizeof(__pyx_k_acl_set), 0, 0, 1, 1}, {&__pyx_n_s_borg_platform_darwin, __pyx_k_borg_platform_darwin, sizeof(__pyx_k_borg_platform_darwin), 0, 0, 1, 1}, {&__pyx_kp_s_borg_platform_darwin_pyx, __pyx_k_borg_platform_darwin_pyx, sizeof(__pyx_k_borg_platform_darwin_pyx), 0, 0, 1, 0}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_entries, __pyx_k_entries, sizeof(__pyx_k_entries), 0, 0, 1, 1}, {&__pyx_n_s_entry, __pyx_k_entry, sizeof(__pyx_k_entry), 0, 0, 1, 1}, {&__pyx_n_s_fields, __pyx_k_fields, sizeof(__pyx_k_fields), 0, 0, 1, 1}, {&__pyx_n_s_fsencode, __pyx_k_fsencode, sizeof(__pyx_k_fsencode), 0, 0, 1, 1}, {&__pyx_n_s_group, __pyx_k_group, sizeof(__pyx_k_group), 0, 0, 1, 1}, {&__pyx_n_s_group2gid, __pyx_k_group2gid, sizeof(__pyx_k_group2gid), 0, 0, 1, 1}, {&__pyx_n_s_helpers, __pyx_k_helpers, sizeof(__pyx_k_helpers), 0, 0, 1, 1}, {&__pyx_n_s_import, __pyx_k_import, sizeof(__pyx_k_import), 0, 0, 1, 1}, {&__pyx_n_s_item, __pyx_k_item, sizeof(__pyx_k_item), 0, 0, 1, 1}, {&__pyx_n_s_join, __pyx_k_join, sizeof(__pyx_k_join), 0, 0, 1, 1}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_n_s_numeric_owner, __pyx_k_numeric_owner, sizeof(__pyx_k_numeric_owner), 0, 0, 1, 1}, {&__pyx_n_s_os, __pyx_k_os, sizeof(__pyx_k_os), 0, 0, 1, 1}, {&__pyx_n_s_path, __pyx_k_path, sizeof(__pyx_k_path), 0, 0, 1, 1}, {&__pyx_n_s_remove_non_numeric_identifier, __pyx_k_remove_non_numeric_identifier, sizeof(__pyx_k_remove_non_numeric_identifier), 0, 0, 1, 1}, {&__pyx_n_s_remove_numeric_id_if_possible, __pyx_k_remove_numeric_id_if_possible, sizeof(__pyx_k_remove_numeric_id_if_possible), 0, 0, 1, 1}, {&__pyx_n_s_safe_decode, __pyx_k_safe_decode, sizeof(__pyx_k_safe_decode), 0, 0, 1, 1}, {&__pyx_n_s_safe_encode, __pyx_k_safe_encode, sizeof(__pyx_k_safe_encode), 0, 0, 1, 1}, {&__pyx_n_s_split, __pyx_k_split, sizeof(__pyx_k_split), 0, 0, 1, 1}, {&__pyx_n_s_st, __pyx_k_st, sizeof(__pyx_k_st), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {&__pyx_n_s_text, __pyx_k_text, sizeof(__pyx_k_text), 0, 0, 1, 1}, {&__pyx_n_s_user, __pyx_k_user, sizeof(__pyx_k_user), 0, 0, 1, 1}, {&__pyx_n_s_user2uid, __pyx_k_user2uid, sizeof(__pyx_k_user2uid), 0, 0, 1, 1}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { __pyx_builtin_KeyError = __Pyx_GetBuiltinName(__pyx_n_s_KeyError); if (!__pyx_builtin_KeyError) __PYX_ERR(0, 78, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "borg/platform_darwin.pyx":19 * * * def _remove_numeric_id_if_possible(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ __pyx_tuple__4 = PyTuple_Pack(4, __pyx_n_s_acl, __pyx_n_s_entries, __pyx_n_s_entry, __pyx_n_s_fields); if (unlikely(!__pyx_tuple__4)) __PYX_ERR(0, 19, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__4); __Pyx_GIVEREF(__pyx_tuple__4); __pyx_codeobj__5 = (PyObject*)__Pyx_PyCode_New(1, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__4, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_darwin_pyx, __pyx_n_s_remove_numeric_id_if_possible, 19, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__5)) __PYX_ERR(0, 19, __pyx_L1_error) /* "borg/platform_darwin.pyx":36 * * * def _remove_non_numeric_identifier(acl): # <<<<<<<<<<<<<< * """Remove user and group names from the acl * """ */ __pyx_tuple__6 = PyTuple_Pack(4, __pyx_n_s_acl, __pyx_n_s_entries, __pyx_n_s_entry, __pyx_n_s_fields); if (unlikely(!__pyx_tuple__6)) __PYX_ERR(0, 36, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__6); __Pyx_GIVEREF(__pyx_tuple__6); __pyx_codeobj__7 = (PyObject*)__Pyx_PyCode_New(1, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__6, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_darwin_pyx, __pyx_n_s_remove_non_numeric_identifier, 36, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__7)) __PYX_ERR(0, 36, __pyx_L1_error) /* "borg/platform_darwin.pyx":51 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * cdef char *text = NULL */ __pyx_tuple__8 = PyTuple_Pack(6, __pyx_n_s_path, __pyx_n_s_item, __pyx_n_s_st, __pyx_n_s_numeric_owner, __pyx_n_s_acl, __pyx_n_s_text); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(0, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__8); __Pyx_GIVEREF(__pyx_tuple__8); __pyx_codeobj__9 = (PyObject*)__Pyx_PyCode_New(4, 0, 6, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__8, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_darwin_pyx, __pyx_n_s_acl_get, 51, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__9)) __PYX_ERR(0, 51, __pyx_L1_error) /* "borg/platform_darwin.pyx":70 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * try: */ __pyx_tuple__10 = PyTuple_Pack(4, __pyx_n_s_path, __pyx_n_s_item, __pyx_n_s_numeric_owner, __pyx_n_s_acl); if (unlikely(!__pyx_tuple__10)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__10); __Pyx_GIVEREF(__pyx_tuple__10); __pyx_codeobj__11 = (PyObject*)__Pyx_PyCode_New(3, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__10, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_darwin_pyx, __pyx_n_s_acl_set, 70, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__11)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(0, 1, __pyx_L1_error); return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC initplatform_darwin(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC initplatform_darwin(void) #else __Pyx_PyMODINIT_FUNC PyInit_platform_darwin(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_platform_darwin(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_platform_darwin(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'platform_darwin' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_platform_darwin(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("platform_darwin", __pyx_methods, 0, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(0, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(0, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(0, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(0, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__platform_darwin) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name, __pyx_n_s_main) < 0) __PYX_ERR(0, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(0, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.platform_darwin")) { if (unlikely(PyDict_SetItemString(modules, "borg.platform_darwin", __pyx_m) < 0)) __PYX_ERR(0, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); (void)__Pyx_modinit_type_init_code(); (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /* "borg/platform_darwin.pyx":1 * import os # <<<<<<<<<<<<<< * from .helpers import user2uid, group2gid, safe_decode, safe_encode * */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_os, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_os, __pyx_t_1) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_darwin.pyx":2 * import os * from .helpers import user2uid, group2gid, safe_decode, safe_encode # <<<<<<<<<<<<<< * * API_VERSION = '1.0_01' */ __pyx_t_1 = PyList_New(4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_n_s_user2uid); __Pyx_GIVEREF(__pyx_n_s_user2uid); PyList_SET_ITEM(__pyx_t_1, 0, __pyx_n_s_user2uid); __Pyx_INCREF(__pyx_n_s_group2gid); __Pyx_GIVEREF(__pyx_n_s_group2gid); PyList_SET_ITEM(__pyx_t_1, 1, __pyx_n_s_group2gid); __Pyx_INCREF(__pyx_n_s_safe_decode); __Pyx_GIVEREF(__pyx_n_s_safe_decode); PyList_SET_ITEM(__pyx_t_1, 2, __pyx_n_s_safe_decode); __Pyx_INCREF(__pyx_n_s_safe_encode); __Pyx_GIVEREF(__pyx_n_s_safe_encode); PyList_SET_ITEM(__pyx_t_1, 3, __pyx_n_s_safe_encode); __pyx_t_2 = __Pyx_Import(__pyx_n_s_helpers, __pyx_t_1, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_user2uid); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_user2uid, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_group2gid); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_group2gid, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_safe_decode, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_safe_encode, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_darwin.pyx":4 * from .helpers import user2uid, group2gid, safe_decode, safe_encode * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * cdef extern from "sys/acl.h": */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(0, 4, __pyx_L1_error) /* "borg/platform_darwin.pyx":19 * * * def _remove_numeric_id_if_possible(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_15platform_darwin_1_remove_numeric_id_if_possible, NULL, __pyx_n_s_borg_platform_darwin); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 19, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_remove_numeric_id_if_possible, __pyx_t_2) < 0) __PYX_ERR(0, 19, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_darwin.pyx":36 * * * def _remove_non_numeric_identifier(acl): # <<<<<<<<<<<<<< * """Remove user and group names from the acl * """ */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_15platform_darwin_3_remove_non_numeric_identifier, NULL, __pyx_n_s_borg_platform_darwin); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 36, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_remove_non_numeric_identifier, __pyx_t_2) < 0) __PYX_ERR(0, 36, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_darwin.pyx":51 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * cdef char *text = NULL */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_15platform_darwin_5acl_get, NULL, __pyx_n_s_borg_platform_darwin); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_get, __pyx_t_2) < 0) __PYX_ERR(0, 51, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_darwin.pyx":70 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl = NULL * try: */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_15platform_darwin_7acl_set, NULL, __pyx_n_s_borg_platform_darwin); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_set, __pyx_t_2) < 0) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_darwin.pyx":1 * import os # <<<<<<<<<<<<<< * from .helpers import user2uid, group2gid, safe_decode, safe_encode * */ __pyx_t_2 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_2) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.platform_darwin", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.platform_darwin"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* GetModuleGlobalName */ #if CYTHON_USE_DICT_VERSIONS static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value) #else static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name) #endif { PyObject *result; #if !CYTHON_AVOID_BORROWED_REFS #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 result = _PyDict_GetItem_KnownHash(__pyx_d, name, ((PyASCIIObject *) name)->hash); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } else if (unlikely(PyErr_Occurred())) { return NULL; } #else result = PyDict_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } #endif #else result = PyObject_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } PyErr_Clear(); #endif return __Pyx_GetBuiltinName(name); } /* PyCFunctionFastCall */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, Py_ssize_t nargs) { PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); int flags = PyCFunction_GET_FLAGS(func); assert(PyCFunction_Check(func)); assert(METH_FASTCALL == (flags & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))); assert(nargs >= 0); assert(nargs == 0 || args != NULL); /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); if ((PY_VERSION_HEX < 0x030700A0) || unlikely(flags & METH_KEYWORDS)) { return (*((__Pyx_PyCFunctionFastWithKeywords)(void*)meth)) (self, args, nargs, NULL); } else { return (*((__Pyx_PyCFunctionFast)(void*)meth)) (self, args, nargs); } } #endif /* PyFunctionFastCall */ #if CYTHON_FAST_PYCALL static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject **fastlocals; Py_ssize_t i; PyObject *result; assert(globals != NULL); /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) { return NULL; } fastlocals = __Pyx_PyFrame_GetLocalsplus(f); for (i = 0; i < na; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } result = PyEval_EvalFrameEx(f,0); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; return result; } #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *closure; #if PY_MAJOR_VERSION >= 3 PyObject *kwdefs; #endif PyObject *kwtuple, **k; PyObject **d; Py_ssize_t nd; Py_ssize_t nk; PyObject *result; assert(kwargs == NULL || PyDict_Check(kwargs)); nk = kwargs ? PyDict_Size(kwargs) : 0; if (Py_EnterRecursiveCall((char*)" while calling a Python object")) { return NULL; } if ( #if PY_MAJOR_VERSION >= 3 co->co_kwonlyargcount == 0 && #endif likely(kwargs == NULL || nk == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { result = __Pyx_PyFunction_FastCallNoKw(co, args, nargs, globals); goto done; } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); result =__Pyx_PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), globals); goto done; } } if (kwargs != NULL) { Py_ssize_t pos, i; kwtuple = PyTuple_New(2 * nk); if (kwtuple == NULL) { result = NULL; goto done; } k = &PyTuple_GET_ITEM(kwtuple, 0); pos = i = 0; while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { Py_INCREF(k[i]); Py_INCREF(k[i+1]); i += 2; } nk = i / 2; } else { kwtuple = NULL; k = NULL; } closure = PyFunction_GET_CLOSURE(func); #if PY_MAJOR_VERSION >= 3 kwdefs = PyFunction_GET_KW_DEFAULTS(func); #endif if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } else { d = NULL; nd = 0; } #if PY_MAJOR_VERSION >= 3 result = PyEval_EvalCodeEx((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, kwdefs, closure); #else result = PyEval_EvalCodeEx(co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, closure); #endif Py_XDECREF(kwtuple); done: Py_LeaveRecursiveCall(); return result; } #endif #endif /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCall2Args */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2) { PyObject *args, *result = NULL; #if CYTHON_FAST_PYCALL if (PyFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyFunction_FastCall(function, args, 2); } #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyCFunction_FastCall(function, args, 2); } #endif args = PyTuple_New(2); if (unlikely(!args)) goto done; Py_INCREF(arg1); PyTuple_SET_ITEM(args, 0, arg1); Py_INCREF(arg2); PyTuple_SET_ITEM(args, 1, arg2); Py_INCREF(function); result = __Pyx_PyObject_Call(function, args, NULL); Py_DECREF(args); Py_DECREF(function); done: return result; } /* PyObjectCallMethO */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg) { PyObject *self, *result; PyCFunction cfunc; cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = cfunc(self, arg); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCallOneArg */ #if CYTHON_COMPILING_IN_CPYTHON static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_New(1); if (unlikely(!args)) return NULL; Py_INCREF(arg); PyTuple_SET_ITEM(args, 0, arg); result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, &arg, 1); } #endif if (likely(PyCFunction_Check(func))) { if (likely(PyCFunction_GET_FLAGS(func) & METH_O)) { return __Pyx_PyObject_CallMethO(func, arg); #if CYTHON_FAST_PYCCALL } else if (PyCFunction_GET_FLAGS(func) & METH_FASTCALL) { return __Pyx_PyCFunction_FastCall(func, &arg, 1); #endif } } return __Pyx__PyObject_CallOneArg(func, arg); } #else static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_Pack(1, arg); if (unlikely(!args)) return NULL; result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } #endif /* GetItemInt */ static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) { PyObject *r; if (!j) return NULL; r = PyObject_GetItem(o, j); Py_DECREF(j); return r; } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyList_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyList_GET_SIZE(o)))) { PyObject *r = PyList_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyTuple_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyList_GET_SIZE(o); if ((!boundscheck) || (likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o))))) { PyObject *r = PyList_GET_ITEM(o, n); Py_INCREF(r); return r; } } else if (PyTuple_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyTuple_GET_SIZE(o); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, n); Py_INCREF(r); return r; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return NULL; PyErr_Clear(); } } return m->sq_item(o, i); } } #else if (is_list || PySequence_Check(o)) { return PySequence_GetItem(o, i); } #endif return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); } /* BytesEquals */ static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2, int equals) { #if CYTHON_COMPILING_IN_PYPY return PyObject_RichCompareBool(s1, s2, equals); #else if (s1 == s2) { return (equals == Py_EQ); } else if (PyBytes_CheckExact(s1) & PyBytes_CheckExact(s2)) { const char *ps1, *ps2; Py_ssize_t length = PyBytes_GET_SIZE(s1); if (length != PyBytes_GET_SIZE(s2)) return (equals == Py_NE); ps1 = PyBytes_AS_STRING(s1); ps2 = PyBytes_AS_STRING(s2); if (ps1[0] != ps2[0]) { return (equals == Py_NE); } else if (length == 1) { return (equals == Py_EQ); } else { int result; #if CYTHON_USE_UNICODE_INTERNALS Py_hash_t hash1, hash2; hash1 = ((PyBytesObject*)s1)->ob_shash; hash2 = ((PyBytesObject*)s2)->ob_shash; if (hash1 != hash2 && hash1 != -1 && hash2 != -1) { return (equals == Py_NE); } #endif result = memcmp(ps1, ps2, (size_t)length); return (equals == Py_EQ) ? (result == 0) : (result != 0); } } else if ((s1 == Py_None) & PyBytes_CheckExact(s2)) { return (equals == Py_NE); } else if ((s2 == Py_None) & PyBytes_CheckExact(s1)) { return (equals == Py_NE); } else { int result; PyObject* py_result = PyObject_RichCompare(s1, s2, equals); if (!py_result) return -1; result = __Pyx_PyObject_IsTrue(py_result); Py_DECREF(py_result); return result; } #endif } /* UnicodeEquals */ static CYTHON_INLINE int __Pyx_PyUnicode_Equals(PyObject* s1, PyObject* s2, int equals) { #if CYTHON_COMPILING_IN_PYPY return PyObject_RichCompareBool(s1, s2, equals); #else #if PY_MAJOR_VERSION < 3 PyObject* owned_ref = NULL; #endif int s1_is_unicode, s2_is_unicode; if (s1 == s2) { goto return_eq; } s1_is_unicode = PyUnicode_CheckExact(s1); s2_is_unicode = PyUnicode_CheckExact(s2); #if PY_MAJOR_VERSION < 3 if ((s1_is_unicode & (!s2_is_unicode)) && PyString_CheckExact(s2)) { owned_ref = PyUnicode_FromObject(s2); if (unlikely(!owned_ref)) return -1; s2 = owned_ref; s2_is_unicode = 1; } else if ((s2_is_unicode & (!s1_is_unicode)) && PyString_CheckExact(s1)) { owned_ref = PyUnicode_FromObject(s1); if (unlikely(!owned_ref)) return -1; s1 = owned_ref; s1_is_unicode = 1; } else if (((!s2_is_unicode) & (!s1_is_unicode))) { return __Pyx_PyBytes_Equals(s1, s2, equals); } #endif if (s1_is_unicode & s2_is_unicode) { Py_ssize_t length; int kind; void *data1, *data2; if (unlikely(__Pyx_PyUnicode_READY(s1) < 0) || unlikely(__Pyx_PyUnicode_READY(s2) < 0)) return -1; length = __Pyx_PyUnicode_GET_LENGTH(s1); if (length != __Pyx_PyUnicode_GET_LENGTH(s2)) { goto return_ne; } #if CYTHON_USE_UNICODE_INTERNALS { Py_hash_t hash1, hash2; #if CYTHON_PEP393_ENABLED hash1 = ((PyASCIIObject*)s1)->hash; hash2 = ((PyASCIIObject*)s2)->hash; #else hash1 = ((PyUnicodeObject*)s1)->hash; hash2 = ((PyUnicodeObject*)s2)->hash; #endif if (hash1 != hash2 && hash1 != -1 && hash2 != -1) { goto return_ne; } } #endif kind = __Pyx_PyUnicode_KIND(s1); if (kind != __Pyx_PyUnicode_KIND(s2)) { goto return_ne; } data1 = __Pyx_PyUnicode_DATA(s1); data2 = __Pyx_PyUnicode_DATA(s2); if (__Pyx_PyUnicode_READ(kind, data1, 0) != __Pyx_PyUnicode_READ(kind, data2, 0)) { goto return_ne; } else if (length == 1) { goto return_eq; } else { int result = memcmp(data1, data2, (size_t)(length * kind)); #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif return (equals == Py_EQ) ? (result == 0) : (result != 0); } } else if ((s1 == Py_None) & s2_is_unicode) { goto return_ne; } else if ((s2 == Py_None) & s1_is_unicode) { goto return_ne; } else { int result; PyObject* py_result = PyObject_RichCompare(s1, s2, equals); #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif if (!py_result) return -1; result = __Pyx_PyObject_IsTrue(py_result); Py_DECREF(py_result); return result; } return_eq: #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif return (equals == Py_EQ); return_ne: #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif return (equals == Py_NE); #endif } /* SetItemInt */ static int __Pyx_SetItemInt_Generic(PyObject *o, PyObject *j, PyObject *v) { int r; if (!j) return -1; r = PyObject_SetItem(o, j, v); Py_DECREF(j); return r; } static CYTHON_INLINE int __Pyx_SetItemInt_Fast(PyObject *o, Py_ssize_t i, PyObject *v, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = (!wraparound) ? i : ((likely(i >= 0)) ? i : i + PyList_GET_SIZE(o)); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o)))) { PyObject* old = PyList_GET_ITEM(o, n); Py_INCREF(v); PyList_SET_ITEM(o, n, v); Py_DECREF(old); return 1; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_ass_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return -1; PyErr_Clear(); } } return m->sq_ass_item(o, i, v); } } #else #if CYTHON_COMPILING_IN_PYPY if (is_list || (PySequence_Check(o) && !PyDict_Check(o))) #else if (is_list || PySequence_Check(o)) #endif { return PySequence_SetItem(o, i, v); } #endif return __Pyx_SetItemInt_Generic(o, PyInt_FromSsize_t(i), v); } /* StringJoin */ #if !CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyBytes_Join(PyObject* sep, PyObject* values) { return PyObject_CallMethodObjArgs(sep, __pyx_n_s_join, values, NULL); } #endif /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* GetException */ #if CYTHON_FAST_THREAD_STATE static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb) #endif { PyObject *local_type, *local_value, *local_tb; #if CYTHON_FAST_THREAD_STATE PyObject *tmp_type, *tmp_value, *tmp_tb; local_type = tstate->curexc_type; local_value = tstate->curexc_value; local_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; #else PyErr_Fetch(&local_type, &local_value, &local_tb); #endif PyErr_NormalizeException(&local_type, &local_value, &local_tb); #if CYTHON_FAST_THREAD_STATE if (unlikely(tstate->curexc_type)) #else if (unlikely(PyErr_Occurred())) #endif goto bad; #if PY_MAJOR_VERSION >= 3 if (local_tb) { if (unlikely(PyException_SetTraceback(local_value, local_tb) < 0)) goto bad; } #endif Py_XINCREF(local_tb); Py_XINCREF(local_type); Py_XINCREF(local_value); *type = local_type; *value = local_value; *tb = local_tb; #if CYTHON_FAST_THREAD_STATE #if CYTHON_USE_EXC_INFO_STACK { _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = local_type; exc_info->exc_value = local_value; exc_info->exc_traceback = local_tb; } #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = local_type; tstate->exc_value = local_value; tstate->exc_traceback = local_tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); #else PyErr_SetExcInfo(local_type, local_value, local_tb); #endif return 0; bad: *type = 0; *value = 0; *tb = 0; Py_XDECREF(local_type); Py_XDECREF(local_value); Py_XDECREF(local_tb); return -1; } /* SwapException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = *type; exc_info->exc_value = *value; exc_info->exc_traceback = *tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = *type; tstate->exc_value = *value; tstate->exc_traceback = *tb; #endif *type = tmp_type; *value = tmp_value; *tb = tmp_tb; } #else static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_GetExcInfo(&tmp_type, &tmp_value, &tmp_tb); PyErr_SetExcInfo(*type, *value, *tb); *type = tmp_type; *value = tmp_value; *tb = tmp_tb; } #endif /* GetTopmostException */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate) { _PyErr_StackItem *exc_info = tstate->exc_info; while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && exc_info->previous_item != NULL) { exc_info = exc_info->previous_item; } return exc_info; } #endif /* SaveResetException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = __Pyx_PyErr_GetTopmostException(tstate); *type = exc_info->exc_type; *value = exc_info->exc_value; *tb = exc_info->exc_traceback; #else *type = tstate->exc_type; *value = tstate->exc_value; *tb = tstate->exc_traceback; #endif Py_XINCREF(*type); Py_XINCREF(*value); Py_XINCREF(*tb); } static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = type; exc_info->exc_value = value; exc_info->exc_traceback = tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = type; tstate->exc_value = value; tstate->exc_traceback = tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } #endif /* ObjectGetItem */ #if CYTHON_USE_TYPE_SLOTS static PyObject *__Pyx_PyObject_GetIndex(PyObject *obj, PyObject* index) { PyObject *runerr; Py_ssize_t key_value; PySequenceMethods *m = Py_TYPE(obj)->tp_as_sequence; if (unlikely(!(m && m->sq_item))) { PyErr_Format(PyExc_TypeError, "'%.200s' object is not subscriptable", Py_TYPE(obj)->tp_name); return NULL; } key_value = __Pyx_PyIndex_AsSsize_t(index); if (likely(key_value != -1 || !(runerr = PyErr_Occurred()))) { return __Pyx_GetItemInt_Fast(obj, key_value, 0, 1, 1); } if (PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError)) { PyErr_Clear(); PyErr_Format(PyExc_IndexError, "cannot fit '%.200s' into an index-sized integer", Py_TYPE(index)->tp_name); } return NULL; } static PyObject *__Pyx_PyObject_GetItem(PyObject *obj, PyObject* key) { PyMappingMethods *m = Py_TYPE(obj)->tp_as_mapping; if (likely(m && m->mp_subscript)) { return m->mp_subscript(obj, key); } return __Pyx_PyObject_GetIndex(obj, key); } #endif /* PyErrExceptionMatches */ #if CYTHON_FAST_THREAD_STATE static int __Pyx_PyErr_ExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; icurexc_type; if (exc_type == err) return 1; if (unlikely(!exc_type)) return 0; if (unlikely(PyTuple_Check(err))) return __Pyx_PyErr_ExceptionMatchesTuple(exc_type, err); return __Pyx_PyErr_GivenExceptionMatches(exc_type, err); } #endif /* Import */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level) { PyObject *empty_list = 0; PyObject *module = 0; PyObject *global_dict = 0; PyObject *empty_dict = 0; PyObject *list; #if PY_MAJOR_VERSION < 3 PyObject *py_import; py_import = __Pyx_PyObject_GetAttrStr(__pyx_b, __pyx_n_s_import); if (!py_import) goto bad; #endif if (from_list) list = from_list; else { empty_list = PyList_New(0); if (!empty_list) goto bad; list = empty_list; } global_dict = PyModule_GetDict(__pyx_m); if (!global_dict) goto bad; empty_dict = PyDict_New(); if (!empty_dict) goto bad; { #if PY_MAJOR_VERSION >= 3 if (level == -1) { if (strchr(__Pyx_MODULE_NAME, '.')) { module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, 1); if (!module) { if (!PyErr_ExceptionMatches(PyExc_ImportError)) goto bad; PyErr_Clear(); } } level = 0; } #endif if (!module) { #if PY_MAJOR_VERSION < 3 PyObject *py_level = PyInt_FromLong(level); if (!py_level) goto bad; module = PyObject_CallFunctionObjArgs(py_import, name, global_dict, empty_dict, list, py_level, (PyObject *)NULL); Py_DECREF(py_level); #else module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, level); #endif } } bad: #if PY_MAJOR_VERSION < 3 Py_XDECREF(py_import); #endif Py_XDECREF(empty_list); Py_XDECREF(empty_dict); return module; } /* ImportFrom */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name) { PyObject* value = __Pyx_PyObject_GetAttrStr(module, name); if (unlikely(!value) && PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Format(PyExc_ImportError, #if PY_MAJOR_VERSION < 3 "cannot import name %.230s", PyString_AS_STRING(name)); #else "cannot import name %S", name); #endif } return value; } /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; ip) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/platform_darwin.pyx0000644000175000017500000000477613431513133020567 0ustar useruser00000000000000import os from .helpers import user2uid, group2gid, safe_decode, safe_encode API_VERSION = '1.0_01' cdef extern from "sys/acl.h": ctypedef struct _acl_t: pass ctypedef _acl_t *acl_t int acl_free(void *obj) acl_t acl_get_link_np(const char *path, int type) int acl_set_link_np(const char *path, int type, acl_t acl) acl_t acl_from_text(const char *buf) char *acl_to_text(acl_t acl, ssize_t *len_p) int ACL_TYPE_EXTENDED def _remove_numeric_id_if_possible(acl): """Replace the user/group field with the local uid/gid if possible """ entries = [] for entry in safe_decode(acl).split('\n'): if entry: fields = entry.split(':') if fields[0] == 'user': if user2uid(fields[2]) is not None: fields[1] = fields[3] = '' elif fields[0] == 'group': if group2gid(fields[2]) is not None: fields[1] = fields[3] = '' entries.append(':'.join(fields)) return safe_encode('\n'.join(entries)) def _remove_non_numeric_identifier(acl): """Remove user and group names from the acl """ entries = [] for entry in safe_decode(acl).split('\n'): if entry: fields = entry.split(':') if fields[0] in ('user', 'group'): fields[2] = '' entries.append(':'.join(fields)) else: entries.append(entry) return safe_encode('\n'.join(entries)) def acl_get(path, item, st, numeric_owner=False): cdef acl_t acl = NULL cdef char *text = NULL try: acl = acl_get_link_np(os.fsencode(path), ACL_TYPE_EXTENDED) if acl == NULL: return text = acl_to_text(acl, NULL) if text == NULL: return if numeric_owner: item[b'acl_extended'] = _remove_non_numeric_identifier(text) else: item[b'acl_extended'] = text finally: acl_free(text) acl_free(acl) def acl_set(path, item, numeric_owner=False): cdef acl_t acl = NULL try: try: if numeric_owner: acl = acl_from_text(item[b'acl_extended']) else: acl = acl_from_text(_remove_numeric_id_if_possible(item[b'acl_extended'])) except KeyError: return if acl == NULL: return if acl_set_link_np(os.fsencode(path), ACL_TYPE_EXTENDED, acl): return finally: acl_free(acl) borgbackup-1.0.13/borg/platform_freebsd.c0000644000175000017500000056536713431514213020327 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__platform_freebsd #define __PYX_HAVE_API__borg__platform_freebsd /* Early includes */ #include "errno.h" #include "sys/types.h" #include "sys/acl.h" #include "unistd.h" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "borg/platform_freebsd.pyx", }; /*--- Type declarations ---*/ struct __pyx_opt_args_4borg_16platform_freebsd__set_acl; /* "borg/platform_freebsd.pyx":63 * * * cdef _set_acl(p, type, item, attribute, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl * text = item.get(attribute) */ struct __pyx_opt_args_4borg_16platform_freebsd__set_acl { int __pyx_n; PyObject *numeric_owner; }; /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* GetModuleGlobalName.proto */ #if CYTHON_USE_DICT_VERSIONS #define __Pyx_GetModuleGlobalName(var, name) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } #define __Pyx_GetModuleGlobalNameUncached(var, name) {\ PY_UINT64_T __pyx_dict_version;\ PyObject *__pyx_dict_cached_value;\ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value); #else #define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name) #define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name) static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name); #endif /* PyCFunctionFastCall.proto */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs); #else #define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL) #endif /* PyFunctionFastCall.proto */ #if CYTHON_FAST_PYCALL #define __Pyx_PyFunction_FastCall(func, args, nargs)\ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL) #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #else #define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs) #endif #define __Pyx_BUILD_ASSERT_EXPR(cond)\ (sizeof(char [1 - 2*!(cond)]) - 1) #ifndef Py_MEMBER_SIZE #define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) #endif static size_t __pyx_pyframe_localsplus_offset = 0; #include "frameobject.h" #define __Pxy_PyFrame_Initialize_Offsets()\ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\ (void)(__pyx_pyframe_localsplus_offset = PyFrame_Type.tp_basicsize - Py_MEMBER_SIZE(PyFrameObject, f_localsplus))) #define __Pyx_PyFrame_GetLocalsplus(frame)\ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset)) #endif /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* PyObjectCall2Args.proto */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2); /* PyObjectCallMethO.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg); #endif /* PyObjectCallOneArg.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg); /* ListAppend.proto */ #if CYTHON_USE_PYLIST_INTERNALS && CYTHON_ASSUME_SAFE_MACROS static CYTHON_INLINE int __Pyx_PyList_Append(PyObject* list, PyObject* x) { PyListObject* L = (PyListObject*) list; Py_ssize_t len = Py_SIZE(list); if (likely(L->allocated > len) & likely(len > (L->allocated >> 1))) { Py_INCREF(x); PyList_SET_ITEM(list, len, x); Py_SIZE(list) = len+1; return 0; } return PyList_Append(list, x); } #else #define __Pyx_PyList_Append(L,x) PyList_Append(L,x) #endif /* GetItemInt.proto */ #define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\ __Pyx_GetItemInt_Generic(o, to_py_func(i)))) #define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); #define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j); static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, int wraparound, int boundscheck); /* SliceObject.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice( PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** py_start, PyObject** py_stop, PyObject** py_slice, int has_cstart, int has_cstop, int wraparound); /* StringJoin.proto */ #if PY_MAJOR_VERSION < 3 #define __Pyx_PyString_Join __Pyx_PyBytes_Join #define __Pyx_PyBaseString_Join(s, v) (PyUnicode_CheckExact(s) ? PyUnicode_Join(s, v) : __Pyx_PyBytes_Join(s, v)) #else #define __Pyx_PyString_Join PyUnicode_Join #define __Pyx_PyBaseString_Join PyUnicode_Join #endif #if CYTHON_COMPILING_IN_CPYTHON #if PY_MAJOR_VERSION < 3 #define __Pyx_PyBytes_Join _PyString_Join #else #define __Pyx_PyBytes_Join _PyBytes_Join #endif #else static CYTHON_INLINE PyObject* __Pyx_PyBytes_Join(PyObject* sep, PyObject* values); #endif /* Import.proto */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level); /* ImportFrom.proto */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name); /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); /* Module declarations from 'borg.platform_freebsd' */ static PyObject *__pyx_f_4borg_16platform_freebsd__get_acl(PyObject *, PyObject *, PyObject *, PyObject *, int); /*proto*/ static PyObject *__pyx_f_4borg_16platform_freebsd__set_acl(PyObject *, PyObject *, PyObject *, PyObject *, struct __pyx_opt_args_4borg_16platform_freebsd__set_acl *__pyx_optional_args); /*proto*/ static PyObject *__pyx_f_4borg_16platform_freebsd__nfs4_use_stored_uid_gid(PyObject *); /*proto*/ #define __Pyx_MODULE_NAME "borg.platform_freebsd" extern int __pyx_module_is_main_borg__platform_freebsd; int __pyx_module_is_main_borg__platform_freebsd = 0; /* Implementation of 'borg.platform_freebsd' */ static const char __pyx_k_[] = "\n"; static const char __pyx_k_p[] = "p"; static const char __pyx_k__2[] = ":"; static const char __pyx_k_os[] = "os"; static const char __pyx_k_st[] = "st"; static const char __pyx_k_get[] = "get"; static const char __pyx_k_ret[] = "ret"; static const char __pyx_k_item[] = "item"; static const char __pyx_k_join[] = "join"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_name[] = "__name__"; static const char __pyx_k_path[] = "path"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_user[] = "user:"; static const char __pyx_k_flags[] = "flags"; static const char __pyx_k_group[] = "group:"; static const char __pyx_k_split[] = "split"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_import[] = "__import__"; static const char __pyx_k_acl_get[] = "acl_get"; static const char __pyx_k_acl_set[] = "acl_set"; static const char __pyx_k_helpers[] = "helpers"; static const char __pyx_k_acl_nfs4[] = "acl_nfs4"; static const char __pyx_k_fsencode[] = "fsencode"; static const char __pyx_k_acl_access[] = "acl_access"; static const char __pyx_k_startswith[] = "startswith"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_acl_default[] = "acl_default"; static const char __pyx_k_safe_decode[] = "safe_decode"; static const char __pyx_k_safe_encode[] = "safe_encode"; static const char __pyx_k_numeric_owner[] = "numeric_owner"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_borg_platform_freebsd[] = "borg.platform_freebsd"; static const char __pyx_k_borg_platform_freebsd_pyx[] = "borg/platform_freebsd.pyx"; static const char __pyx_k_posix_acl_use_stored_uid_gid[] = "posix_acl_use_stored_uid_gid"; static PyObject *__pyx_kp_s_; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_kp_s__2; static PyObject *__pyx_n_b_acl_access; static PyObject *__pyx_n_b_acl_default; static PyObject *__pyx_n_s_acl_get; static PyObject *__pyx_n_b_acl_nfs4; static PyObject *__pyx_n_s_acl_set; static PyObject *__pyx_n_s_borg_platform_freebsd; static PyObject *__pyx_kp_s_borg_platform_freebsd_pyx; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_flags; static PyObject *__pyx_n_s_fsencode; static PyObject *__pyx_n_s_get; static PyObject *__pyx_kp_s_group; static PyObject *__pyx_n_s_helpers; static PyObject *__pyx_n_s_import; static PyObject *__pyx_n_s_item; static PyObject *__pyx_n_s_join; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_name; static PyObject *__pyx_n_s_numeric_owner; static PyObject *__pyx_n_s_os; static PyObject *__pyx_n_s_p; static PyObject *__pyx_n_s_path; static PyObject *__pyx_n_s_posix_acl_use_stored_uid_gid; static PyObject *__pyx_n_s_ret; static PyObject *__pyx_n_s_safe_decode; static PyObject *__pyx_n_s_safe_encode; static PyObject *__pyx_n_s_split; static PyObject *__pyx_n_s_st; static PyObject *__pyx_n_s_startswith; static PyObject *__pyx_n_s_test; static PyObject *__pyx_kp_s_user; static PyObject *__pyx_pf_4borg_16platform_freebsd_acl_get(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, CYTHON_UNUSED PyObject *__pyx_v_st, PyObject *__pyx_v_numeric_owner); /* proto */ static PyObject *__pyx_pf_4borg_16platform_freebsd_2acl_set(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_numeric_owner); /* proto */ static PyObject *__pyx_int_2; static PyObject *__pyx_int_neg_1; static PyObject *__pyx_slice__3; static PyObject *__pyx_tuple__4; static PyObject *__pyx_tuple__6; static PyObject *__pyx_codeobj__5; static PyObject *__pyx_codeobj__7; /* Late includes */ /* "borg/platform_freebsd.pyx":33 * * * cdef _get_acl(p, type, item, attribute, int flags): # <<<<<<<<<<<<<< * cdef acl_t acl * cdef char *text */ static PyObject *__pyx_f_4borg_16platform_freebsd__get_acl(PyObject *__pyx_v_p, PyObject *__pyx_v_type, PyObject *__pyx_v_item, PyObject *__pyx_v_attribute, int __pyx_v_flags) { acl_t __pyx_v_acl; char *__pyx_v_text; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations char const *__pyx_t_1; int __pyx_t_2; int __pyx_t_3; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("_get_acl", 0); /* "borg/platform_freebsd.pyx":36 * cdef acl_t acl * cdef char *text * acl = acl_get_link_np(p, type) # <<<<<<<<<<<<<< * if acl: * text = acl_to_text_np(acl, NULL, flags) */ __pyx_t_1 = __Pyx_PyObject_AsString(__pyx_v_p); if (unlikely((!__pyx_t_1) && PyErr_Occurred())) __PYX_ERR(0, 36, __pyx_L1_error) __pyx_t_2 = __Pyx_PyInt_As_int(__pyx_v_type); if (unlikely((__pyx_t_2 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 36, __pyx_L1_error) __pyx_v_acl = acl_get_link_np(__pyx_t_1, __pyx_t_2); /* "borg/platform_freebsd.pyx":37 * cdef char *text * acl = acl_get_link_np(p, type) * if acl: # <<<<<<<<<<<<<< * text = acl_to_text_np(acl, NULL, flags) * if text: */ __pyx_t_3 = (__pyx_v_acl != 0); if (__pyx_t_3) { /* "borg/platform_freebsd.pyx":38 * acl = acl_get_link_np(p, type) * if acl: * text = acl_to_text_np(acl, NULL, flags) # <<<<<<<<<<<<<< * if text: * item[attribute] = text */ __pyx_v_text = acl_to_text_np(__pyx_v_acl, NULL, __pyx_v_flags); /* "borg/platform_freebsd.pyx":39 * if acl: * text = acl_to_text_np(acl, NULL, flags) * if text: # <<<<<<<<<<<<<< * item[attribute] = text * acl_free(text) */ __pyx_t_3 = (__pyx_v_text != 0); if (__pyx_t_3) { /* "borg/platform_freebsd.pyx":40 * text = acl_to_text_np(acl, NULL, flags) * if text: * item[attribute] = text # <<<<<<<<<<<<<< * acl_free(text) * acl_free(acl) */ __pyx_t_4 = __Pyx_PyBytes_FromString(__pyx_v_text); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); if (unlikely(PyObject_SetItem(__pyx_v_item, __pyx_v_attribute, __pyx_t_4) < 0)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/platform_freebsd.pyx":41 * if text: * item[attribute] = text * acl_free(text) # <<<<<<<<<<<<<< * acl_free(acl) * */ (void)(acl_free(__pyx_v_text)); /* "borg/platform_freebsd.pyx":39 * if acl: * text = acl_to_text_np(acl, NULL, flags) * if text: # <<<<<<<<<<<<<< * item[attribute] = text * acl_free(text) */ } /* "borg/platform_freebsd.pyx":42 * item[attribute] = text * acl_free(text) * acl_free(acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_acl)); /* "borg/platform_freebsd.pyx":37 * cdef char *text * acl = acl_get_link_np(p, type) * if acl: # <<<<<<<<<<<<<< * text = acl_to_text_np(acl, NULL, flags) * if text: */ } /* "borg/platform_freebsd.pyx":33 * * * cdef _get_acl(p, type, item, attribute, int flags): # <<<<<<<<<<<<<< * cdef acl_t acl * cdef char *text */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.platform_freebsd._get_acl", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_freebsd.pyx":45 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_16platform_freebsd_1acl_get(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_16platform_freebsd_acl_get[] = "Saves ACL Entries\n\n If `numeric_owner` is True the user/group field is not preserved only uid/gid\n "; static PyMethodDef __pyx_mdef_4borg_16platform_freebsd_1acl_get = {"acl_get", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_16platform_freebsd_1acl_get, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_16platform_freebsd_acl_get}; static PyObject *__pyx_pw_4borg_16platform_freebsd_1acl_get(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_path = 0; PyObject *__pyx_v_item = 0; CYTHON_UNUSED PyObject *__pyx_v_st = 0; PyObject *__pyx_v_numeric_owner = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_get (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_path,&__pyx_n_s_item,&__pyx_n_s_st,&__pyx_n_s_numeric_owner,0}; PyObject* values[4] = {0,0,0,0}; values[3] = ((PyObject *)Py_False); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_item)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, 1); __PYX_ERR(0, 45, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_st)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, 2); __PYX_ERR(0, 45, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_numeric_owner); if (value) { values[3] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "acl_get") < 0)) __PYX_ERR(0, 45, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_path = values[0]; __pyx_v_item = values[1]; __pyx_v_st = values[2]; __pyx_v_numeric_owner = values[3]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 45, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.platform_freebsd.acl_get", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_16platform_freebsd_acl_get(__pyx_self, __pyx_v_path, __pyx_v_item, __pyx_v_st, __pyx_v_numeric_owner); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_16platform_freebsd_acl_get(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, CYTHON_UNUSED PyObject *__pyx_v_st, PyObject *__pyx_v_numeric_owner) { int __pyx_v_flags; PyObject *__pyx_v_p = NULL; long __pyx_v_ret; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; char const *__pyx_t_4; int __pyx_t_5; int __pyx_t_6; long __pyx_t_7; __Pyx_RefNannySetupContext("acl_get", 0); /* "borg/platform_freebsd.pyx":50 * If `numeric_owner` is True the user/group field is not preserved only uid/gid * """ * cdef int flags = ACL_TEXT_APPEND_ID # <<<<<<<<<<<<<< * p = os.fsencode(path) * ret = lpathconf(p, _PC_ACL_NFS4) */ __pyx_v_flags = ACL_TEXT_APPEND_ID; /* "borg/platform_freebsd.pyx":51 * """ * cdef int flags = ACL_TEXT_APPEND_ID * p = os.fsencode(path) # <<<<<<<<<<<<<< * ret = lpathconf(p, _PC_ACL_NFS4) * if ret < 0 and errno == EINVAL: */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_os); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_path); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 51, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_v_p = __pyx_t_1; __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":52 * cdef int flags = ACL_TEXT_APPEND_ID * p = os.fsencode(path) * ret = lpathconf(p, _PC_ACL_NFS4) # <<<<<<<<<<<<<< * if ret < 0 and errno == EINVAL: * return */ __pyx_t_4 = __Pyx_PyObject_AsString(__pyx_v_p); if (unlikely((!__pyx_t_4) && PyErr_Occurred())) __PYX_ERR(0, 52, __pyx_L1_error) __pyx_v_ret = lpathconf(__pyx_t_4, _PC_ACL_NFS4); /* "borg/platform_freebsd.pyx":53 * p = os.fsencode(path) * ret = lpathconf(p, _PC_ACL_NFS4) * if ret < 0 and errno == EINVAL: # <<<<<<<<<<<<<< * return * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 */ __pyx_t_6 = ((__pyx_v_ret < 0) != 0); if (__pyx_t_6) { } else { __pyx_t_5 = __pyx_t_6; goto __pyx_L4_bool_binop_done; } __pyx_t_6 = ((errno == EINVAL) != 0); __pyx_t_5 = __pyx_t_6; __pyx_L4_bool_binop_done:; if (__pyx_t_5) { /* "borg/platform_freebsd.pyx":54 * ret = lpathconf(p, _PC_ACL_NFS4) * if ret < 0 and errno == EINVAL: * return # <<<<<<<<<<<<<< * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 * if ret > 0: */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; /* "borg/platform_freebsd.pyx":53 * p = os.fsencode(path) * ret = lpathconf(p, _PC_ACL_NFS4) * if ret < 0 and errno == EINVAL: # <<<<<<<<<<<<<< * return * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 */ } /* "borg/platform_freebsd.pyx":55 * if ret < 0 and errno == EINVAL: * return * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 # <<<<<<<<<<<<<< * if ret > 0: * _get_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', flags) */ __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 55, __pyx_L1_error) if (__pyx_t_5) { __pyx_t_7 = ACL_TEXT_NUMERIC_IDS; } else { __pyx_t_7 = 0; } __pyx_v_flags = (__pyx_v_flags | __pyx_t_7); /* "borg/platform_freebsd.pyx":56 * return * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 * if ret > 0: # <<<<<<<<<<<<<< * _get_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', flags) * else: */ __pyx_t_5 = ((__pyx_v_ret > 0) != 0); if (__pyx_t_5) { /* "borg/platform_freebsd.pyx":57 * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 * if ret > 0: * _get_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', flags) # <<<<<<<<<<<<<< * else: * _get_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', flags) */ __pyx_t_1 = __Pyx_PyInt_From_int(ACL_TYPE_NFS4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __pyx_f_4borg_16platform_freebsd__get_acl(__pyx_v_p, __pyx_t_1, __pyx_v_item, __pyx_n_b_acl_nfs4, __pyx_v_flags); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 57, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_freebsd.pyx":56 * return * flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 * if ret > 0: # <<<<<<<<<<<<<< * _get_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', flags) * else: */ goto __pyx_L6; } /* "borg/platform_freebsd.pyx":59 * _get_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', flags) * else: * _get_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', flags) # <<<<<<<<<<<<<< * _get_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', flags) * */ /*else*/ { __pyx_t_3 = __Pyx_PyInt_From_int(ACL_TYPE_ACCESS); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_1 = __pyx_f_4borg_16platform_freebsd__get_acl(__pyx_v_p, __pyx_t_3, __pyx_v_item, __pyx_n_b_acl_access, __pyx_v_flags); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":60 * else: * _get_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', flags) * _get_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', flags) # <<<<<<<<<<<<<< * * */ __pyx_t_1 = __Pyx_PyInt_From_int(ACL_TYPE_DEFAULT); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 60, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __pyx_f_4borg_16platform_freebsd__get_acl(__pyx_v_p, __pyx_t_1, __pyx_v_item, __pyx_n_b_acl_default, __pyx_v_flags); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 60, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; } __pyx_L6:; /* "borg/platform_freebsd.pyx":45 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.platform_freebsd.acl_get", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_p); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_freebsd.pyx":63 * * * cdef _set_acl(p, type, item, attribute, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl * text = item.get(attribute) */ static PyObject *__pyx_f_4borg_16platform_freebsd__set_acl(PyObject *__pyx_v_p, PyObject *__pyx_v_type, PyObject *__pyx_v_item, PyObject *__pyx_v_attribute, struct __pyx_opt_args_4borg_16platform_freebsd__set_acl *__pyx_optional_args) { PyObject *__pyx_v_numeric_owner = ((PyObject *)Py_False); acl_t __pyx_v_acl; PyObject *__pyx_v_text = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; int __pyx_t_5; int __pyx_t_6; char const *__pyx_t_7; char const *__pyx_t_8; int __pyx_t_9; __Pyx_RefNannySetupContext("_set_acl", 0); if (__pyx_optional_args) { if (__pyx_optional_args->__pyx_n > 0) { __pyx_v_numeric_owner = __pyx_optional_args->numeric_owner; } } /* "borg/platform_freebsd.pyx":65 * cdef _set_acl(p, type, item, attribute, numeric_owner=False): * cdef acl_t acl * text = item.get(attribute) # <<<<<<<<<<<<<< * if text: * if numeric_owner and type == ACL_TYPE_NFS4: */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_item, __pyx_n_s_get); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 65, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_3 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_3)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_3); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_3) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_3, __pyx_v_attribute) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_v_attribute); __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 65, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_text = __pyx_t_1; __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":66 * cdef acl_t acl * text = item.get(attribute) * if text: # <<<<<<<<<<<<<< * if numeric_owner and type == ACL_TYPE_NFS4: * text = _nfs4_use_stored_uid_gid(text) */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_text); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 66, __pyx_L1_error) if (__pyx_t_4) { /* "borg/platform_freebsd.pyx":67 * text = item.get(attribute) * if text: * if numeric_owner and type == ACL_TYPE_NFS4: # <<<<<<<<<<<<<< * text = _nfs4_use_stored_uid_gid(text) * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): */ __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 67, __pyx_L1_error) if (__pyx_t_5) { } else { __pyx_t_4 = __pyx_t_5; goto __pyx_L5_bool_binop_done; } __pyx_t_1 = __Pyx_PyInt_From_int(ACL_TYPE_NFS4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 67, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = PyObject_RichCompare(__pyx_v_type, __pyx_t_1, Py_EQ); __Pyx_XGOTREF(__pyx_t_2); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 67, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 67, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_4 = __pyx_t_5; __pyx_L5_bool_binop_done:; if (__pyx_t_4) { /* "borg/platform_freebsd.pyx":68 * if text: * if numeric_owner and type == ACL_TYPE_NFS4: * text = _nfs4_use_stored_uid_gid(text) # <<<<<<<<<<<<<< * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): * text = posix_acl_use_stored_uid_gid(text) */ __pyx_t_2 = __pyx_f_4borg_16platform_freebsd__nfs4_use_stored_uid_gid(__pyx_v_text); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF_SET(__pyx_v_text, __pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_freebsd.pyx":67 * text = item.get(attribute) * if text: * if numeric_owner and type == ACL_TYPE_NFS4: # <<<<<<<<<<<<<< * text = _nfs4_use_stored_uid_gid(text) * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): */ goto __pyx_L4; } /* "borg/platform_freebsd.pyx":69 * if numeric_owner and type == ACL_TYPE_NFS4: * text = _nfs4_use_stored_uid_gid(text) * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): # <<<<<<<<<<<<<< * text = posix_acl_use_stored_uid_gid(text) * acl = acl_from_text(text) */ __pyx_t_5 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_5 < 0)) __PYX_ERR(0, 69, __pyx_L1_error) if (__pyx_t_5) { } else { __pyx_t_4 = __pyx_t_5; goto __pyx_L7_bool_binop_done; } __Pyx_INCREF(__pyx_v_type); __pyx_t_2 = __pyx_v_type; __pyx_t_1 = __Pyx_PyInt_From_int(ACL_TYPE_ACCESS); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = PyObject_RichCompare(__pyx_t_2, __pyx_t_1, Py_EQ); __Pyx_XGOTREF(__pyx_t_3); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_6 = __Pyx_PyObject_IsTrue(__pyx_t_3); if (unlikely(__pyx_t_6 < 0)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (!__pyx_t_6) { } else { __pyx_t_5 = __pyx_t_6; goto __pyx_L9_bool_binop_done; } __pyx_t_3 = __Pyx_PyInt_From_int(ACL_TYPE_DEFAULT); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_1 = PyObject_RichCompare(__pyx_t_2, __pyx_t_3, Py_EQ); __Pyx_XGOTREF(__pyx_t_1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_6 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_6 < 0)) __PYX_ERR(0, 69, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_5 = __pyx_t_6; __pyx_L9_bool_binop_done:; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_6 = (__pyx_t_5 != 0); __pyx_t_4 = __pyx_t_6; __pyx_L7_bool_binop_done:; if (__pyx_t_4) { /* "borg/platform_freebsd.pyx":70 * text = _nfs4_use_stored_uid_gid(text) * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): * text = posix_acl_use_stored_uid_gid(text) # <<<<<<<<<<<<<< * acl = acl_from_text(text) * if acl: */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_posix_acl_use_stored_uid_gid); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_3 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_3)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_3); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_2 = (__pyx_t_3) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_3, __pyx_v_text) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_v_text); __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF_SET(__pyx_v_text, __pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_freebsd.pyx":69 * if numeric_owner and type == ACL_TYPE_NFS4: * text = _nfs4_use_stored_uid_gid(text) * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): # <<<<<<<<<<<<<< * text = posix_acl_use_stored_uid_gid(text) * acl = acl_from_text(text) */ } __pyx_L4:; /* "borg/platform_freebsd.pyx":71 * elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): * text = posix_acl_use_stored_uid_gid(text) * acl = acl_from_text(text) # <<<<<<<<<<<<<< * if acl: * acl_set_link_np(p, type, acl) */ if (unlikely(__pyx_v_text == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 71, __pyx_L1_error) } __pyx_t_7 = __Pyx_PyBytes_AsString(__pyx_v_text); if (unlikely((!__pyx_t_7) && PyErr_Occurred())) __PYX_ERR(0, 71, __pyx_L1_error) __pyx_v_acl = acl_from_text(__pyx_t_7); /* "borg/platform_freebsd.pyx":72 * text = posix_acl_use_stored_uid_gid(text) * acl = acl_from_text(text) * if acl: # <<<<<<<<<<<<<< * acl_set_link_np(p, type, acl) * acl_free(acl) */ __pyx_t_4 = (__pyx_v_acl != 0); if (__pyx_t_4) { /* "borg/platform_freebsd.pyx":73 * acl = acl_from_text(text) * if acl: * acl_set_link_np(p, type, acl) # <<<<<<<<<<<<<< * acl_free(acl) * */ __pyx_t_8 = __Pyx_PyObject_AsString(__pyx_v_p); if (unlikely((!__pyx_t_8) && PyErr_Occurred())) __PYX_ERR(0, 73, __pyx_L1_error) __pyx_t_9 = __Pyx_PyInt_As_int(__pyx_v_type); if (unlikely((__pyx_t_9 == (int)-1) && PyErr_Occurred())) __PYX_ERR(0, 73, __pyx_L1_error) (void)(acl_set_link_np(__pyx_t_8, __pyx_t_9, __pyx_v_acl)); /* "borg/platform_freebsd.pyx":74 * if acl: * acl_set_link_np(p, type, acl) * acl_free(acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_acl)); /* "borg/platform_freebsd.pyx":72 * text = posix_acl_use_stored_uid_gid(text) * acl = acl_from_text(text) * if acl: # <<<<<<<<<<<<<< * acl_set_link_np(p, type, acl) * acl_free(acl) */ } /* "borg/platform_freebsd.pyx":66 * cdef acl_t acl * text = item.get(attribute) * if text: # <<<<<<<<<<<<<< * if numeric_owner and type == ACL_TYPE_NFS4: * text = _nfs4_use_stored_uid_gid(text) */ } /* "borg/platform_freebsd.pyx":63 * * * cdef _set_acl(p, type, item, attribute, numeric_owner=False): # <<<<<<<<<<<<<< * cdef acl_t acl * text = item.get(attribute) */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.platform_freebsd._set_acl", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XDECREF(__pyx_v_text); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_freebsd.pyx":77 * * * cdef _nfs4_use_stored_uid_gid(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the stored uid/gid * """ */ static PyObject *__pyx_f_4borg_16platform_freebsd__nfs4_use_stored_uid_gid(PyObject *__pyx_v_acl) { PyObject *__pyx_v_entries = NULL; PyObject *__pyx_v_entry = NULL; PyObject *__pyx_v_fields = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; Py_ssize_t __pyx_t_5; PyObject *(*__pyx_t_6)(PyObject *); int __pyx_t_7; int __pyx_t_8; PyObject *__pyx_t_9 = NULL; int __pyx_t_10; __Pyx_RefNannySetupContext("_nfs4_use_stored_uid_gid", 0); /* "borg/platform_freebsd.pyx":80 * """Replace the user/group field with the stored uid/gid * """ * entries = [] # <<<<<<<<<<<<<< * for entry in safe_decode(acl).split('\n'): * if entry: */ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_entries = ((PyObject*)__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":81 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * if entry.startswith('user:') or entry.startswith('group:'): */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_4, __pyx_v_acl) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_acl); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_split); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_kp_s_) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_kp_s_); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (likely(PyList_CheckExact(__pyx_t_1)) || PyTuple_CheckExact(__pyx_t_1)) { __pyx_t_3 = __pyx_t_1; __Pyx_INCREF(__pyx_t_3); __pyx_t_5 = 0; __pyx_t_6 = NULL; } else { __pyx_t_5 = -1; __pyx_t_3 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = Py_TYPE(__pyx_t_3)->tp_iternext; if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 81, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; for (;;) { if (likely(!__pyx_t_6)) { if (likely(PyList_CheckExact(__pyx_t_3))) { if (__pyx_t_5 >= PyList_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyList_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 81, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } else { if (__pyx_t_5 >= PyTuple_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyTuple_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 81, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 81, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } } else { __pyx_t_1 = __pyx_t_6(__pyx_t_3); if (unlikely(!__pyx_t_1)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 81, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_1); } __Pyx_XDECREF_SET(__pyx_v_entry, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":82 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * if entry.startswith('user:') or entry.startswith('group:'): * fields = entry.split(':') */ __pyx_t_7 = __Pyx_PyObject_IsTrue(__pyx_v_entry); if (unlikely(__pyx_t_7 < 0)) __PYX_ERR(0, 82, __pyx_L1_error) if (__pyx_t_7) { /* "borg/platform_freebsd.pyx":83 * for entry in safe_decode(acl).split('\n'): * if entry: * if entry.startswith('user:') or entry.startswith('group:'): # <<<<<<<<<<<<<< * fields = entry.split(':') * entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_startswith); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 83, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_kp_s_user) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s_user); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 83, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 83, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (!__pyx_t_8) { } else { __pyx_t_7 = __pyx_t_8; goto __pyx_L7_bool_binop_done; } __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_startswith); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 83, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_kp_s_group) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s_group); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 83, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 83, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_7 = __pyx_t_8; __pyx_L7_bool_binop_done:; if (__pyx_t_7) { /* "borg/platform_freebsd.pyx":84 * if entry: * if entry.startswith('user:') or entry.startswith('group:'): * fields = entry.split(':') # <<<<<<<<<<<<<< * entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) * else: */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_split); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 84, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_kp_s__2) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s__2); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 84, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF_SET(__pyx_v_fields, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":85 * if entry.startswith('user:') or entry.startswith('group:'): * fields = entry.split(':') * entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) # <<<<<<<<<<<<<< * else: * entries.append(entry) */ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_kp_s__2, __pyx_n_s_join); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_fields, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = __Pyx_GetItemInt(__pyx_v_fields, 5, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_9 = PyTuple_New(2); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_9, 0, __pyx_t_2); __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_9, 1, __pyx_t_4); __pyx_t_2 = 0; __pyx_t_4 = 0; __pyx_t_4 = __Pyx_PyObject_GetSlice(__pyx_v_fields, 2, -1L, NULL, NULL, &__pyx_slice__3, 1, 1, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_2 = __Pyx_PySequence_Tuple(__pyx_t_4); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_4 = PyNumber_Add(__pyx_t_9, __pyx_t_2); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_t_4, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_10 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_2); if (unlikely(__pyx_t_10 == ((int)-1))) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_freebsd.pyx":83 * for entry in safe_decode(acl).split('\n'): * if entry: * if entry.startswith('user:') or entry.startswith('group:'): # <<<<<<<<<<<<<< * fields = entry.split(':') * entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) */ goto __pyx_L6; } /* "borg/platform_freebsd.pyx":87 * entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) * else: * entries.append(entry) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ /*else*/ { __pyx_t_10 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_v_entry); if (unlikely(__pyx_t_10 == ((int)-1))) __PYX_ERR(0, 87, __pyx_L1_error) } __pyx_L6:; /* "borg/platform_freebsd.pyx":82 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * if entry.startswith('user:') or entry.startswith('group:'): * fields = entry.split(':') */ } /* "borg/platform_freebsd.pyx":81 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * if entry.startswith('user:') or entry.startswith('group:'): */ } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_freebsd.pyx":88 * else: * entries.append(entry) * return safe_encode('\n'.join(entries)) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 88, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = __Pyx_PyString_Join(__pyx_kp_s_, __pyx_v_entries); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 88, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_1 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_1 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_1)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_1); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_3 = (__pyx_t_1) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_1, __pyx_t_4) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_t_4); __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 88, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* "borg/platform_freebsd.pyx":77 * * * cdef _nfs4_use_stored_uid_gid(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the stored uid/gid * """ */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_9); __Pyx_AddTraceback("borg.platform_freebsd._nfs4_use_stored_uid_gid", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XDECREF(__pyx_v_entries); __Pyx_XDECREF(__pyx_v_entry); __Pyx_XDECREF(__pyx_v_fields); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_freebsd.pyx":91 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_16platform_freebsd_3acl_set(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_16platform_freebsd_2acl_set[] = "Restore ACL Entries\n\n If `numeric_owner` is True the stored uid/gid is used instead\n of the user/group names\n "; static PyMethodDef __pyx_mdef_4borg_16platform_freebsd_3acl_set = {"acl_set", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_16platform_freebsd_3acl_set, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_16platform_freebsd_2acl_set}; static PyObject *__pyx_pw_4borg_16platform_freebsd_3acl_set(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_path = 0; PyObject *__pyx_v_item = 0; PyObject *__pyx_v_numeric_owner = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_set (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_path,&__pyx_n_s_item,&__pyx_n_s_numeric_owner,0}; PyObject* values[3] = {0,0,0}; values[2] = ((PyObject *)Py_False); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_item)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_set", 0, 2, 3, 1); __PYX_ERR(0, 91, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_numeric_owner); if (value) { values[2] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "acl_set") < 0)) __PYX_ERR(0, 91, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_path = values[0]; __pyx_v_item = values[1]; __pyx_v_numeric_owner = values[2]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("acl_set", 0, 2, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 91, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.platform_freebsd.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_16platform_freebsd_2acl_set(__pyx_self, __pyx_v_path, __pyx_v_item, __pyx_v_numeric_owner); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_16platform_freebsd_2acl_set(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_numeric_owner) { PyObject *__pyx_v_p = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; struct __pyx_opt_args_4borg_16platform_freebsd__set_acl __pyx_t_4; __Pyx_RefNannySetupContext("acl_set", 0); /* "borg/platform_freebsd.pyx":97 * of the user/group names * """ * p = os.fsencode(path) # <<<<<<<<<<<<<< * _set_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', numeric_owner) * _set_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', numeric_owner) */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_os); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_path); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 97, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_v_p = __pyx_t_1; __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":98 * """ * p = os.fsencode(path) * _set_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', numeric_owner) # <<<<<<<<<<<<<< * _set_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', numeric_owner) * _set_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', numeric_owner) */ __pyx_t_1 = __Pyx_PyInt_From_int(ACL_TYPE_NFS4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 98, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_4.__pyx_n = 1; __pyx_t_4.numeric_owner = __pyx_v_numeric_owner; __pyx_t_3 = __pyx_f_4borg_16platform_freebsd__set_acl(__pyx_v_p, __pyx_t_1, __pyx_v_item, __pyx_n_b_acl_nfs4, &__pyx_t_4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 98, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_freebsd.pyx":99 * p = os.fsencode(path) * _set_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', numeric_owner) * _set_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', numeric_owner) # <<<<<<<<<<<<<< * _set_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', numeric_owner) */ __pyx_t_3 = __Pyx_PyInt_From_int(ACL_TYPE_ACCESS); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 99, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4.__pyx_n = 1; __pyx_t_4.numeric_owner = __pyx_v_numeric_owner; __pyx_t_1 = __pyx_f_4borg_16platform_freebsd__set_acl(__pyx_v_p, __pyx_t_3, __pyx_v_item, __pyx_n_b_acl_access, &__pyx_t_4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 99, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":100 * _set_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', numeric_owner) * _set_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', numeric_owner) * _set_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', numeric_owner) # <<<<<<<<<<<<<< */ __pyx_t_1 = __Pyx_PyInt_From_int(ACL_TYPE_DEFAULT); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 100, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_4.__pyx_n = 1; __pyx_t_4.numeric_owner = __pyx_v_numeric_owner; __pyx_t_3 = __pyx_f_4borg_16platform_freebsd__set_acl(__pyx_v_p, __pyx_t_1, __pyx_v_item, __pyx_n_b_acl_default, &__pyx_t_4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 100, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_freebsd.pyx":91 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.platform_freebsd.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_p); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_platform_freebsd(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_platform_freebsd}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "platform_freebsd", 0, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_, __pyx_k_, sizeof(__pyx_k_), 0, 0, 1, 0}, {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_kp_s__2, __pyx_k__2, sizeof(__pyx_k__2), 0, 0, 1, 0}, {&__pyx_n_b_acl_access, __pyx_k_acl_access, sizeof(__pyx_k_acl_access), 0, 0, 0, 1}, {&__pyx_n_b_acl_default, __pyx_k_acl_default, sizeof(__pyx_k_acl_default), 0, 0, 0, 1}, {&__pyx_n_s_acl_get, __pyx_k_acl_get, sizeof(__pyx_k_acl_get), 0, 0, 1, 1}, {&__pyx_n_b_acl_nfs4, __pyx_k_acl_nfs4, sizeof(__pyx_k_acl_nfs4), 0, 0, 0, 1}, {&__pyx_n_s_acl_set, __pyx_k_acl_set, sizeof(__pyx_k_acl_set), 0, 0, 1, 1}, {&__pyx_n_s_borg_platform_freebsd, __pyx_k_borg_platform_freebsd, sizeof(__pyx_k_borg_platform_freebsd), 0, 0, 1, 1}, {&__pyx_kp_s_borg_platform_freebsd_pyx, __pyx_k_borg_platform_freebsd_pyx, sizeof(__pyx_k_borg_platform_freebsd_pyx), 0, 0, 1, 0}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_flags, __pyx_k_flags, sizeof(__pyx_k_flags), 0, 0, 1, 1}, {&__pyx_n_s_fsencode, __pyx_k_fsencode, sizeof(__pyx_k_fsencode), 0, 0, 1, 1}, {&__pyx_n_s_get, __pyx_k_get, sizeof(__pyx_k_get), 0, 0, 1, 1}, {&__pyx_kp_s_group, __pyx_k_group, sizeof(__pyx_k_group), 0, 0, 1, 0}, {&__pyx_n_s_helpers, __pyx_k_helpers, sizeof(__pyx_k_helpers), 0, 0, 1, 1}, {&__pyx_n_s_import, __pyx_k_import, sizeof(__pyx_k_import), 0, 0, 1, 1}, {&__pyx_n_s_item, __pyx_k_item, sizeof(__pyx_k_item), 0, 0, 1, 1}, {&__pyx_n_s_join, __pyx_k_join, sizeof(__pyx_k_join), 0, 0, 1, 1}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_n_s_numeric_owner, __pyx_k_numeric_owner, sizeof(__pyx_k_numeric_owner), 0, 0, 1, 1}, {&__pyx_n_s_os, __pyx_k_os, sizeof(__pyx_k_os), 0, 0, 1, 1}, {&__pyx_n_s_p, __pyx_k_p, sizeof(__pyx_k_p), 0, 0, 1, 1}, {&__pyx_n_s_path, __pyx_k_path, sizeof(__pyx_k_path), 0, 0, 1, 1}, {&__pyx_n_s_posix_acl_use_stored_uid_gid, __pyx_k_posix_acl_use_stored_uid_gid, sizeof(__pyx_k_posix_acl_use_stored_uid_gid), 0, 0, 1, 1}, {&__pyx_n_s_ret, __pyx_k_ret, sizeof(__pyx_k_ret), 0, 0, 1, 1}, {&__pyx_n_s_safe_decode, __pyx_k_safe_decode, sizeof(__pyx_k_safe_decode), 0, 0, 1, 1}, {&__pyx_n_s_safe_encode, __pyx_k_safe_encode, sizeof(__pyx_k_safe_encode), 0, 0, 1, 1}, {&__pyx_n_s_split, __pyx_k_split, sizeof(__pyx_k_split), 0, 0, 1, 1}, {&__pyx_n_s_st, __pyx_k_st, sizeof(__pyx_k_st), 0, 0, 1, 1}, {&__pyx_n_s_startswith, __pyx_k_startswith, sizeof(__pyx_k_startswith), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {&__pyx_kp_s_user, __pyx_k_user, sizeof(__pyx_k_user), 0, 0, 1, 0}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { return 0; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "borg/platform_freebsd.pyx":85 * if entry.startswith('user:') or entry.startswith('group:'): * fields = entry.split(':') * entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) # <<<<<<<<<<<<<< * else: * entries.append(entry) */ __pyx_slice__3 = PySlice_New(__pyx_int_2, __pyx_int_neg_1, Py_None); if (unlikely(!__pyx_slice__3)) __PYX_ERR(0, 85, __pyx_L1_error) __Pyx_GOTREF(__pyx_slice__3); __Pyx_GIVEREF(__pyx_slice__3); /* "borg/platform_freebsd.pyx":45 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ __pyx_tuple__4 = PyTuple_Pack(7, __pyx_n_s_path, __pyx_n_s_item, __pyx_n_s_st, __pyx_n_s_numeric_owner, __pyx_n_s_flags, __pyx_n_s_p, __pyx_n_s_ret); if (unlikely(!__pyx_tuple__4)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__4); __Pyx_GIVEREF(__pyx_tuple__4); __pyx_codeobj__5 = (PyObject*)__Pyx_PyCode_New(4, 0, 7, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__4, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_freebsd_pyx, __pyx_n_s_acl_get, 45, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__5)) __PYX_ERR(0, 45, __pyx_L1_error) /* "borg/platform_freebsd.pyx":91 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ __pyx_tuple__6 = PyTuple_Pack(4, __pyx_n_s_path, __pyx_n_s_item, __pyx_n_s_numeric_owner, __pyx_n_s_p); if (unlikely(!__pyx_tuple__6)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__6); __Pyx_GIVEREF(__pyx_tuple__6); __pyx_codeobj__7 = (PyObject*)__Pyx_PyCode_New(3, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__6, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_freebsd_pyx, __pyx_n_s_acl_set, 91, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__7)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(0, 1, __pyx_L1_error); __pyx_int_2 = PyInt_FromLong(2); if (unlikely(!__pyx_int_2)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_int_neg_1 = PyInt_FromLong(-1); if (unlikely(!__pyx_int_neg_1)) __PYX_ERR(0, 1, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC initplatform_freebsd(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC initplatform_freebsd(void) #else __Pyx_PyMODINIT_FUNC PyInit_platform_freebsd(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_platform_freebsd(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_platform_freebsd(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'platform_freebsd' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_platform_freebsd(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("platform_freebsd", __pyx_methods, 0, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(0, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(0, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(0, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(0, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__platform_freebsd) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name, __pyx_n_s_main) < 0) __PYX_ERR(0, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(0, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.platform_freebsd")) { if (unlikely(PyDict_SetItemString(modules, "borg.platform_freebsd", __pyx_m) < 0)) __PYX_ERR(0, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); (void)__Pyx_modinit_type_init_code(); (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /* "borg/platform_freebsd.pyx":1 * import os # <<<<<<<<<<<<<< * from .helpers import posix_acl_use_stored_uid_gid, safe_encode, safe_decode * */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_os, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_os, __pyx_t_1) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_freebsd.pyx":2 * import os * from .helpers import posix_acl_use_stored_uid_gid, safe_encode, safe_decode # <<<<<<<<<<<<<< * * API_VERSION = '1.0_01' */ __pyx_t_1 = PyList_New(3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_n_s_posix_acl_use_stored_uid_gid); __Pyx_GIVEREF(__pyx_n_s_posix_acl_use_stored_uid_gid); PyList_SET_ITEM(__pyx_t_1, 0, __pyx_n_s_posix_acl_use_stored_uid_gid); __Pyx_INCREF(__pyx_n_s_safe_encode); __Pyx_GIVEREF(__pyx_n_s_safe_encode); PyList_SET_ITEM(__pyx_t_1, 1, __pyx_n_s_safe_encode); __Pyx_INCREF(__pyx_n_s_safe_decode); __Pyx_GIVEREF(__pyx_n_s_safe_decode); PyList_SET_ITEM(__pyx_t_1, 2, __pyx_n_s_safe_decode); __pyx_t_2 = __Pyx_Import(__pyx_n_s_helpers, __pyx_t_1, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_posix_acl_use_stored_uid_gid); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_posix_acl_use_stored_uid_gid, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_safe_encode, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_safe_decode, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_freebsd.pyx":4 * from .helpers import posix_acl_use_stored_uid_gid, safe_encode, safe_decode * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * cdef extern from "errno.h": */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(0, 4, __pyx_L1_error) /* "borg/platform_freebsd.pyx":45 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_16platform_freebsd_1acl_get, NULL, __pyx_n_s_borg_platform_freebsd); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_get, __pyx_t_2) < 0) __PYX_ERR(0, 45, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_freebsd.pyx":91 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ __pyx_t_2 = PyCFunction_NewEx(&__pyx_mdef_4borg_16platform_freebsd_3acl_set, NULL, __pyx_n_s_borg_platform_freebsd); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_set, __pyx_t_2) < 0) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_freebsd.pyx":1 * import os # <<<<<<<<<<<<<< * from .helpers import posix_acl_use_stored_uid_gid, safe_encode, safe_decode * */ __pyx_t_2 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_2) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.platform_freebsd", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.platform_freebsd"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* GetModuleGlobalName */ #if CYTHON_USE_DICT_VERSIONS static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value) #else static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name) #endif { PyObject *result; #if !CYTHON_AVOID_BORROWED_REFS #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 result = _PyDict_GetItem_KnownHash(__pyx_d, name, ((PyASCIIObject *) name)->hash); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } else if (unlikely(PyErr_Occurred())) { return NULL; } #else result = PyDict_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } #endif #else result = PyObject_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } PyErr_Clear(); #endif return __Pyx_GetBuiltinName(name); } /* PyCFunctionFastCall */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, Py_ssize_t nargs) { PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); int flags = PyCFunction_GET_FLAGS(func); assert(PyCFunction_Check(func)); assert(METH_FASTCALL == (flags & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))); assert(nargs >= 0); assert(nargs == 0 || args != NULL); /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); if ((PY_VERSION_HEX < 0x030700A0) || unlikely(flags & METH_KEYWORDS)) { return (*((__Pyx_PyCFunctionFastWithKeywords)(void*)meth)) (self, args, nargs, NULL); } else { return (*((__Pyx_PyCFunctionFast)(void*)meth)) (self, args, nargs); } } #endif /* PyFunctionFastCall */ #if CYTHON_FAST_PYCALL static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject **fastlocals; Py_ssize_t i; PyObject *result; assert(globals != NULL); /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) { return NULL; } fastlocals = __Pyx_PyFrame_GetLocalsplus(f); for (i = 0; i < na; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } result = PyEval_EvalFrameEx(f,0); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; return result; } #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *closure; #if PY_MAJOR_VERSION >= 3 PyObject *kwdefs; #endif PyObject *kwtuple, **k; PyObject **d; Py_ssize_t nd; Py_ssize_t nk; PyObject *result; assert(kwargs == NULL || PyDict_Check(kwargs)); nk = kwargs ? PyDict_Size(kwargs) : 0; if (Py_EnterRecursiveCall((char*)" while calling a Python object")) { return NULL; } if ( #if PY_MAJOR_VERSION >= 3 co->co_kwonlyargcount == 0 && #endif likely(kwargs == NULL || nk == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { result = __Pyx_PyFunction_FastCallNoKw(co, args, nargs, globals); goto done; } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); result =__Pyx_PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), globals); goto done; } } if (kwargs != NULL) { Py_ssize_t pos, i; kwtuple = PyTuple_New(2 * nk); if (kwtuple == NULL) { result = NULL; goto done; } k = &PyTuple_GET_ITEM(kwtuple, 0); pos = i = 0; while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { Py_INCREF(k[i]); Py_INCREF(k[i+1]); i += 2; } nk = i / 2; } else { kwtuple = NULL; k = NULL; } closure = PyFunction_GET_CLOSURE(func); #if PY_MAJOR_VERSION >= 3 kwdefs = PyFunction_GET_KW_DEFAULTS(func); #endif if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } else { d = NULL; nd = 0; } #if PY_MAJOR_VERSION >= 3 result = PyEval_EvalCodeEx((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, kwdefs, closure); #else result = PyEval_EvalCodeEx(co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, closure); #endif Py_XDECREF(kwtuple); done: Py_LeaveRecursiveCall(); return result; } #endif #endif /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCall2Args */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2) { PyObject *args, *result = NULL; #if CYTHON_FAST_PYCALL if (PyFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyFunction_FastCall(function, args, 2); } #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyCFunction_FastCall(function, args, 2); } #endif args = PyTuple_New(2); if (unlikely(!args)) goto done; Py_INCREF(arg1); PyTuple_SET_ITEM(args, 0, arg1); Py_INCREF(arg2); PyTuple_SET_ITEM(args, 1, arg2); Py_INCREF(function); result = __Pyx_PyObject_Call(function, args, NULL); Py_DECREF(args); Py_DECREF(function); done: return result; } /* PyObjectCallMethO */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg) { PyObject *self, *result; PyCFunction cfunc; cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = cfunc(self, arg); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCallOneArg */ #if CYTHON_COMPILING_IN_CPYTHON static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_New(1); if (unlikely(!args)) return NULL; Py_INCREF(arg); PyTuple_SET_ITEM(args, 0, arg); result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, &arg, 1); } #endif if (likely(PyCFunction_Check(func))) { if (likely(PyCFunction_GET_FLAGS(func) & METH_O)) { return __Pyx_PyObject_CallMethO(func, arg); #if CYTHON_FAST_PYCCALL } else if (PyCFunction_GET_FLAGS(func) & METH_FASTCALL) { return __Pyx_PyCFunction_FastCall(func, &arg, 1); #endif } } return __Pyx__PyObject_CallOneArg(func, arg); } #else static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_Pack(1, arg); if (unlikely(!args)) return NULL; result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } #endif /* GetItemInt */ static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) { PyObject *r; if (!j) return NULL; r = PyObject_GetItem(o, j); Py_DECREF(j); return r; } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyList_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyList_GET_SIZE(o)))) { PyObject *r = PyList_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyTuple_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyList_GET_SIZE(o); if ((!boundscheck) || (likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o))))) { PyObject *r = PyList_GET_ITEM(o, n); Py_INCREF(r); return r; } } else if (PyTuple_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyTuple_GET_SIZE(o); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, n); Py_INCREF(r); return r; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return NULL; PyErr_Clear(); } } return m->sq_item(o, i); } } #else if (is_list || PySequence_Check(o)) { return PySequence_GetItem(o, i); } #endif return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); } /* SliceObject */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice(PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** _py_start, PyObject** _py_stop, PyObject** _py_slice, int has_cstart, int has_cstop, CYTHON_UNUSED int wraparound) { #if CYTHON_USE_TYPE_SLOTS PyMappingMethods* mp; #if PY_MAJOR_VERSION < 3 PySequenceMethods* ms = Py_TYPE(obj)->tp_as_sequence; if (likely(ms && ms->sq_slice)) { if (!has_cstart) { if (_py_start && (*_py_start != Py_None)) { cstart = __Pyx_PyIndex_AsSsize_t(*_py_start); if ((cstart == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstart = 0; } if (!has_cstop) { if (_py_stop && (*_py_stop != Py_None)) { cstop = __Pyx_PyIndex_AsSsize_t(*_py_stop); if ((cstop == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstop = PY_SSIZE_T_MAX; } if (wraparound && unlikely((cstart < 0) | (cstop < 0)) && likely(ms->sq_length)) { Py_ssize_t l = ms->sq_length(obj); if (likely(l >= 0)) { if (cstop < 0) { cstop += l; if (cstop < 0) cstop = 0; } if (cstart < 0) { cstart += l; if (cstart < 0) cstart = 0; } } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) goto bad; PyErr_Clear(); } } return ms->sq_slice(obj, cstart, cstop); } #endif mp = Py_TYPE(obj)->tp_as_mapping; if (likely(mp && mp->mp_subscript)) #endif { PyObject* result; PyObject *py_slice, *py_start, *py_stop; if (_py_slice) { py_slice = *_py_slice; } else { PyObject* owned_start = NULL; PyObject* owned_stop = NULL; if (_py_start) { py_start = *_py_start; } else { if (has_cstart) { owned_start = py_start = PyInt_FromSsize_t(cstart); if (unlikely(!py_start)) goto bad; } else py_start = Py_None; } if (_py_stop) { py_stop = *_py_stop; } else { if (has_cstop) { owned_stop = py_stop = PyInt_FromSsize_t(cstop); if (unlikely(!py_stop)) { Py_XDECREF(owned_start); goto bad; } } else py_stop = Py_None; } py_slice = PySlice_New(py_start, py_stop, Py_None); Py_XDECREF(owned_start); Py_XDECREF(owned_stop); if (unlikely(!py_slice)) goto bad; } #if CYTHON_USE_TYPE_SLOTS result = mp->mp_subscript(obj, py_slice); #else result = PyObject_GetItem(obj, py_slice); #endif if (!_py_slice) { Py_DECREF(py_slice); } return result; } PyErr_Format(PyExc_TypeError, "'%.200s' object is unsliceable", Py_TYPE(obj)->tp_name); bad: return NULL; } /* StringJoin */ #if !CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyBytes_Join(PyObject* sep, PyObject* values) { return PyObject_CallMethodObjArgs(sep, __pyx_n_s_join, values, NULL); } #endif /* Import */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level) { PyObject *empty_list = 0; PyObject *module = 0; PyObject *global_dict = 0; PyObject *empty_dict = 0; PyObject *list; #if PY_MAJOR_VERSION < 3 PyObject *py_import; py_import = __Pyx_PyObject_GetAttrStr(__pyx_b, __pyx_n_s_import); if (!py_import) goto bad; #endif if (from_list) list = from_list; else { empty_list = PyList_New(0); if (!empty_list) goto bad; list = empty_list; } global_dict = PyModule_GetDict(__pyx_m); if (!global_dict) goto bad; empty_dict = PyDict_New(); if (!empty_dict) goto bad; { #if PY_MAJOR_VERSION >= 3 if (level == -1) { if (strchr(__Pyx_MODULE_NAME, '.')) { module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, 1); if (!module) { if (!PyErr_ExceptionMatches(PyExc_ImportError)) goto bad; PyErr_Clear(); } } level = 0; } #endif if (!module) { #if PY_MAJOR_VERSION < 3 PyObject *py_level = PyInt_FromLong(level); if (!py_level) goto bad; module = PyObject_CallFunctionObjArgs(py_import, name, global_dict, empty_dict, list, py_level, (PyObject *)NULL); Py_DECREF(py_level); #else module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, level); #endif } } bad: #if PY_MAJOR_VERSION < 3 Py_XDECREF(py_import); #endif Py_XDECREF(empty_list); Py_XDECREF(empty_dict); return module; } /* ImportFrom */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name) { PyObject* value = __Pyx_PyObject_GetAttrStr(module, name); if (unlikely(!value) && PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Format(PyExc_ImportError, #if PY_MAJOR_VERSION < 3 "cannot import name %.230s", PyString_AS_STRING(name)); #else "cannot import name %S", name); #endif } return value; } /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_int(int value) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(int) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(int) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(int) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(int), little, !is_unsigned); } } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; ip) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/platform_freebsd.pyx0000644000175000017500000000573013431513133020704 0ustar useruser00000000000000import os from .helpers import posix_acl_use_stored_uid_gid, safe_encode, safe_decode API_VERSION = '1.0_01' cdef extern from "errno.h": int errno int EINVAL cdef extern from "sys/types.h": int ACL_TYPE_ACCESS int ACL_TYPE_DEFAULT int ACL_TYPE_NFS4 cdef extern from "sys/acl.h": ctypedef struct _acl_t: pass ctypedef _acl_t *acl_t int acl_free(void *obj) acl_t acl_get_link_np(const char *path, int type) int acl_set_link_np(const char *path, int type, acl_t acl) acl_t acl_from_text(const char *buf) char *acl_to_text_np(acl_t acl, ssize_t *len, int flags) int ACL_TEXT_NUMERIC_IDS int ACL_TEXT_APPEND_ID cdef extern from "unistd.h": long lpathconf(const char *path, int name) int _PC_ACL_NFS4 cdef _get_acl(p, type, item, attribute, int flags): cdef acl_t acl cdef char *text acl = acl_get_link_np(p, type) if acl: text = acl_to_text_np(acl, NULL, flags) if text: item[attribute] = text acl_free(text) acl_free(acl) def acl_get(path, item, st, numeric_owner=False): """Saves ACL Entries If `numeric_owner` is True the user/group field is not preserved only uid/gid """ cdef int flags = ACL_TEXT_APPEND_ID p = os.fsencode(path) ret = lpathconf(p, _PC_ACL_NFS4) if ret < 0 and errno == EINVAL: return flags |= ACL_TEXT_NUMERIC_IDS if numeric_owner else 0 if ret > 0: _get_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', flags) else: _get_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', flags) _get_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', flags) cdef _set_acl(p, type, item, attribute, numeric_owner=False): cdef acl_t acl text = item.get(attribute) if text: if numeric_owner and type == ACL_TYPE_NFS4: text = _nfs4_use_stored_uid_gid(text) elif numeric_owner and type in(ACL_TYPE_ACCESS, ACL_TYPE_DEFAULT): text = posix_acl_use_stored_uid_gid(text) acl = acl_from_text(text) if acl: acl_set_link_np(p, type, acl) acl_free(acl) cdef _nfs4_use_stored_uid_gid(acl): """Replace the user/group field with the stored uid/gid """ entries = [] for entry in safe_decode(acl).split('\n'): if entry: if entry.startswith('user:') or entry.startswith('group:'): fields = entry.split(':') entries.append(':'.join(fields[0], fields[5], *fields[2:-1])) else: entries.append(entry) return safe_encode('\n'.join(entries)) def acl_set(path, item, numeric_owner=False): """Restore ACL Entries If `numeric_owner` is True the stored uid/gid is used instead of the user/group names """ p = os.fsencode(path) _set_acl(p, ACL_TYPE_NFS4, item, b'acl_nfs4', numeric_owner) _set_acl(p, ACL_TYPE_ACCESS, item, b'acl_access', numeric_owner) _set_acl(p, ACL_TYPE_DEFAULT, item, b'acl_default', numeric_owner) borgbackup-1.0.13/borg/platform_linux.c0000644000175000017500000107004613431514213020036 0ustar useruser00000000000000/* Generated by Cython 0.29 */ #define PY_SSIZE_T_CLEAN #include "Python.h" #ifndef Py_PYTHON_H #error Python headers needed to compile C extensions, please install development version of Python. #elif PY_VERSION_HEX < 0x02060000 || (0x03000000 <= PY_VERSION_HEX && PY_VERSION_HEX < 0x03030000) #error Cython requires Python 2.6+ or Python 3.3+. #else #define CYTHON_ABI "0_29" #define CYTHON_HEX_VERSION 0x001D00F0 #define CYTHON_FUTURE_DIVISION 0 #include #ifndef offsetof #define offsetof(type, member) ( (size_t) & ((type*)0) -> member ) #endif #if !defined(WIN32) && !defined(MS_WINDOWS) #ifndef __stdcall #define __stdcall #endif #ifndef __cdecl #define __cdecl #endif #ifndef __fastcall #define __fastcall #endif #endif #ifndef DL_IMPORT #define DL_IMPORT(t) t #endif #ifndef DL_EXPORT #define DL_EXPORT(t) t #endif #define __PYX_COMMA , #ifndef HAVE_LONG_LONG #if PY_VERSION_HEX >= 0x02070000 #define HAVE_LONG_LONG #endif #endif #ifndef PY_LONG_LONG #define PY_LONG_LONG LONG_LONG #endif #ifndef Py_HUGE_VAL #define Py_HUGE_VAL HUGE_VAL #endif #ifdef PYPY_VERSION #define CYTHON_COMPILING_IN_PYPY 1 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 0 #undef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 0 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #if PY_VERSION_HEX < 0x03050000 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #undef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #undef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 1 #undef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 0 #undef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 0 #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #elif defined(PYSTON_VERSION) #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 1 #define CYTHON_COMPILING_IN_CPYTHON 0 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #undef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 0 #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #undef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 0 #undef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 0 #undef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT 0 #undef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE 0 #undef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS 0 #undef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK 0 #else #define CYTHON_COMPILING_IN_PYPY 0 #define CYTHON_COMPILING_IN_PYSTON 0 #define CYTHON_COMPILING_IN_CPYTHON 1 #ifndef CYTHON_USE_TYPE_SLOTS #define CYTHON_USE_TYPE_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYTYPE_LOOKUP #define CYTHON_USE_PYTYPE_LOOKUP 0 #elif !defined(CYTHON_USE_PYTYPE_LOOKUP) #define CYTHON_USE_PYTYPE_LOOKUP 1 #endif #if PY_MAJOR_VERSION < 3 #undef CYTHON_USE_ASYNC_SLOTS #define CYTHON_USE_ASYNC_SLOTS 0 #elif !defined(CYTHON_USE_ASYNC_SLOTS) #define CYTHON_USE_ASYNC_SLOTS 1 #endif #if PY_VERSION_HEX < 0x02070000 #undef CYTHON_USE_PYLONG_INTERNALS #define CYTHON_USE_PYLONG_INTERNALS 0 #elif !defined(CYTHON_USE_PYLONG_INTERNALS) #define CYTHON_USE_PYLONG_INTERNALS 1 #endif #ifndef CYTHON_USE_PYLIST_INTERNALS #define CYTHON_USE_PYLIST_INTERNALS 1 #endif #ifndef CYTHON_USE_UNICODE_INTERNALS #define CYTHON_USE_UNICODE_INTERNALS 1 #endif #if PY_VERSION_HEX < 0x030300F0 #undef CYTHON_USE_UNICODE_WRITER #define CYTHON_USE_UNICODE_WRITER 0 #elif !defined(CYTHON_USE_UNICODE_WRITER) #define CYTHON_USE_UNICODE_WRITER 1 #endif #ifndef CYTHON_AVOID_BORROWED_REFS #define CYTHON_AVOID_BORROWED_REFS 0 #endif #ifndef CYTHON_ASSUME_SAFE_MACROS #define CYTHON_ASSUME_SAFE_MACROS 1 #endif #ifndef CYTHON_UNPACK_METHODS #define CYTHON_UNPACK_METHODS 1 #endif #ifndef CYTHON_FAST_THREAD_STATE #define CYTHON_FAST_THREAD_STATE 1 #endif #ifndef CYTHON_FAST_PYCALL #define CYTHON_FAST_PYCALL 1 #endif #ifndef CYTHON_PEP489_MULTI_PHASE_INIT #define CYTHON_PEP489_MULTI_PHASE_INIT (PY_VERSION_HEX >= 0x03050000) #endif #ifndef CYTHON_USE_TP_FINALIZE #define CYTHON_USE_TP_FINALIZE (PY_VERSION_HEX >= 0x030400a1) #endif #ifndef CYTHON_USE_DICT_VERSIONS #define CYTHON_USE_DICT_VERSIONS (PY_VERSION_HEX >= 0x030600B1) #endif #ifndef CYTHON_USE_EXC_INFO_STACK #define CYTHON_USE_EXC_INFO_STACK (PY_VERSION_HEX >= 0x030700A3) #endif #endif #if !defined(CYTHON_FAST_PYCCALL) #define CYTHON_FAST_PYCCALL (CYTHON_FAST_PYCALL && PY_VERSION_HEX >= 0x030600B1) #endif #if CYTHON_USE_PYLONG_INTERNALS #include "longintrepr.h" #undef SHIFT #undef BASE #undef MASK #endif #ifndef __has_attribute #define __has_attribute(x) 0 #endif #ifndef __has_cpp_attribute #define __has_cpp_attribute(x) 0 #endif #ifndef CYTHON_RESTRICT #if defined(__GNUC__) #define CYTHON_RESTRICT __restrict__ #elif defined(_MSC_VER) && _MSC_VER >= 1400 #define CYTHON_RESTRICT __restrict #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_RESTRICT restrict #else #define CYTHON_RESTRICT #endif #endif #ifndef CYTHON_UNUSED # if defined(__GNUC__) # if !(defined(__cplusplus)) || (__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 4)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif # elif defined(__ICC) || (defined(__INTEL_COMPILER) && !defined(_MSC_VER)) # define CYTHON_UNUSED __attribute__ ((__unused__)) # else # define CYTHON_UNUSED # endif #endif #ifndef CYTHON_MAYBE_UNUSED_VAR # if defined(__cplusplus) template void CYTHON_MAYBE_UNUSED_VAR( const T& ) { } # else # define CYTHON_MAYBE_UNUSED_VAR(x) (void)(x) # endif #endif #ifndef CYTHON_NCP_UNUSED # if CYTHON_COMPILING_IN_CPYTHON # define CYTHON_NCP_UNUSED # else # define CYTHON_NCP_UNUSED CYTHON_UNUSED # endif #endif #define __Pyx_void_to_None(void_result) ((void)(void_result), Py_INCREF(Py_None), Py_None) #ifdef _MSC_VER #ifndef _MSC_STDINT_H_ #if _MSC_VER < 1300 typedef unsigned char uint8_t; typedef unsigned int uint32_t; #else typedef unsigned __int8 uint8_t; typedef unsigned __int32 uint32_t; #endif #endif #else #include #endif #ifndef CYTHON_FALLTHROUGH #if defined(__cplusplus) && __cplusplus >= 201103L #if __has_cpp_attribute(fallthrough) #define CYTHON_FALLTHROUGH [[fallthrough]] #elif __has_cpp_attribute(clang::fallthrough) #define CYTHON_FALLTHROUGH [[clang::fallthrough]] #elif __has_cpp_attribute(gnu::fallthrough) #define CYTHON_FALLTHROUGH [[gnu::fallthrough]] #endif #endif #ifndef CYTHON_FALLTHROUGH #if __has_attribute(fallthrough) #define CYTHON_FALLTHROUGH __attribute__((fallthrough)) #else #define CYTHON_FALLTHROUGH #endif #endif #if defined(__clang__ ) && defined(__apple_build_version__) #if __apple_build_version__ < 7000000 #undef CYTHON_FALLTHROUGH #define CYTHON_FALLTHROUGH #endif #endif #endif #ifndef CYTHON_INLINE #if defined(__clang__) #define CYTHON_INLINE __inline__ __attribute__ ((__unused__)) #elif defined(__GNUC__) #define CYTHON_INLINE __inline__ #elif defined(_MSC_VER) #define CYTHON_INLINE __inline #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define CYTHON_INLINE inline #else #define CYTHON_INLINE #endif #endif #if CYTHON_COMPILING_IN_PYPY && PY_VERSION_HEX < 0x02070600 && !defined(Py_OptimizeFlag) #define Py_OptimizeFlag 0 #endif #define __PYX_BUILD_PY_SSIZE_T "n" #define CYTHON_FORMAT_SSIZE_T "z" #if PY_MAJOR_VERSION < 3 #define __Pyx_BUILTIN_MODULE_NAME "__builtin__" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a+k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyClass_Type #else #define __Pyx_BUILTIN_MODULE_NAME "builtins" #define __Pyx_PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)\ PyCode_New(a, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos) #define __Pyx_DefaultClassType PyType_Type #endif #ifndef Py_TPFLAGS_CHECKTYPES #define Py_TPFLAGS_CHECKTYPES 0 #endif #ifndef Py_TPFLAGS_HAVE_INDEX #define Py_TPFLAGS_HAVE_INDEX 0 #endif #ifndef Py_TPFLAGS_HAVE_NEWBUFFER #define Py_TPFLAGS_HAVE_NEWBUFFER 0 #endif #ifndef Py_TPFLAGS_HAVE_FINALIZE #define Py_TPFLAGS_HAVE_FINALIZE 0 #endif #ifndef METH_STACKLESS #define METH_STACKLESS 0 #endif #if PY_VERSION_HEX <= 0x030700A3 || !defined(METH_FASTCALL) #ifndef METH_FASTCALL #define METH_FASTCALL 0x80 #endif typedef PyObject *(*__Pyx_PyCFunctionFast) (PyObject *self, PyObject *const *args, Py_ssize_t nargs); typedef PyObject *(*__Pyx_PyCFunctionFastWithKeywords) (PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames); #else #define __Pyx_PyCFunctionFast _PyCFunctionFast #define __Pyx_PyCFunctionFastWithKeywords _PyCFunctionFastWithKeywords #endif #if CYTHON_FAST_PYCCALL #define __Pyx_PyFastCFunction_Check(func)\ ((PyCFunction_Check(func) && (METH_FASTCALL == (PyCFunction_GET_FLAGS(func) & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))))) #else #define __Pyx_PyFastCFunction_Check(func) 0 #endif #if CYTHON_USE_DICT_VERSIONS #define __PYX_GET_DICT_VERSION(dict) (((PyDictObject*)(dict))->ma_version_tag) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var)\ (version_var) = __PYX_GET_DICT_VERSION(dict);\ (cache_var) = (value); #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ if (likely(__PYX_GET_DICT_VERSION(DICT) == __pyx_dict_version)) {\ (VAR) = __pyx_dict_cached_value;\ } else {\ (VAR) = __pyx_dict_cached_value = (LOOKUP);\ __pyx_dict_version = __PYX_GET_DICT_VERSION(DICT);\ }\ } #else #define __PYX_GET_DICT_VERSION(dict) (0) #define __PYX_UPDATE_DICT_CACHE(dict, value, cache_var, version_var) #define __PYX_PY_DICT_LOOKUP_IF_MODIFIED(VAR, DICT, LOOKUP) (VAR) = (LOOKUP); #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Malloc) #define PyObject_Malloc(s) PyMem_Malloc(s) #define PyObject_Free(p) PyMem_Free(p) #define PyObject_Realloc(p) PyMem_Realloc(p) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX < 0x030400A1 #define PyMem_RawMalloc(n) PyMem_Malloc(n) #define PyMem_RawRealloc(p, n) PyMem_Realloc(p, n) #define PyMem_RawFree(p) PyMem_Free(p) #endif #if CYTHON_COMPILING_IN_PYSTON #define __Pyx_PyCode_HasFreeVars(co) PyCode_HasFreeVars(co) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) PyFrame_SetLineNumber(frame, lineno) #else #define __Pyx_PyCode_HasFreeVars(co) (PyCode_GetNumFree(co) > 0) #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno) #endif #if !CYTHON_FAST_THREAD_STATE || PY_VERSION_HEX < 0x02070000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #elif PY_VERSION_HEX >= 0x03060000 #define __Pyx_PyThreadState_Current _PyThreadState_UncheckedGet() #elif PY_VERSION_HEX >= 0x03000000 #define __Pyx_PyThreadState_Current PyThreadState_GET() #else #define __Pyx_PyThreadState_Current _PyThreadState_Current #endif #if PY_VERSION_HEX < 0x030700A2 && !defined(PyThread_tss_create) && !defined(Py_tss_NEEDS_INIT) #include "pythread.h" #define Py_tss_NEEDS_INIT 0 typedef int Py_tss_t; static CYTHON_INLINE int PyThread_tss_create(Py_tss_t *key) { *key = PyThread_create_key(); return 0; // PyThread_create_key reports success always } static CYTHON_INLINE Py_tss_t * PyThread_tss_alloc(void) { Py_tss_t *key = (Py_tss_t *)PyObject_Malloc(sizeof(Py_tss_t)); *key = Py_tss_NEEDS_INIT; return key; } static CYTHON_INLINE void PyThread_tss_free(Py_tss_t *key) { PyObject_Free(key); } static CYTHON_INLINE int PyThread_tss_is_created(Py_tss_t *key) { return *key != Py_tss_NEEDS_INIT; } static CYTHON_INLINE void PyThread_tss_delete(Py_tss_t *key) { PyThread_delete_key(*key); *key = Py_tss_NEEDS_INIT; } static CYTHON_INLINE int PyThread_tss_set(Py_tss_t *key, void *value) { return PyThread_set_key_value(*key, value); } static CYTHON_INLINE void * PyThread_tss_get(Py_tss_t *key) { return PyThread_get_key_value(*key); } #endif // TSS (Thread Specific Storage) API #if CYTHON_COMPILING_IN_CPYTHON || defined(_PyDict_NewPresized) #define __Pyx_PyDict_NewPresized(n) ((n <= 8) ? PyDict_New() : _PyDict_NewPresized(n)) #else #define __Pyx_PyDict_NewPresized(n) PyDict_New() #endif #if PY_MAJOR_VERSION >= 3 || CYTHON_FUTURE_DIVISION #define __Pyx_PyNumber_Divide(x,y) PyNumber_TrueDivide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceTrueDivide(x,y) #else #define __Pyx_PyNumber_Divide(x,y) PyNumber_Divide(x,y) #define __Pyx_PyNumber_InPlaceDivide(x,y) PyNumber_InPlaceDivide(x,y) #endif #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 && CYTHON_USE_UNICODE_INTERNALS #define __Pyx_PyDict_GetItemStr(dict, name) _PyDict_GetItem_KnownHash(dict, name, ((PyASCIIObject *) name)->hash) #else #define __Pyx_PyDict_GetItemStr(dict, name) PyDict_GetItem(dict, name) #endif #if PY_VERSION_HEX > 0x03030000 && defined(PyUnicode_KIND) #define CYTHON_PEP393_ENABLED 1 #define __Pyx_PyUnicode_READY(op) (likely(PyUnicode_IS_READY(op)) ?\ 0 : _PyUnicode_Ready((PyObject *)(op))) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_LENGTH(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) PyUnicode_READ_CHAR(u, i) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) PyUnicode_MAX_CHAR_VALUE(u) #define __Pyx_PyUnicode_KIND(u) PyUnicode_KIND(u) #define __Pyx_PyUnicode_DATA(u) PyUnicode_DATA(u) #define __Pyx_PyUnicode_READ(k, d, i) PyUnicode_READ(k, d, i) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) PyUnicode_WRITE(k, d, i, ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != (likely(PyUnicode_IS_READY(u)) ? PyUnicode_GET_LENGTH(u) : PyUnicode_GET_SIZE(u))) #else #define CYTHON_PEP393_ENABLED 0 #define PyUnicode_1BYTE_KIND 1 #define PyUnicode_2BYTE_KIND 2 #define PyUnicode_4BYTE_KIND 4 #define __Pyx_PyUnicode_READY(op) (0) #define __Pyx_PyUnicode_GET_LENGTH(u) PyUnicode_GET_SIZE(u) #define __Pyx_PyUnicode_READ_CHAR(u, i) ((Py_UCS4)(PyUnicode_AS_UNICODE(u)[i])) #define __Pyx_PyUnicode_MAX_CHAR_VALUE(u) ((sizeof(Py_UNICODE) == 2) ? 65535 : 1114111) #define __Pyx_PyUnicode_KIND(u) (sizeof(Py_UNICODE)) #define __Pyx_PyUnicode_DATA(u) ((void*)PyUnicode_AS_UNICODE(u)) #define __Pyx_PyUnicode_READ(k, d, i) ((void)(k), (Py_UCS4)(((Py_UNICODE*)d)[i])) #define __Pyx_PyUnicode_WRITE(k, d, i, ch) (((void)(k)), ((Py_UNICODE*)d)[i] = ch) #define __Pyx_PyUnicode_IS_TRUE(u) (0 != PyUnicode_GET_SIZE(u)) #endif #if CYTHON_COMPILING_IN_PYPY #define __Pyx_PyUnicode_Concat(a, b) PyNumber_Add(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) PyNumber_Add(a, b) #else #define __Pyx_PyUnicode_Concat(a, b) PyUnicode_Concat(a, b) #define __Pyx_PyUnicode_ConcatSafe(a, b) ((unlikely((a) == Py_None) || unlikely((b) == Py_None)) ?\ PyNumber_Add(a, b) : __Pyx_PyUnicode_Concat(a, b)) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyUnicode_Contains) #define PyUnicode_Contains(u, s) PySequence_Contains(u, s) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyByteArray_Check) #define PyByteArray_Check(obj) PyObject_TypeCheck(obj, &PyByteArray_Type) #endif #if CYTHON_COMPILING_IN_PYPY && !defined(PyObject_Format) #define PyObject_Format(obj, fmt) PyObject_CallMethod(obj, "__format__", "O", fmt) #endif #define __Pyx_PyString_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyString_Check(b) && !PyString_CheckExact(b)))) ? PyNumber_Remainder(a, b) : __Pyx_PyString_Format(a, b)) #define __Pyx_PyUnicode_FormatSafe(a, b) ((unlikely((a) == Py_None || (PyUnicode_Check(b) && !PyUnicode_CheckExact(b)))) ? PyNumber_Remainder(a, b) : PyUnicode_Format(a, b)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Format(a, b) PyUnicode_Format(a, b) #else #define __Pyx_PyString_Format(a, b) PyString_Format(a, b) #endif #if PY_MAJOR_VERSION < 3 && !defined(PyObject_ASCII) #define PyObject_ASCII(o) PyObject_Repr(o) #endif #if PY_MAJOR_VERSION >= 3 #define PyBaseString_Type PyUnicode_Type #define PyStringObject PyUnicodeObject #define PyString_Type PyUnicode_Type #define PyString_Check PyUnicode_Check #define PyString_CheckExact PyUnicode_CheckExact #define PyObject_Unicode PyObject_Str #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyBaseString_Check(obj) PyUnicode_Check(obj) #define __Pyx_PyBaseString_CheckExact(obj) PyUnicode_CheckExact(obj) #else #define __Pyx_PyBaseString_Check(obj) (PyString_Check(obj) || PyUnicode_Check(obj)) #define __Pyx_PyBaseString_CheckExact(obj) (PyString_CheckExact(obj) || PyUnicode_CheckExact(obj)) #endif #ifndef PySet_CheckExact #define PySet_CheckExact(obj) (Py_TYPE(obj) == &PySet_Type) #endif #if CYTHON_ASSUME_SAFE_MACROS #define __Pyx_PySequence_SIZE(seq) Py_SIZE(seq) #else #define __Pyx_PySequence_SIZE(seq) PySequence_Size(seq) #endif #if PY_MAJOR_VERSION >= 3 #define PyIntObject PyLongObject #define PyInt_Type PyLong_Type #define PyInt_Check(op) PyLong_Check(op) #define PyInt_CheckExact(op) PyLong_CheckExact(op) #define PyInt_FromString PyLong_FromString #define PyInt_FromUnicode PyLong_FromUnicode #define PyInt_FromLong PyLong_FromLong #define PyInt_FromSize_t PyLong_FromSize_t #define PyInt_FromSsize_t PyLong_FromSsize_t #define PyInt_AsLong PyLong_AsLong #define PyInt_AS_LONG PyLong_AS_LONG #define PyInt_AsSsize_t PyLong_AsSsize_t #define PyInt_AsUnsignedLongMask PyLong_AsUnsignedLongMask #define PyInt_AsUnsignedLongLongMask PyLong_AsUnsignedLongLongMask #define PyNumber_Int PyNumber_Long #endif #if PY_MAJOR_VERSION >= 3 #define PyBoolObject PyLongObject #endif #if PY_MAJOR_VERSION >= 3 && CYTHON_COMPILING_IN_PYPY #ifndef PyUnicode_InternFromString #define PyUnicode_InternFromString(s) PyUnicode_FromString(s) #endif #endif #if PY_VERSION_HEX < 0x030200A4 typedef long Py_hash_t; #define __Pyx_PyInt_FromHash_t PyInt_FromLong #define __Pyx_PyInt_AsHash_t PyInt_AsLong #else #define __Pyx_PyInt_FromHash_t PyInt_FromSsize_t #define __Pyx_PyInt_AsHash_t PyInt_AsSsize_t #endif #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : (Py_INCREF(func), func)) #else #define __Pyx_PyMethod_New(func, self, klass) PyMethod_New(func, self, klass) #endif #if CYTHON_USE_ASYNC_SLOTS #if PY_VERSION_HEX >= 0x030500B1 #define __Pyx_PyAsyncMethodsStruct PyAsyncMethods #define __Pyx_PyType_AsAsync(obj) (Py_TYPE(obj)->tp_as_async) #else #define __Pyx_PyType_AsAsync(obj) ((__Pyx_PyAsyncMethodsStruct*) (Py_TYPE(obj)->tp_reserved)) #endif #else #define __Pyx_PyType_AsAsync(obj) NULL #endif #ifndef __Pyx_PyAsyncMethodsStruct typedef struct { unaryfunc am_await; unaryfunc am_aiter; unaryfunc am_anext; } __Pyx_PyAsyncMethodsStruct; #endif #if defined(WIN32) || defined(MS_WINDOWS) #define _USE_MATH_DEFINES #endif #include #ifdef NAN #define __PYX_NAN() ((float) NAN) #else static CYTHON_INLINE float __PYX_NAN() { float value; memset(&value, 0xFF, sizeof(value)); return value; } #endif #if defined(__CYGWIN__) && defined(_LDBL_EQ_DBL) #define __Pyx_truncl trunc #else #define __Pyx_truncl truncl #endif #define __PYX_ERR(f_index, lineno, Ln_error) \ { \ __pyx_filename = __pyx_f[f_index]; __pyx_lineno = lineno; __pyx_clineno = __LINE__; goto Ln_error; \ } #ifndef __PYX_EXTERN_C #ifdef __cplusplus #define __PYX_EXTERN_C extern "C" #else #define __PYX_EXTERN_C extern #endif #endif #define __PYX_HAVE__borg__platform_linux #define __PYX_HAVE_API__borg__platform_linux /* Early includes */ #include "sys/types.h" #include "sys/acl.h" #include "acl/libacl.h" #ifdef _OPENMP #include #endif /* _OPENMP */ #if defined(PYREX_WITHOUT_ASSERTIONS) && !defined(CYTHON_WITHOUT_ASSERTIONS) #define CYTHON_WITHOUT_ASSERTIONS #endif typedef struct {PyObject **p; const char *s; const Py_ssize_t n; const char* encoding; const char is_unicode; const char is_str; const char intern; } __Pyx_StringTabEntry; #define __PYX_DEFAULT_STRING_ENCODING_IS_ASCII 0 #define __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT 0 #define __PYX_DEFAULT_STRING_ENCODING "" #define __Pyx_PyObject_FromString __Pyx_PyBytes_FromString #define __Pyx_PyObject_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #define __Pyx_uchar_cast(c) ((unsigned char)c) #define __Pyx_long_cast(x) ((long)x) #define __Pyx_fits_Py_ssize_t(v, type, is_signed) (\ (sizeof(type) < sizeof(Py_ssize_t)) ||\ (sizeof(type) > sizeof(Py_ssize_t) &&\ likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX) &&\ (!is_signed || likely(v > (type)PY_SSIZE_T_MIN ||\ v == (type)PY_SSIZE_T_MIN))) ||\ (sizeof(type) == sizeof(Py_ssize_t) &&\ (is_signed || likely(v < (type)PY_SSIZE_T_MAX ||\ v == (type)PY_SSIZE_T_MAX))) ) static CYTHON_INLINE int __Pyx_is_valid_index(Py_ssize_t i, Py_ssize_t limit) { return (size_t) i < (size_t) limit; } #if defined (__cplusplus) && __cplusplus >= 201103L #include #define __Pyx_sst_abs(value) std::abs(value) #elif SIZEOF_INT >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) abs(value) #elif SIZEOF_LONG >= SIZEOF_SIZE_T #define __Pyx_sst_abs(value) labs(value) #elif defined (_MSC_VER) #define __Pyx_sst_abs(value) ((Py_ssize_t)_abs64(value)) #elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L #define __Pyx_sst_abs(value) llabs(value) #elif defined (__GNUC__) #define __Pyx_sst_abs(value) __builtin_llabs(value) #else #define __Pyx_sst_abs(value) ((value<0) ? -value : value) #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject*); static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject*, Py_ssize_t* length); #define __Pyx_PyByteArray_FromString(s) PyByteArray_FromStringAndSize((const char*)s, strlen((const char*)s)) #define __Pyx_PyByteArray_FromStringAndSize(s, l) PyByteArray_FromStringAndSize((const char*)s, l) #define __Pyx_PyBytes_FromString PyBytes_FromString #define __Pyx_PyBytes_FromStringAndSize PyBytes_FromStringAndSize static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char*); #if PY_MAJOR_VERSION < 3 #define __Pyx_PyStr_FromString __Pyx_PyBytes_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyBytes_FromStringAndSize #else #define __Pyx_PyStr_FromString __Pyx_PyUnicode_FromString #define __Pyx_PyStr_FromStringAndSize __Pyx_PyUnicode_FromStringAndSize #endif #define __Pyx_PyBytes_AsWritableString(s) ((char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableSString(s) ((signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsWritableUString(s) ((unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsString(s) ((const char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsSString(s) ((const signed char*) PyBytes_AS_STRING(s)) #define __Pyx_PyBytes_AsUString(s) ((const unsigned char*) PyBytes_AS_STRING(s)) #define __Pyx_PyObject_AsWritableString(s) ((char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableSString(s) ((signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsWritableUString(s) ((unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsSString(s) ((const signed char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_AsUString(s) ((const unsigned char*) __Pyx_PyObject_AsString(s)) #define __Pyx_PyObject_FromCString(s) __Pyx_PyObject_FromString((const char*)s) #define __Pyx_PyBytes_FromCString(s) __Pyx_PyBytes_FromString((const char*)s) #define __Pyx_PyByteArray_FromCString(s) __Pyx_PyByteArray_FromString((const char*)s) #define __Pyx_PyStr_FromCString(s) __Pyx_PyStr_FromString((const char*)s) #define __Pyx_PyUnicode_FromCString(s) __Pyx_PyUnicode_FromString((const char*)s) static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u) { const Py_UNICODE *u_end = u; while (*u_end++) ; return (size_t)(u_end - u - 1); } #define __Pyx_PyUnicode_FromUnicode(u) PyUnicode_FromUnicode(u, __Pyx_Py_UNICODE_strlen(u)) #define __Pyx_PyUnicode_FromUnicodeAndLength PyUnicode_FromUnicode #define __Pyx_PyUnicode_AsUnicode PyUnicode_AsUnicode #define __Pyx_NewRef(obj) (Py_INCREF(obj), obj) #define __Pyx_Owned_Py_None(b) __Pyx_NewRef(Py_None) static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b); static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject*); static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject*); static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x); #define __Pyx_PySequence_Tuple(obj)\ (likely(PyTuple_CheckExact(obj)) ? __Pyx_NewRef(obj) : PySequence_Tuple(obj)) static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject*); static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t); #if CYTHON_ASSUME_SAFE_MACROS #define __pyx_PyFloat_AsDouble(x) (PyFloat_CheckExact(x) ? PyFloat_AS_DOUBLE(x) : PyFloat_AsDouble(x)) #else #define __pyx_PyFloat_AsDouble(x) PyFloat_AsDouble(x) #endif #define __pyx_PyFloat_AsFloat(x) ((float) __pyx_PyFloat_AsDouble(x)) #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyNumber_Int(x) (PyLong_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Long(x)) #else #define __Pyx_PyNumber_Int(x) (PyInt_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Int(x)) #endif #define __Pyx_PyNumber_Float(x) (PyFloat_CheckExact(x) ? __Pyx_NewRef(x) : PyNumber_Float(x)) #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII static int __Pyx_sys_getdefaultencoding_not_ascii; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; PyObject* ascii_chars_u = NULL; PyObject* ascii_chars_b = NULL; const char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; if (strcmp(default_encoding_c, "ascii") == 0) { __Pyx_sys_getdefaultencoding_not_ascii = 0; } else { char ascii_chars[128]; int c; for (c = 0; c < 128; c++) { ascii_chars[c] = c; } __Pyx_sys_getdefaultencoding_not_ascii = 1; ascii_chars_u = PyUnicode_DecodeASCII(ascii_chars, 128, NULL); if (!ascii_chars_u) goto bad; ascii_chars_b = PyUnicode_AsEncodedString(ascii_chars_u, default_encoding_c, NULL); if (!ascii_chars_b || !PyBytes_Check(ascii_chars_b) || memcmp(ascii_chars, PyBytes_AS_STRING(ascii_chars_b), 128) != 0) { PyErr_Format( PyExc_ValueError, "This module compiled with c_string_encoding=ascii, but default encoding '%.200s' is not a superset of ascii.", default_encoding_c); goto bad; } Py_DECREF(ascii_chars_u); Py_DECREF(ascii_chars_b); } Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); Py_XDECREF(ascii_chars_u); Py_XDECREF(ascii_chars_b); return -1; } #endif #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT && PY_MAJOR_VERSION >= 3 #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_DecodeUTF8(c_str, size, NULL) #else #define __Pyx_PyUnicode_FromStringAndSize(c_str, size) PyUnicode_Decode(c_str, size, __PYX_DEFAULT_STRING_ENCODING, NULL) #if __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT static char* __PYX_DEFAULT_STRING_ENCODING; static int __Pyx_init_sys_getdefaultencoding_params(void) { PyObject* sys; PyObject* default_encoding = NULL; char* default_encoding_c; sys = PyImport_ImportModule("sys"); if (!sys) goto bad; default_encoding = PyObject_CallMethod(sys, (char*) (const char*) "getdefaultencoding", NULL); Py_DECREF(sys); if (!default_encoding) goto bad; default_encoding_c = PyBytes_AsString(default_encoding); if (!default_encoding_c) goto bad; __PYX_DEFAULT_STRING_ENCODING = (char*) malloc(strlen(default_encoding_c) + 1); if (!__PYX_DEFAULT_STRING_ENCODING) goto bad; strcpy(__PYX_DEFAULT_STRING_ENCODING, default_encoding_c); Py_DECREF(default_encoding); return 0; bad: Py_XDECREF(default_encoding); return -1; } #endif #endif /* Test for GCC > 2.95 */ #if defined(__GNUC__) && (__GNUC__ > 2 || (__GNUC__ == 2 && (__GNUC_MINOR__ > 95))) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else /* !__GNUC__ or GCC < 2.95 */ #define likely(x) (x) #define unlikely(x) (x) #endif /* __GNUC__ */ static CYTHON_INLINE void __Pyx_pretend_to_initialize(void* ptr) { (void)ptr; } static PyObject *__pyx_m = NULL; static PyObject *__pyx_d; static PyObject *__pyx_b; static PyObject *__pyx_cython_runtime = NULL; static PyObject *__pyx_empty_tuple; static PyObject *__pyx_empty_bytes; static PyObject *__pyx_empty_unicode; static int __pyx_lineno; static int __pyx_clineno = 0; static const char * __pyx_cfilenm= __FILE__; static const char *__pyx_filename; static const char *__pyx_f[] = { "borg/platform_linux.pyx", }; /*--- Type declarations ---*/ /* --- Runtime support code (head) --- */ /* Refnanny.proto */ #ifndef CYTHON_REFNANNY #define CYTHON_REFNANNY 0 #endif #if CYTHON_REFNANNY typedef struct { void (*INCREF)(void*, PyObject*, int); void (*DECREF)(void*, PyObject*, int); void (*GOTREF)(void*, PyObject*, int); void (*GIVEREF)(void*, PyObject*, int); void* (*SetupContext)(const char*, int, const char*); void (*FinishContext)(void**); } __Pyx_RefNannyAPIStruct; static __Pyx_RefNannyAPIStruct *__Pyx_RefNanny = NULL; static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname); #define __Pyx_RefNannyDeclarations void *__pyx_refnanny = NULL; #ifdef WITH_THREAD #define __Pyx_RefNannySetupContext(name, acquire_gil)\ if (acquire_gil) {\ PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ PyGILState_Release(__pyx_gilstate_save);\ } else {\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__);\ } #else #define __Pyx_RefNannySetupContext(name, acquire_gil)\ __pyx_refnanny = __Pyx_RefNanny->SetupContext((name), __LINE__, __FILE__) #endif #define __Pyx_RefNannyFinishContext()\ __Pyx_RefNanny->FinishContext(&__pyx_refnanny) #define __Pyx_INCREF(r) __Pyx_RefNanny->INCREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_DECREF(r) __Pyx_RefNanny->DECREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GOTREF(r) __Pyx_RefNanny->GOTREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_GIVEREF(r) __Pyx_RefNanny->GIVEREF(__pyx_refnanny, (PyObject *)(r), __LINE__) #define __Pyx_XINCREF(r) do { if((r) != NULL) {__Pyx_INCREF(r); }} while(0) #define __Pyx_XDECREF(r) do { if((r) != NULL) {__Pyx_DECREF(r); }} while(0) #define __Pyx_XGOTREF(r) do { if((r) != NULL) {__Pyx_GOTREF(r); }} while(0) #define __Pyx_XGIVEREF(r) do { if((r) != NULL) {__Pyx_GIVEREF(r);}} while(0) #else #define __Pyx_RefNannyDeclarations #define __Pyx_RefNannySetupContext(name, acquire_gil) #define __Pyx_RefNannyFinishContext() #define __Pyx_INCREF(r) Py_INCREF(r) #define __Pyx_DECREF(r) Py_DECREF(r) #define __Pyx_GOTREF(r) #define __Pyx_GIVEREF(r) #define __Pyx_XINCREF(r) Py_XINCREF(r) #define __Pyx_XDECREF(r) Py_XDECREF(r) #define __Pyx_XGOTREF(r) #define __Pyx_XGIVEREF(r) #endif #define __Pyx_XDECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_XDECREF(tmp);\ } while (0) #define __Pyx_DECREF_SET(r, v) do {\ PyObject *tmp = (PyObject *) r;\ r = v; __Pyx_DECREF(tmp);\ } while (0) #define __Pyx_CLEAR(r) do { PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);} while(0) #define __Pyx_XCLEAR(r) do { if((r) != NULL) {PyObject* tmp = ((PyObject*)(r)); r = NULL; __Pyx_DECREF(tmp);}} while(0) /* PyObjectGetAttrStr.proto */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name); #else #define __Pyx_PyObject_GetAttrStr(o,n) PyObject_GetAttr(o,n) #endif /* GetBuiltinName.proto */ static PyObject *__Pyx_GetBuiltinName(PyObject *name); /* GetModuleGlobalName.proto */ #if CYTHON_USE_DICT_VERSIONS #define __Pyx_GetModuleGlobalName(var, name) {\ static PY_UINT64_T __pyx_dict_version = 0;\ static PyObject *__pyx_dict_cached_value = NULL;\ (var) = (likely(__pyx_dict_version == __PYX_GET_DICT_VERSION(__pyx_d))) ?\ (likely(__pyx_dict_cached_value) ? __Pyx_NewRef(__pyx_dict_cached_value) : __Pyx_GetBuiltinName(name)) :\ __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } #define __Pyx_GetModuleGlobalNameUncached(var, name) {\ PY_UINT64_T __pyx_dict_version;\ PyObject *__pyx_dict_cached_value;\ (var) = __Pyx__GetModuleGlobalName(name, &__pyx_dict_version, &__pyx_dict_cached_value);\ } static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value); #else #define __Pyx_GetModuleGlobalName(var, name) (var) = __Pyx__GetModuleGlobalName(name) #define __Pyx_GetModuleGlobalNameUncached(var, name) (var) = __Pyx__GetModuleGlobalName(name) static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name); #endif /* PyCFunctionFastCall.proto */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject *__Pyx_PyCFunction_FastCall(PyObject *func, PyObject **args, Py_ssize_t nargs); #else #define __Pyx_PyCFunction_FastCall(func, args, nargs) (assert(0), NULL) #endif /* PyFunctionFastCall.proto */ #if CYTHON_FAST_PYCALL #define __Pyx_PyFunction_FastCall(func, args, nargs)\ __Pyx_PyFunction_FastCallDict((func), (args), (nargs), NULL) #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs); #else #define __Pyx_PyFunction_FastCallDict(func, args, nargs, kwargs) _PyFunction_FastCallDict(func, args, nargs, kwargs) #endif #define __Pyx_BUILD_ASSERT_EXPR(cond)\ (sizeof(char [1 - 2*!(cond)]) - 1) #ifndef Py_MEMBER_SIZE #define Py_MEMBER_SIZE(type, member) sizeof(((type *)0)->member) #endif static size_t __pyx_pyframe_localsplus_offset = 0; #include "frameobject.h" #define __Pxy_PyFrame_Initialize_Offsets()\ ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, f_localsplus)),\ (void)(__pyx_pyframe_localsplus_offset = PyFrame_Type.tp_basicsize - Py_MEMBER_SIZE(PyFrameObject, f_localsplus))) #define __Pyx_PyFrame_GetLocalsplus(frame)\ (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + __pyx_pyframe_localsplus_offset)) #endif /* PyObjectCall.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw); #else #define __Pyx_PyObject_Call(func, arg, kw) PyObject_Call(func, arg, kw) #endif /* PyObjectCall2Args.proto */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2); /* PyObjectCallMethO.proto */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg); #endif /* PyObjectCallOneArg.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg); /* GetItemInt.proto */ #define __Pyx_GetItemInt(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Fast(o, (Py_ssize_t)i, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL) :\ __Pyx_GetItemInt_Generic(o, to_py_func(i)))) #define __Pyx_GetItemInt_List(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_List_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "list index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); #define __Pyx_GetItemInt_Tuple(o, i, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_GetItemInt_Tuple_Fast(o, (Py_ssize_t)i, wraparound, boundscheck) :\ (PyErr_SetString(PyExc_IndexError, "tuple index out of range"), (PyObject*)NULL)) static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, int wraparound, int boundscheck); static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j); static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, int wraparound, int boundscheck); /* IncludeStringH.proto */ #include /* BytesEquals.proto */ static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2, int equals); /* UnicodeEquals.proto */ static CYTHON_INLINE int __Pyx_PyUnicode_Equals(PyObject* s1, PyObject* s2, int equals); /* StrEquals.proto */ #if PY_MAJOR_VERSION >= 3 #define __Pyx_PyString_Equals __Pyx_PyUnicode_Equals #else #define __Pyx_PyString_Equals __Pyx_PyBytes_Equals #endif /* SetItemInt.proto */ #define __Pyx_SetItemInt(o, i, v, type, is_signed, to_py_func, is_list, wraparound, boundscheck)\ (__Pyx_fits_Py_ssize_t(i, type, is_signed) ?\ __Pyx_SetItemInt_Fast(o, (Py_ssize_t)i, v, is_list, wraparound, boundscheck) :\ (is_list ? (PyErr_SetString(PyExc_IndexError, "list assignment index out of range"), -1) :\ __Pyx_SetItemInt_Generic(o, to_py_func(i), v))) static int __Pyx_SetItemInt_Generic(PyObject *o, PyObject *j, PyObject *v); static CYTHON_INLINE int __Pyx_SetItemInt_Fast(PyObject *o, Py_ssize_t i, PyObject *v, int is_list, int wraparound, int boundscheck); /* ListAppend.proto */ #if CYTHON_USE_PYLIST_INTERNALS && CYTHON_ASSUME_SAFE_MACROS static CYTHON_INLINE int __Pyx_PyList_Append(PyObject* list, PyObject* x) { PyListObject* L = (PyListObject*) list; Py_ssize_t len = Py_SIZE(list); if (likely(L->allocated > len) & likely(len > (L->allocated >> 1))) { Py_INCREF(x); PyList_SET_ITEM(list, len, x); Py_SIZE(list) = len+1; return 0; } return PyList_Append(list, x); } #else #define __Pyx_PyList_Append(L,x) PyList_Append(L,x) #endif /* StringJoin.proto */ #if PY_MAJOR_VERSION < 3 #define __Pyx_PyString_Join __Pyx_PyBytes_Join #define __Pyx_PyBaseString_Join(s, v) (PyUnicode_CheckExact(s) ? PyUnicode_Join(s, v) : __Pyx_PyBytes_Join(s, v)) #else #define __Pyx_PyString_Join PyUnicode_Join #define __Pyx_PyBaseString_Join PyUnicode_Join #endif #if CYTHON_COMPILING_IN_CPYTHON #if PY_MAJOR_VERSION < 3 #define __Pyx_PyBytes_Join _PyString_Join #else #define __Pyx_PyBytes_Join _PyBytes_Join #endif #else static CYTHON_INLINE PyObject* __Pyx_PyBytes_Join(PyObject* sep, PyObject* values); #endif /* SliceObject.proto */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice( PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** py_start, PyObject** py_stop, PyObject** py_slice, int has_cstart, int has_cstop, int wraparound); /* RaiseTooManyValuesToUnpack.proto */ static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected); /* RaiseNeedMoreValuesToUnpack.proto */ static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index); /* IterFinish.proto */ static CYTHON_INLINE int __Pyx_IterFinish(void); /* UnpackItemEndCheck.proto */ static int __Pyx_IternextUnpackEndCheck(PyObject *retval, Py_ssize_t expected); /* RaiseArgTupleInvalid.proto */ static void __Pyx_RaiseArgtupleInvalid(const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found); /* RaiseDoubleKeywords.proto */ static void __Pyx_RaiseDoubleKeywordsError(const char* func_name, PyObject* kw_name); /* ParseKeywords.proto */ static int __Pyx_ParseOptionalKeywords(PyObject *kwds, PyObject **argnames[],\ PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args,\ const char* function_name); /* PyThreadStateGet.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyThreadState_declare PyThreadState *__pyx_tstate; #define __Pyx_PyThreadState_assign __pyx_tstate = __Pyx_PyThreadState_Current; #define __Pyx_PyErr_Occurred() __pyx_tstate->curexc_type #else #define __Pyx_PyThreadState_declare #define __Pyx_PyThreadState_assign #define __Pyx_PyErr_Occurred() PyErr_Occurred() #endif /* PyErrFetchRestore.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_PyErr_Clear() __Pyx_ErrRestore(NULL, NULL, NULL) #define __Pyx_ErrRestoreWithState(type, value, tb) __Pyx_ErrRestoreInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) __Pyx_ErrFetchInState(PyThreadState_GET(), type, value, tb) #define __Pyx_ErrRestore(type, value, tb) __Pyx_ErrRestoreInState(__pyx_tstate, type, value, tb) #define __Pyx_ErrFetch(type, value, tb) __Pyx_ErrFetchInState(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_PyErr_SetNone(exc) (Py_INCREF(exc), __Pyx_ErrRestore((exc), NULL, NULL)) #else #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #endif #else #define __Pyx_PyErr_Clear() PyErr_Clear() #define __Pyx_PyErr_SetNone(exc) PyErr_SetNone(exc) #define __Pyx_ErrRestoreWithState(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchWithState(type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestoreInState(tstate, type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetchInState(tstate, type, value, tb) PyErr_Fetch(type, value, tb) #define __Pyx_ErrRestore(type, value, tb) PyErr_Restore(type, value, tb) #define __Pyx_ErrFetch(type, value, tb) PyErr_Fetch(type, value, tb) #endif /* GetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_GetException(type, value, tb) __Pyx__GetException(__pyx_tstate, type, value, tb) static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb); #endif /* SwapException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSwap(type, value, tb) __Pyx__ExceptionSwap(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #else static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb); #endif /* GetTopmostException.proto */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate); #endif /* SaveResetException.proto */ #if CYTHON_FAST_THREAD_STATE #define __Pyx_ExceptionSave(type, value, tb) __Pyx__ExceptionSave(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb); #define __Pyx_ExceptionReset(type, value, tb) __Pyx__ExceptionReset(__pyx_tstate, type, value, tb) static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb); #else #define __Pyx_ExceptionSave(type, value, tb) PyErr_GetExcInfo(type, value, tb) #define __Pyx_ExceptionReset(type, value, tb) PyErr_SetExcInfo(type, value, tb) #endif /* Import.proto */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level); /* ImportFrom.proto */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name); /* CLineInTraceback.proto */ #ifdef CYTHON_CLINE_IN_TRACEBACK #define __Pyx_CLineForTraceback(tstate, c_line) (((CYTHON_CLINE_IN_TRACEBACK)) ? c_line : 0) #else static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line); #endif /* CodeObjectCache.proto */ typedef struct { PyCodeObject* code_object; int code_line; } __Pyx_CodeObjectCacheEntry; struct __Pyx_CodeObjectCache { int count; int max_count; __Pyx_CodeObjectCacheEntry* entries; }; static struct __Pyx_CodeObjectCache __pyx_code_cache = {0,0,NULL}; static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line); static PyCodeObject *__pyx_find_code_object(int code_line); static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object); /* AddTraceback.proto */ static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename); /* CIntToPy.proto */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value); /* CIntFromPy.proto */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *); /* CIntFromPy.proto */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *); /* FastTypeChecks.proto */ #if CYTHON_COMPILING_IN_CPYTHON #define __Pyx_TypeCheck(obj, type) __Pyx_IsSubtype(Py_TYPE(obj), (PyTypeObject *)type) static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches(PyObject *err, PyObject *type); static CYTHON_INLINE int __Pyx_PyErr_GivenExceptionMatches2(PyObject *err, PyObject *type1, PyObject *type2); #else #define __Pyx_TypeCheck(obj, type) PyObject_TypeCheck(obj, (PyTypeObject *)type) #define __Pyx_PyErr_GivenExceptionMatches(err, type) PyErr_GivenExceptionMatches(err, type) #define __Pyx_PyErr_GivenExceptionMatches2(err, type1, type2) (PyErr_GivenExceptionMatches(err, type1) || PyErr_GivenExceptionMatches(err, type2)) #endif #define __Pyx_PyException_Check(obj) __Pyx_TypeCheck(obj, PyExc_Exception) /* CheckBinaryVersion.proto */ static int __Pyx_check_binary_version(void); /* InitStrings.proto */ static int __Pyx_InitStrings(__Pyx_StringTabEntry *t); /* Module declarations from 'borg.platform_linux' */ static PyObject *__pyx_f_4borg_14platform_linux_acl_append_numeric_ids(PyObject *); /*proto*/ static PyObject *__pyx_f_4borg_14platform_linux_acl_numeric_ids(PyObject *); /*proto*/ #define __Pyx_MODULE_NAME "borg.platform_linux" extern int __pyx_module_is_main_borg__platform_linux; int __pyx_module_is_main_borg__platform_linux = 0; /* Implementation of 'borg.platform_linux' */ static const char __pyx_k_[] = "\n"; static const char __pyx_k_M[] = "M"; static const char __pyx_k_p[] = "p"; static const char __pyx_k_u[] = "-u"; static const char __pyx_k__2[] = ":"; static const char __pyx_k__4[] = ""; static const char __pyx_k__5[] = " *#.*"; static const char __pyx_k_os[] = "os"; static const char __pyx_k_re[] = "re"; static const char __pyx_k_st[] = "st"; static const char __pyx_k_acl[] = "acl"; static const char __pyx_k_env[] = "env"; static const char __pyx_k_get[] = "get"; static const char __pyx_k_sub[] = "sub"; static const char __pyx_k_call[] = "call"; static const char __pyx_k_item[] = "item"; static const char __pyx_k_join[] = "join"; static const char __pyx_k_main[] = "__main__"; static const char __pyx_k_name[] = "__name__"; static const char __pyx_k_path[] = "path"; static const char __pyx_k_stat[] = "stat"; static const char __pyx_k_test[] = "__test__"; static const char __pyx_k_user[] = "user"; static const char __pyx_k_entry[] = "entry"; static const char __pyx_k_group[] = "group"; static const char __pyx_k_split[] = "split"; static const char __pyx_k_1_0_01[] = "1.0_01"; static const char __pyx_k_fields[] = "fields"; static const char __pyx_k_import[] = "__import__"; static const char __pyx_k_system[] = "system"; static const char __pyx_k_umount[] = "umount"; static const char __pyx_k_S_ISLNK[] = "S_ISLNK"; static const char __pyx_k_acl_get[] = "acl_get"; static const char __pyx_k_acl_set[] = "acl_set"; static const char __pyx_k_compile[] = "compile"; static const char __pyx_k_entries[] = "entries"; static const char __pyx_k_helpers[] = "helpers"; static const char __pyx_k_st_mode[] = "st_mode"; static const char __pyx_k_fsencode[] = "fsencode"; static const char __pyx_k_user2uid[] = "user2uid"; static const char __pyx_k_converter[] = "converter"; static const char __pyx_k_group2gid[] = "group2gid"; static const char __pyx_k_access_acl[] = "access_acl"; static const char __pyx_k_acl_access[] = "acl_access"; static const char __pyx_k_comment_re[] = "_comment_re"; static const char __pyx_k_fusermount[] = "fusermount"; static const char __pyx_k_mountpoint[] = "mountpoint"; static const char __pyx_k_subprocess[] = "subprocess"; static const char __pyx_k_API_VERSION[] = "API_VERSION"; static const char __pyx_k_access_text[] = "access_text"; static const char __pyx_k_acl_default[] = "acl_default"; static const char __pyx_k_default_acl[] = "default_acl"; static const char __pyx_k_safe_decode[] = "safe_decode"; static const char __pyx_k_safe_encode[] = "safe_encode"; static const char __pyx_k_default_text[] = "default_text"; static const char __pyx_k_numeric_owner[] = "numeric_owner"; static const char __pyx_k_cline_in_traceback[] = "cline_in_traceback"; static const char __pyx_k_borg_platform_linux[] = "borg.platform_linux"; static const char __pyx_k_acl_use_local_uid_gid[] = "acl_use_local_uid_gid"; static const char __pyx_k_prepare_subprocess_env[] = "prepare_subprocess_env"; static const char __pyx_k_borg_platform_linux_pyx[] = "borg/platform_linux.pyx"; static const char __pyx_k_posix_acl_use_stored_uid_gid[] = "posix_acl_use_stored_uid_gid"; static PyObject *__pyx_kp_s_; static PyObject *__pyx_kp_s_1_0_01; static PyObject *__pyx_n_s_API_VERSION; static PyObject *__pyx_n_s_M; static PyObject *__pyx_n_s_S_ISLNK; static PyObject *__pyx_kp_s__2; static PyObject *__pyx_kp_s__4; static PyObject *__pyx_kp_s__5; static PyObject *__pyx_n_s_access_acl; static PyObject *__pyx_n_s_access_text; static PyObject *__pyx_n_s_acl; static PyObject *__pyx_n_b_acl_access; static PyObject *__pyx_n_b_acl_default; static PyObject *__pyx_n_s_acl_get; static PyObject *__pyx_n_s_acl_set; static PyObject *__pyx_n_s_acl_use_local_uid_gid; static PyObject *__pyx_n_s_borg_platform_linux; static PyObject *__pyx_kp_s_borg_platform_linux_pyx; static PyObject *__pyx_n_s_call; static PyObject *__pyx_n_s_cline_in_traceback; static PyObject *__pyx_n_s_comment_re; static PyObject *__pyx_n_s_compile; static PyObject *__pyx_n_s_converter; static PyObject *__pyx_n_s_default_acl; static PyObject *__pyx_n_s_default_text; static PyObject *__pyx_n_s_entries; static PyObject *__pyx_n_s_entry; static PyObject *__pyx_n_s_env; static PyObject *__pyx_n_s_fields; static PyObject *__pyx_n_s_fsencode; static PyObject *__pyx_n_s_fusermount; static PyObject *__pyx_n_s_get; static PyObject *__pyx_n_s_group; static PyObject *__pyx_n_s_group2gid; static PyObject *__pyx_n_s_helpers; static PyObject *__pyx_n_s_import; static PyObject *__pyx_n_s_item; static PyObject *__pyx_n_s_join; static PyObject *__pyx_n_s_main; static PyObject *__pyx_n_s_mountpoint; static PyObject *__pyx_n_s_name; static PyObject *__pyx_n_s_numeric_owner; static PyObject *__pyx_n_s_os; static PyObject *__pyx_n_s_p; static PyObject *__pyx_n_s_path; static PyObject *__pyx_n_s_posix_acl_use_stored_uid_gid; static PyObject *__pyx_n_s_prepare_subprocess_env; static PyObject *__pyx_n_s_re; static PyObject *__pyx_n_s_safe_decode; static PyObject *__pyx_n_s_safe_encode; static PyObject *__pyx_n_s_split; static PyObject *__pyx_n_s_st; static PyObject *__pyx_n_s_st_mode; static PyObject *__pyx_n_s_stat; static PyObject *__pyx_n_s_sub; static PyObject *__pyx_n_s_subprocess; static PyObject *__pyx_n_s_system; static PyObject *__pyx_n_s_test; static PyObject *__pyx_kp_s_u; static PyObject *__pyx_n_s_umount; static PyObject *__pyx_n_s_user; static PyObject *__pyx_n_s_user2uid; static PyObject *__pyx_pf_4borg_14platform_linux_acl_use_local_uid_gid(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_acl); /* proto */ static PyObject *__pyx_pf_4borg_14platform_linux_2acl_get(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_st, PyObject *__pyx_v_numeric_owner); /* proto */ static PyObject *__pyx_pf_4borg_14platform_linux_4acl_set(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_numeric_owner); /* proto */ static PyObject *__pyx_pf_4borg_14platform_linux_6umount(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_mountpoint); /* proto */ static PyObject *__pyx_int_3; static PyObject *__pyx_slice__3; static PyObject *__pyx_tuple__6; static PyObject *__pyx_tuple__8; static PyObject *__pyx_tuple__10; static PyObject *__pyx_tuple__12; static PyObject *__pyx_codeobj__7; static PyObject *__pyx_codeobj__9; static PyObject *__pyx_codeobj__11; static PyObject *__pyx_codeobj__13; /* Late includes */ /* "borg/platform_linux.pyx":31 * * * def acl_use_local_uid_gid(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_14platform_linux_1acl_use_local_uid_gid(PyObject *__pyx_self, PyObject *__pyx_v_acl); /*proto*/ static char __pyx_doc_4borg_14platform_linux_acl_use_local_uid_gid[] = "Replace the user/group field with the local uid/gid if possible\n "; static PyMethodDef __pyx_mdef_4borg_14platform_linux_1acl_use_local_uid_gid = {"acl_use_local_uid_gid", (PyCFunction)__pyx_pw_4borg_14platform_linux_1acl_use_local_uid_gid, METH_O, __pyx_doc_4borg_14platform_linux_acl_use_local_uid_gid}; static PyObject *__pyx_pw_4borg_14platform_linux_1acl_use_local_uid_gid(PyObject *__pyx_self, PyObject *__pyx_v_acl) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_use_local_uid_gid (wrapper)", 0); __pyx_r = __pyx_pf_4borg_14platform_linux_acl_use_local_uid_gid(__pyx_self, ((PyObject *)__pyx_v_acl)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_14platform_linux_acl_use_local_uid_gid(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_acl) { PyObject *__pyx_v_entries = NULL; PyObject *__pyx_v_entry = NULL; PyObject *__pyx_v_fields = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; Py_ssize_t __pyx_t_5; PyObject *(*__pyx_t_6)(PyObject *); int __pyx_t_7; int __pyx_t_8; PyObject *__pyx_t_9 = NULL; PyObject *__pyx_t_10 = NULL; int __pyx_t_11; PyObject *__pyx_t_12 = NULL; int __pyx_t_13; __Pyx_RefNannySetupContext("acl_use_local_uid_gid", 0); /* "borg/platform_linux.pyx":34 * """Replace the user/group field with the local uid/gid if possible * """ * entries = [] # <<<<<<<<<<<<<< * for entry in safe_decode(acl).split('\n'): * if entry: */ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 34, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_entries = ((PyObject*)__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":35 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * fields = entry.split(':') */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_2 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_4, __pyx_v_acl) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_acl); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_split); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_kp_s_) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_kp_s_); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (likely(PyList_CheckExact(__pyx_t_1)) || PyTuple_CheckExact(__pyx_t_1)) { __pyx_t_3 = __pyx_t_1; __Pyx_INCREF(__pyx_t_3); __pyx_t_5 = 0; __pyx_t_6 = NULL; } else { __pyx_t_5 = -1; __pyx_t_3 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = Py_TYPE(__pyx_t_3)->tp_iternext; if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 35, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; for (;;) { if (likely(!__pyx_t_6)) { if (likely(PyList_CheckExact(__pyx_t_3))) { if (__pyx_t_5 >= PyList_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyList_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 35, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } else { if (__pyx_t_5 >= PyTuple_GET_SIZE(__pyx_t_3)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyTuple_GET_ITEM(__pyx_t_3, __pyx_t_5); __Pyx_INCREF(__pyx_t_1); __pyx_t_5++; if (unlikely(0 < 0)) __PYX_ERR(0, 35, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_3, __pyx_t_5); __pyx_t_5++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 35, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } } else { __pyx_t_1 = __pyx_t_6(__pyx_t_3); if (unlikely(!__pyx_t_1)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 35, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_1); } __Pyx_XDECREF_SET(__pyx_v_entry, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":36 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * fields = entry.split(':') * if fields[0] == 'user' and fields[1]: */ __pyx_t_7 = __Pyx_PyObject_IsTrue(__pyx_v_entry); if (unlikely(__pyx_t_7 < 0)) __PYX_ERR(0, 36, __pyx_L1_error) if (__pyx_t_7) { /* "borg/platform_linux.pyx":37 * for entry in safe_decode(acl).split('\n'): * if entry: * fields = entry.split(':') # <<<<<<<<<<<<<< * if fields[0] == 'user' and fields[1]: * fields[1] = str(user2uid(fields[1], fields[3])) */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_split); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 37, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_4) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_4, __pyx_kp_s__2) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s__2); __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 37, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF_SET(__pyx_v_fields, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":38 * if entry: * fields = entry.split(':') * if fields[0] == 'user' and fields[1]: # <<<<<<<<<<<<<< * fields[1] = str(user2uid(fields[1], fields[3])) * elif fields[0] == 'group' and fields[1]: */ __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_fields, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 38, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_8 = (__Pyx_PyString_Equals(__pyx_t_1, __pyx_n_s_user, Py_EQ)); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 38, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (__pyx_t_8) { } else { __pyx_t_7 = __pyx_t_8; goto __pyx_L7_bool_binop_done; } __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_fields, 1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 38, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_1); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 38, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_7 = __pyx_t_8; __pyx_L7_bool_binop_done:; if (__pyx_t_7) { /* "borg/platform_linux.pyx":39 * fields = entry.split(':') * if fields[0] == 'user' and fields[1]: * fields[1] = str(user2uid(fields[1], fields[3])) # <<<<<<<<<<<<<< * elif fields[0] == 'group' and fields[1]: * fields[1] = str(group2gid(fields[1], fields[3])) */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_user2uid); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_4 = __Pyx_GetItemInt(__pyx_v_fields, 1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_9 = __Pyx_GetItemInt(__pyx_v_fields, 3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __pyx_t_10 = NULL; __pyx_t_11 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_10 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_10)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_10); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); __pyx_t_11 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_2)) { PyObject *__pyx_temp[3] = {__pyx_t_10, __pyx_t_4, __pyx_t_9}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_2, __pyx_temp+1-__pyx_t_11, 2+__pyx_t_11); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_10); __pyx_t_10 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_2)) { PyObject *__pyx_temp[3] = {__pyx_t_10, __pyx_t_4, __pyx_t_9}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_2, __pyx_temp+1-__pyx_t_11, 2+__pyx_t_11); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_10); __pyx_t_10 = 0; __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; } else #endif { __pyx_t_12 = PyTuple_New(2+__pyx_t_11); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); if (__pyx_t_10) { __Pyx_GIVEREF(__pyx_t_10); PyTuple_SET_ITEM(__pyx_t_12, 0, __pyx_t_10); __pyx_t_10 = NULL; } __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_12, 0+__pyx_t_11, __pyx_t_4); __Pyx_GIVEREF(__pyx_t_9); PyTuple_SET_ITEM(__pyx_t_12, 1+__pyx_t_11, __pyx_t_9); __pyx_t_4 = 0; __pyx_t_9 = 0; __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_t_12, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; } __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 1, __pyx_t_2, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 39, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_linux.pyx":38 * if entry: * fields = entry.split(':') * if fields[0] == 'user' and fields[1]: # <<<<<<<<<<<<<< * fields[1] = str(user2uid(fields[1], fields[3])) * elif fields[0] == 'group' and fields[1]: */ goto __pyx_L6; } /* "borg/platform_linux.pyx":40 * if fields[0] == 'user' and fields[1]: * fields[1] = str(user2uid(fields[1], fields[3])) * elif fields[0] == 'group' and fields[1]: # <<<<<<<<<<<<<< * fields[1] = str(group2gid(fields[1], fields[3])) * entries.append(':'.join(fields[:3])) */ __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_fields, 0, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_8 = (__Pyx_PyString_Equals(__pyx_t_2, __pyx_n_s_group, Py_EQ)); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (__pyx_t_8) { } else { __pyx_t_7 = __pyx_t_8; goto __pyx_L9_bool_binop_done; } __pyx_t_2 = __Pyx_GetItemInt(__pyx_v_fields, 1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_8 = __Pyx_PyObject_IsTrue(__pyx_t_2); if (unlikely(__pyx_t_8 < 0)) __PYX_ERR(0, 40, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_7 = __pyx_t_8; __pyx_L9_bool_binop_done:; if (__pyx_t_7) { /* "borg/platform_linux.pyx":41 * fields[1] = str(user2uid(fields[1], fields[3])) * elif fields[0] == 'group' and fields[1]: * fields[1] = str(group2gid(fields[1], fields[3])) # <<<<<<<<<<<<<< * entries.append(':'.join(fields[:3])) * return safe_encode('\n'.join(entries)) */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_group2gid); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_12 = __Pyx_GetItemInt(__pyx_v_fields, 1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_12)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_12); __pyx_t_9 = __Pyx_GetItemInt(__pyx_v_fields, 3, long, 1, __Pyx_PyInt_From_long, 0, 0, 1); if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_9); __pyx_t_4 = NULL; __pyx_t_11 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_4 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_4)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_4); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); __pyx_t_11 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_1)) { PyObject *__pyx_temp[3] = {__pyx_t_4, __pyx_t_12, __pyx_t_9}; __pyx_t_2 = __Pyx_PyFunction_FastCall(__pyx_t_1, __pyx_temp+1-__pyx_t_11, 2+__pyx_t_11); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_1)) { PyObject *__pyx_temp[3] = {__pyx_t_4, __pyx_t_12, __pyx_t_9}; __pyx_t_2 = __Pyx_PyCFunction_FastCall(__pyx_t_1, __pyx_temp+1-__pyx_t_11, 2+__pyx_t_11); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_4); __pyx_t_4 = 0; __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_12); __pyx_t_12 = 0; __Pyx_DECREF(__pyx_t_9); __pyx_t_9 = 0; } else #endif { __pyx_t_10 = PyTuple_New(2+__pyx_t_11); if (unlikely(!__pyx_t_10)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_10); if (__pyx_t_4) { __Pyx_GIVEREF(__pyx_t_4); PyTuple_SET_ITEM(__pyx_t_10, 0, __pyx_t_4); __pyx_t_4 = NULL; } __Pyx_GIVEREF(__pyx_t_12); PyTuple_SET_ITEM(__pyx_t_10, 0+__pyx_t_11, __pyx_t_12); __Pyx_GIVEREF(__pyx_t_9); PyTuple_SET_ITEM(__pyx_t_10, 1+__pyx_t_11, __pyx_t_9); __pyx_t_12 = 0; __pyx_t_9 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_t_10, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_10); __pyx_t_10 = 0; } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_t_2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(__Pyx_SetItemInt(__pyx_v_fields, 1, __pyx_t_1, long, 1, __Pyx_PyInt_From_long, 0, 0, 1) < 0)) __PYX_ERR(0, 41, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":40 * if fields[0] == 'user' and fields[1]: * fields[1] = str(user2uid(fields[1], fields[3])) * elif fields[0] == 'group' and fields[1]: # <<<<<<<<<<<<<< * fields[1] = str(group2gid(fields[1], fields[3])) * entries.append(':'.join(fields[:3])) */ } __pyx_L6:; /* "borg/platform_linux.pyx":42 * elif fields[0] == 'group' and fields[1]: * fields[1] = str(group2gid(fields[1], fields[3])) * entries.append(':'.join(fields[:3])) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ __pyx_t_1 = __Pyx_PyObject_GetSlice(__pyx_v_fields, 0, 3, NULL, NULL, &__pyx_slice__3, 0, 1, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_t_1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_2); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_linux.pyx":36 * entries = [] * for entry in safe_decode(acl).split('\n'): * if entry: # <<<<<<<<<<<<<< * fields = entry.split(':') * if fields[0] == 'user' and fields[1]: */ } /* "borg/platform_linux.pyx":35 * """ * entries = [] * for entry in safe_decode(acl).split('\n'): # <<<<<<<<<<<<<< * if entry: * fields = entry.split(':') */ } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":43 * fields[1] = str(group2gid(fields[1], fields[3])) * entries.append(':'.join(fields[:3])) * return safe_encode('\n'.join(entries)) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 43, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_1 = __Pyx_PyString_Join(__pyx_kp_s_, __pyx_v_entries); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 43, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_10 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_2))) { __pyx_t_10 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_10)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_10); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_3 = (__pyx_t_10) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_10, __pyx_t_1) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_t_1); __Pyx_XDECREF(__pyx_t_10); __pyx_t_10 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 43, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_r = __pyx_t_3; __pyx_t_3 = 0; goto __pyx_L0; /* "borg/platform_linux.pyx":31 * * * def acl_use_local_uid_gid(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_9); __Pyx_XDECREF(__pyx_t_10); __Pyx_XDECREF(__pyx_t_12); __Pyx_AddTraceback("borg.platform_linux.acl_use_local_uid_gid", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_entries); __Pyx_XDECREF(__pyx_v_entry); __Pyx_XDECREF(__pyx_v_fields); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_linux.pyx":46 * * * cdef acl_append_numeric_ids(acl): # <<<<<<<<<<<<<< * """Extend the "POSIX 1003.1e draft standard 17" format with an additional uid/gid field * """ */ static PyObject *__pyx_f_4borg_14platform_linux_acl_append_numeric_ids(PyObject *__pyx_v_acl) { PyObject *__pyx_v_entries = NULL; PyObject *__pyx_v_entry = NULL; PyObject *__pyx_v_type = NULL; PyObject *__pyx_v_name = NULL; PyObject *__pyx_v_permission = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; int __pyx_t_7; Py_ssize_t __pyx_t_8; PyObject *(*__pyx_t_9)(PyObject *); int __pyx_t_10; PyObject *(*__pyx_t_11)(PyObject *); int __pyx_t_12; int __pyx_t_13; __Pyx_RefNannySetupContext("acl_append_numeric_ids", 0); /* "borg/platform_linux.pyx":49 * """Extend the "POSIX 1003.1e draft standard 17" format with an additional uid/gid field * """ * entries = [] # <<<<<<<<<<<<<< * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: */ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 49, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_entries = ((PyObject*)__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":50 * """ * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): # <<<<<<<<<<<<<< * if entry: * type, name, permission = entry.split(':') */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_comment_re); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyObject_GetAttrStr(__pyx_t_3, __pyx_n_s_sub); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_5))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_3 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_6, __pyx_v_acl) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_acl); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = NULL; __pyx_t_7 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_4))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_4); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_4); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_4, function); __pyx_t_7 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_4)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_kp_s__4, __pyx_t_3}; __pyx_t_2 = __Pyx_PyFunction_FastCall(__pyx_t_4, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_4)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_kp_s__4, __pyx_t_3}; __pyx_t_2 = __Pyx_PyCFunction_FastCall(__pyx_t_4, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; } else #endif { __pyx_t_6 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (__pyx_t_5) { __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_5); __pyx_t_5 = NULL; } __Pyx_INCREF(__pyx_kp_s__4); __Pyx_GIVEREF(__pyx_kp_s__4); PyTuple_SET_ITEM(__pyx_t_6, 0+__pyx_t_7, __pyx_kp_s__4); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_6, 1+__pyx_t_7, __pyx_t_3); __pyx_t_3 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_4, __pyx_t_6, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_4 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_split); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_4))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_4); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_4); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_4, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_4, __pyx_t_2, __pyx_kp_s_) : __Pyx_PyObject_CallOneArg(__pyx_t_4, __pyx_kp_s_); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (likely(PyList_CheckExact(__pyx_t_1)) || PyTuple_CheckExact(__pyx_t_1)) { __pyx_t_4 = __pyx_t_1; __Pyx_INCREF(__pyx_t_4); __pyx_t_8 = 0; __pyx_t_9 = NULL; } else { __pyx_t_8 = -1; __pyx_t_4 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_9 = Py_TYPE(__pyx_t_4)->tp_iternext; if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 50, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; for (;;) { if (likely(!__pyx_t_9)) { if (likely(PyList_CheckExact(__pyx_t_4))) { if (__pyx_t_8 >= PyList_GET_SIZE(__pyx_t_4)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyList_GET_ITEM(__pyx_t_4, __pyx_t_8); __Pyx_INCREF(__pyx_t_1); __pyx_t_8++; if (unlikely(0 < 0)) __PYX_ERR(0, 50, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_4, __pyx_t_8); __pyx_t_8++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } else { if (__pyx_t_8 >= PyTuple_GET_SIZE(__pyx_t_4)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyTuple_GET_ITEM(__pyx_t_4, __pyx_t_8); __Pyx_INCREF(__pyx_t_1); __pyx_t_8++; if (unlikely(0 < 0)) __PYX_ERR(0, 50, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_4, __pyx_t_8); __pyx_t_8++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 50, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } } else { __pyx_t_1 = __pyx_t_9(__pyx_t_4); if (unlikely(!__pyx_t_1)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 50, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_1); } __Pyx_XDECREF_SET(__pyx_v_entry, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":51 * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: # <<<<<<<<<<<<<< * type, name, permission = entry.split(':') * if name and type == 'user': */ __pyx_t_10 = __Pyx_PyObject_IsTrue(__pyx_v_entry); if (unlikely(__pyx_t_10 < 0)) __PYX_ERR(0, 51, __pyx_L1_error) if (__pyx_t_10) { /* "borg/platform_linux.pyx":52 * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: * type, name, permission = entry.split(':') # <<<<<<<<<<<<<< * if name and type == 'user': * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_split); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 52, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_6, __pyx_kp_s__2) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s__2); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 52, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if ((likely(PyTuple_CheckExact(__pyx_t_1))) || (PyList_CheckExact(__pyx_t_1))) { PyObject* sequence = __pyx_t_1; Py_ssize_t size = __Pyx_PySequence_SIZE(sequence); if (unlikely(size != 3)) { if (size > 3) __Pyx_RaiseTooManyValuesError(3); else if (size >= 0) __Pyx_RaiseNeedMoreValuesError(size); __PYX_ERR(0, 52, __pyx_L1_error) } #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS if (likely(PyTuple_CheckExact(sequence))) { __pyx_t_2 = PyTuple_GET_ITEM(sequence, 0); __pyx_t_6 = PyTuple_GET_ITEM(sequence, 1); __pyx_t_3 = PyTuple_GET_ITEM(sequence, 2); } else { __pyx_t_2 = PyList_GET_ITEM(sequence, 0); __pyx_t_6 = PyList_GET_ITEM(sequence, 1); __pyx_t_3 = PyList_GET_ITEM(sequence, 2); } __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(__pyx_t_3); #else __pyx_t_2 = PySequence_ITEM(sequence, 0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 52, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_6 = PySequence_ITEM(sequence, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 52, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_3 = PySequence_ITEM(sequence, 2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 52, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); #endif __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; } else { Py_ssize_t index = -1; __pyx_t_5 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 52, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_11 = Py_TYPE(__pyx_t_5)->tp_iternext; index = 0; __pyx_t_2 = __pyx_t_11(__pyx_t_5); if (unlikely(!__pyx_t_2)) goto __pyx_L6_unpacking_failed; __Pyx_GOTREF(__pyx_t_2); index = 1; __pyx_t_6 = __pyx_t_11(__pyx_t_5); if (unlikely(!__pyx_t_6)) goto __pyx_L6_unpacking_failed; __Pyx_GOTREF(__pyx_t_6); index = 2; __pyx_t_3 = __pyx_t_11(__pyx_t_5); if (unlikely(!__pyx_t_3)) goto __pyx_L6_unpacking_failed; __Pyx_GOTREF(__pyx_t_3); if (__Pyx_IternextUnpackEndCheck(__pyx_t_11(__pyx_t_5), 3) < 0) __PYX_ERR(0, 52, __pyx_L1_error) __pyx_t_11 = NULL; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; goto __pyx_L7_unpacking_done; __pyx_L6_unpacking_failed:; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_11 = NULL; if (__Pyx_IterFinish() == 0) __Pyx_RaiseNeedMoreValuesError(index); __PYX_ERR(0, 52, __pyx_L1_error) __pyx_L7_unpacking_done:; } __Pyx_XDECREF_SET(__pyx_v_type, __pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF_SET(__pyx_v_name, __pyx_t_6); __pyx_t_6 = 0; __Pyx_XDECREF_SET(__pyx_v_permission, __pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":53 * if entry: * type, name, permission = entry.split(':') * if name and type == 'user': # <<<<<<<<<<<<<< * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) * elif name and type == 'group': */ __pyx_t_12 = __Pyx_PyObject_IsTrue(__pyx_v_name); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 53, __pyx_L1_error) if (__pyx_t_12) { } else { __pyx_t_10 = __pyx_t_12; goto __pyx_L9_bool_binop_done; } __pyx_t_12 = (__Pyx_PyString_Equals(__pyx_v_type, __pyx_n_s_user, Py_EQ)); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 53, __pyx_L1_error) __pyx_t_10 = __pyx_t_12; __pyx_L9_bool_binop_done:; if (__pyx_t_10) { /* "borg/platform_linux.pyx":54 * type, name, permission = entry.split(':') * if name and type == 'user': * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) # <<<<<<<<<<<<<< * elif name and type == 'group': * entries.append(':'.join([type, name, permission, str(group2gid(name, name))])) */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_user2uid); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = NULL; __pyx_t_7 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_7 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_6, __pyx_v_name, __pyx_v_name}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_6, __pyx_v_name, __pyx_v_name}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif { __pyx_t_2 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (__pyx_t_6) { __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_6); __pyx_t_6 = NULL; } __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_2, 0+__pyx_t_7, __pyx_v_name); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_2, 1+__pyx_t_7, __pyx_v_name); __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = PyList_New(4); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_v_type); __Pyx_GIVEREF(__pyx_v_type); PyList_SET_ITEM(__pyx_t_1, 0, __pyx_v_type); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyList_SET_ITEM(__pyx_t_1, 1, __pyx_v_name); __Pyx_INCREF(__pyx_v_permission); __Pyx_GIVEREF(__pyx_v_permission); PyList_SET_ITEM(__pyx_t_1, 2, __pyx_v_permission); __Pyx_GIVEREF(__pyx_t_3); PyList_SET_ITEM(__pyx_t_1, 3, __pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_3); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 54, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":53 * if entry: * type, name, permission = entry.split(':') * if name and type == 'user': # <<<<<<<<<<<<<< * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) * elif name and type == 'group': */ goto __pyx_L8; } /* "borg/platform_linux.pyx":55 * if name and type == 'user': * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) * elif name and type == 'group': # <<<<<<<<<<<<<< * entries.append(':'.join([type, name, permission, str(group2gid(name, name))])) * else: */ __pyx_t_12 = __Pyx_PyObject_IsTrue(__pyx_v_name); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 55, __pyx_L1_error) if (__pyx_t_12) { } else { __pyx_t_10 = __pyx_t_12; goto __pyx_L11_bool_binop_done; } __pyx_t_12 = (__Pyx_PyString_Equals(__pyx_v_type, __pyx_n_s_group, Py_EQ)); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 55, __pyx_L1_error) __pyx_t_10 = __pyx_t_12; __pyx_L11_bool_binop_done:; if (__pyx_t_10) { /* "borg/platform_linux.pyx":56 * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) * elif name and type == 'group': * entries.append(':'.join([type, name, permission, str(group2gid(name, name))])) # <<<<<<<<<<<<<< * else: * entries.append(entry) */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_group2gid); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = NULL; __pyx_t_7 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); __pyx_t_7 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_1)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_name, __pyx_v_name}; __pyx_t_3 = __Pyx_PyFunction_FastCall(__pyx_t_1, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_3); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_1)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_name, __pyx_v_name}; __pyx_t_3 = __Pyx_PyCFunction_FastCall(__pyx_t_1, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_3); } else #endif { __pyx_t_6 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (__pyx_t_2) { __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_2); __pyx_t_2 = NULL; } __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_6, 0+__pyx_t_7, __pyx_v_name); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_6, 1+__pyx_t_7, __pyx_v_name); __pyx_t_3 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_t_6, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = PyList_New(4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_INCREF(__pyx_v_type); __Pyx_GIVEREF(__pyx_v_type); PyList_SET_ITEM(__pyx_t_3, 0, __pyx_v_type); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyList_SET_ITEM(__pyx_t_3, 1, __pyx_v_name); __Pyx_INCREF(__pyx_v_permission); __Pyx_GIVEREF(__pyx_v_permission); PyList_SET_ITEM(__pyx_t_3, 2, __pyx_v_permission); __Pyx_GIVEREF(__pyx_t_1); PyList_SET_ITEM(__pyx_t_3, 3, __pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_1); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 56, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":55 * if name and type == 'user': * entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) * elif name and type == 'group': # <<<<<<<<<<<<<< * entries.append(':'.join([type, name, permission, str(group2gid(name, name))])) * else: */ goto __pyx_L8; } /* "borg/platform_linux.pyx":58 * entries.append(':'.join([type, name, permission, str(group2gid(name, name))])) * else: * entries.append(entry) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ /*else*/ { __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_v_entry); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 58, __pyx_L1_error) } __pyx_L8:; /* "borg/platform_linux.pyx":51 * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: # <<<<<<<<<<<<<< * type, name, permission = entry.split(':') * if name and type == 'user': */ } /* "borg/platform_linux.pyx":50 * """ * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): # <<<<<<<<<<<<<< * if entry: * type, name, permission = entry.split(':') */ } __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/platform_linux.pyx":59 * else: * entries.append(entry) * return safe_encode('\n'.join(entries)) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __Pyx_PyString_Join(__pyx_kp_s_, __pyx_v_entries); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_4 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_6, __pyx_t_3) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_t_3); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 59, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L0; /* "borg/platform_linux.pyx":46 * * * cdef acl_append_numeric_ids(acl): # <<<<<<<<<<<<<< * """Extend the "POSIX 1003.1e draft standard 17" format with an additional uid/gid field * """ */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.platform_linux.acl_append_numeric_ids", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XDECREF(__pyx_v_entries); __Pyx_XDECREF(__pyx_v_entry); __Pyx_XDECREF(__pyx_v_type); __Pyx_XDECREF(__pyx_v_name); __Pyx_XDECREF(__pyx_v_permission); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_linux.pyx":62 * * * cdef acl_numeric_ids(acl): # <<<<<<<<<<<<<< * """Replace the "POSIX 1003.1e draft standard 17" user/group field with uid/gid * """ */ static PyObject *__pyx_f_4borg_14platform_linux_acl_numeric_ids(PyObject *__pyx_v_acl) { PyObject *__pyx_v_entries = NULL; PyObject *__pyx_v_entry = NULL; PyObject *__pyx_v_type = NULL; PyObject *__pyx_v_name = NULL; PyObject *__pyx_v_permission = NULL; PyObject *__pyx_v_uid = NULL; PyObject *__pyx_v_gid = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; PyObject *__pyx_t_5 = NULL; PyObject *__pyx_t_6 = NULL; int __pyx_t_7; Py_ssize_t __pyx_t_8; PyObject *(*__pyx_t_9)(PyObject *); int __pyx_t_10; PyObject *(*__pyx_t_11)(PyObject *); int __pyx_t_12; int __pyx_t_13; __Pyx_RefNannySetupContext("acl_numeric_ids", 0); /* "borg/platform_linux.pyx":65 * """Replace the "POSIX 1003.1e draft standard 17" user/group field with uid/gid * """ * entries = [] # <<<<<<<<<<<<<< * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: */ __pyx_t_1 = PyList_New(0); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 65, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_v_entries = ((PyObject*)__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":66 * """ * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): # <<<<<<<<<<<<<< * if entry: * type, name, permission = entry.split(':') */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_comment_re); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_4 = __Pyx_PyObject_GetAttrStr(__pyx_t_3, __pyx_n_s_sub); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __Pyx_GetModuleGlobalName(__pyx_t_5, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_5))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_5); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_5); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_5, function); } } __pyx_t_3 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_5, __pyx_t_6, __pyx_v_acl) : __Pyx_PyObject_CallOneArg(__pyx_t_5, __pyx_v_acl); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_5 = NULL; __pyx_t_7 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_4))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_4); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_4); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_4, function); __pyx_t_7 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_4)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_kp_s__4, __pyx_t_3}; __pyx_t_2 = __Pyx_PyFunction_FastCall(__pyx_t_4, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_4)) { PyObject *__pyx_temp[3] = {__pyx_t_5, __pyx_kp_s__4, __pyx_t_3}; __pyx_t_2 = __Pyx_PyCFunction_FastCall(__pyx_t_4, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; } else #endif { __pyx_t_6 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (__pyx_t_5) { __Pyx_GIVEREF(__pyx_t_5); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_5); __pyx_t_5 = NULL; } __Pyx_INCREF(__pyx_kp_s__4); __Pyx_GIVEREF(__pyx_kp_s__4); PyTuple_SET_ITEM(__pyx_t_6, 0+__pyx_t_7, __pyx_kp_s__4); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_6, 1+__pyx_t_7, __pyx_t_3); __pyx_t_3 = 0; __pyx_t_2 = __Pyx_PyObject_Call(__pyx_t_4, __pyx_t_6, NULL); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; __pyx_t_4 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_split); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_4))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_4); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_4); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_4, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_4, __pyx_t_2, __pyx_kp_s_) : __Pyx_PyObject_CallOneArg(__pyx_t_4, __pyx_kp_s_); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; if (likely(PyList_CheckExact(__pyx_t_1)) || PyTuple_CheckExact(__pyx_t_1)) { __pyx_t_4 = __pyx_t_1; __Pyx_INCREF(__pyx_t_4); __pyx_t_8 = 0; __pyx_t_9 = NULL; } else { __pyx_t_8 = -1; __pyx_t_4 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __pyx_t_9 = Py_TYPE(__pyx_t_4)->tp_iternext; if (unlikely(!__pyx_t_9)) __PYX_ERR(0, 66, __pyx_L1_error) } __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; for (;;) { if (likely(!__pyx_t_9)) { if (likely(PyList_CheckExact(__pyx_t_4))) { if (__pyx_t_8 >= PyList_GET_SIZE(__pyx_t_4)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyList_GET_ITEM(__pyx_t_4, __pyx_t_8); __Pyx_INCREF(__pyx_t_1); __pyx_t_8++; if (unlikely(0 < 0)) __PYX_ERR(0, 66, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_4, __pyx_t_8); __pyx_t_8++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } else { if (__pyx_t_8 >= PyTuple_GET_SIZE(__pyx_t_4)) break; #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS __pyx_t_1 = PyTuple_GET_ITEM(__pyx_t_4, __pyx_t_8); __Pyx_INCREF(__pyx_t_1); __pyx_t_8++; if (unlikely(0 < 0)) __PYX_ERR(0, 66, __pyx_L1_error) #else __pyx_t_1 = PySequence_ITEM(__pyx_t_4, __pyx_t_8); __pyx_t_8++; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 66, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); #endif } } else { __pyx_t_1 = __pyx_t_9(__pyx_t_4); if (unlikely(!__pyx_t_1)) { PyObject* exc_type = PyErr_Occurred(); if (exc_type) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) PyErr_Clear(); else __PYX_ERR(0, 66, __pyx_L1_error) } break; } __Pyx_GOTREF(__pyx_t_1); } __Pyx_XDECREF_SET(__pyx_v_entry, __pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":67 * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: # <<<<<<<<<<<<<< * type, name, permission = entry.split(':') * if name and type == 'user': */ __pyx_t_10 = __Pyx_PyObject_IsTrue(__pyx_v_entry); if (unlikely(__pyx_t_10 < 0)) __PYX_ERR(0, 67, __pyx_L1_error) if (__pyx_t_10) { /* "borg/platform_linux.pyx":68 * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: * type, name, permission = entry.split(':') # <<<<<<<<<<<<<< * if name and type == 'user': * uid = str(user2uid(name, name)) */ __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_entry, __pyx_n_s_split); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_2))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_2); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_2); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_2, function); } } __pyx_t_1 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_2, __pyx_t_6, __pyx_kp_s__2) : __Pyx_PyObject_CallOneArg(__pyx_t_2, __pyx_kp_s__2); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if ((likely(PyTuple_CheckExact(__pyx_t_1))) || (PyList_CheckExact(__pyx_t_1))) { PyObject* sequence = __pyx_t_1; Py_ssize_t size = __Pyx_PySequence_SIZE(sequence); if (unlikely(size != 3)) { if (size > 3) __Pyx_RaiseTooManyValuesError(3); else if (size >= 0) __Pyx_RaiseNeedMoreValuesError(size); __PYX_ERR(0, 68, __pyx_L1_error) } #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS if (likely(PyTuple_CheckExact(sequence))) { __pyx_t_2 = PyTuple_GET_ITEM(sequence, 0); __pyx_t_6 = PyTuple_GET_ITEM(sequence, 1); __pyx_t_3 = PyTuple_GET_ITEM(sequence, 2); } else { __pyx_t_2 = PyList_GET_ITEM(sequence, 0); __pyx_t_6 = PyList_GET_ITEM(sequence, 1); __pyx_t_3 = PyList_GET_ITEM(sequence, 2); } __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(__pyx_t_3); #else __pyx_t_2 = PySequence_ITEM(sequence, 0); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_6 = PySequence_ITEM(sequence, 1); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); __pyx_t_3 = PySequence_ITEM(sequence, 2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); #endif __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; } else { Py_ssize_t index = -1; __pyx_t_5 = PyObject_GetIter(__pyx_t_1); if (unlikely(!__pyx_t_5)) __PYX_ERR(0, 68, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_5); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_11 = Py_TYPE(__pyx_t_5)->tp_iternext; index = 0; __pyx_t_2 = __pyx_t_11(__pyx_t_5); if (unlikely(!__pyx_t_2)) goto __pyx_L6_unpacking_failed; __Pyx_GOTREF(__pyx_t_2); index = 1; __pyx_t_6 = __pyx_t_11(__pyx_t_5); if (unlikely(!__pyx_t_6)) goto __pyx_L6_unpacking_failed; __Pyx_GOTREF(__pyx_t_6); index = 2; __pyx_t_3 = __pyx_t_11(__pyx_t_5); if (unlikely(!__pyx_t_3)) goto __pyx_L6_unpacking_failed; __Pyx_GOTREF(__pyx_t_3); if (__Pyx_IternextUnpackEndCheck(__pyx_t_11(__pyx_t_5), 3) < 0) __PYX_ERR(0, 68, __pyx_L1_error) __pyx_t_11 = NULL; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; goto __pyx_L7_unpacking_done; __pyx_L6_unpacking_failed:; __Pyx_DECREF(__pyx_t_5); __pyx_t_5 = 0; __pyx_t_11 = NULL; if (__Pyx_IterFinish() == 0) __Pyx_RaiseNeedMoreValuesError(index); __PYX_ERR(0, 68, __pyx_L1_error) __pyx_L7_unpacking_done:; } __Pyx_XDECREF_SET(__pyx_v_type, __pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF_SET(__pyx_v_name, __pyx_t_6); __pyx_t_6 = 0; __Pyx_XDECREF_SET(__pyx_v_permission, __pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":69 * if entry: * type, name, permission = entry.split(':') * if name and type == 'user': # <<<<<<<<<<<<<< * uid = str(user2uid(name, name)) * entries.append(':'.join([type, uid, permission, uid])) */ __pyx_t_12 = __Pyx_PyObject_IsTrue(__pyx_v_name); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 69, __pyx_L1_error) if (__pyx_t_12) { } else { __pyx_t_10 = __pyx_t_12; goto __pyx_L9_bool_binop_done; } __pyx_t_12 = (__Pyx_PyString_Equals(__pyx_v_type, __pyx_n_s_user, Py_EQ)); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 69, __pyx_L1_error) __pyx_t_10 = __pyx_t_12; __pyx_L9_bool_binop_done:; if (__pyx_t_10) { /* "borg/platform_linux.pyx":70 * type, name, permission = entry.split(':') * if name and type == 'user': * uid = str(user2uid(name, name)) # <<<<<<<<<<<<<< * entries.append(':'.join([type, uid, permission, uid])) * elif name and type == 'group': */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_user2uid); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = NULL; __pyx_t_7 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_7 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_6, __pyx_v_name, __pyx_v_name}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_6, __pyx_v_name, __pyx_v_name}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif { __pyx_t_2 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (__pyx_t_6) { __Pyx_GIVEREF(__pyx_t_6); PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_6); __pyx_t_6 = NULL; } __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_2, 0+__pyx_t_7, __pyx_v_name); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_2, 1+__pyx_t_7, __pyx_v_name); __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_2, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 70, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF_SET(__pyx_v_uid, __pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":71 * if name and type == 'user': * uid = str(user2uid(name, name)) * entries.append(':'.join([type, uid, permission, uid])) # <<<<<<<<<<<<<< * elif name and type == 'group': * gid = str(group2gid(name, name)) */ __pyx_t_3 = PyList_New(4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 71, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_INCREF(__pyx_v_type); __Pyx_GIVEREF(__pyx_v_type); PyList_SET_ITEM(__pyx_t_3, 0, __pyx_v_type); __Pyx_INCREF(__pyx_v_uid); __Pyx_GIVEREF(__pyx_v_uid); PyList_SET_ITEM(__pyx_t_3, 1, __pyx_v_uid); __Pyx_INCREF(__pyx_v_permission); __Pyx_GIVEREF(__pyx_v_permission); PyList_SET_ITEM(__pyx_t_3, 2, __pyx_v_permission); __Pyx_INCREF(__pyx_v_uid); __Pyx_GIVEREF(__pyx_v_uid); PyList_SET_ITEM(__pyx_t_3, 3, __pyx_v_uid); __pyx_t_1 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 71, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_1); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 71, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":69 * if entry: * type, name, permission = entry.split(':') * if name and type == 'user': # <<<<<<<<<<<<<< * uid = str(user2uid(name, name)) * entries.append(':'.join([type, uid, permission, uid])) */ goto __pyx_L8; } /* "borg/platform_linux.pyx":72 * uid = str(user2uid(name, name)) * entries.append(':'.join([type, uid, permission, uid])) * elif name and type == 'group': # <<<<<<<<<<<<<< * gid = str(group2gid(name, name)) * entries.append(':'.join([type, gid, permission, gid])) */ __pyx_t_12 = __Pyx_PyObject_IsTrue(__pyx_v_name); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 72, __pyx_L1_error) if (__pyx_t_12) { } else { __pyx_t_10 = __pyx_t_12; goto __pyx_L11_bool_binop_done; } __pyx_t_12 = (__Pyx_PyString_Equals(__pyx_v_type, __pyx_n_s_group, Py_EQ)); if (unlikely(__pyx_t_12 < 0)) __PYX_ERR(0, 72, __pyx_L1_error) __pyx_t_10 = __pyx_t_12; __pyx_L11_bool_binop_done:; if (__pyx_t_10) { /* "borg/platform_linux.pyx":73 * entries.append(':'.join([type, uid, permission, uid])) * elif name and type == 'group': * gid = str(group2gid(name, name)) # <<<<<<<<<<<<<< * entries.append(':'.join([type, gid, permission, gid])) * else: */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_group2gid); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 73, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_2 = NULL; __pyx_t_7 = 0; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); __pyx_t_7 = 1; } } #if CYTHON_FAST_PYCALL if (PyFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_name, __pyx_v_name}; __pyx_t_1 = __Pyx_PyFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 73, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(__pyx_t_3)) { PyObject *__pyx_temp[3] = {__pyx_t_2, __pyx_v_name, __pyx_v_name}; __pyx_t_1 = __Pyx_PyCFunction_FastCall(__pyx_t_3, __pyx_temp+1-__pyx_t_7, 2+__pyx_t_7); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 73, __pyx_L1_error) __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_GOTREF(__pyx_t_1); } else #endif { __pyx_t_6 = PyTuple_New(2+__pyx_t_7); if (unlikely(!__pyx_t_6)) __PYX_ERR(0, 73, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_6); if (__pyx_t_2) { __Pyx_GIVEREF(__pyx_t_2); PyTuple_SET_ITEM(__pyx_t_6, 0, __pyx_t_2); __pyx_t_2 = NULL; } __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_6, 0+__pyx_t_7, __pyx_v_name); __Pyx_INCREF(__pyx_v_name); __Pyx_GIVEREF(__pyx_v_name); PyTuple_SET_ITEM(__pyx_t_6, 1+__pyx_t_7, __pyx_v_name); __pyx_t_1 = __Pyx_PyObject_Call(__pyx_t_3, __pyx_t_6, NULL); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 73, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_6); __pyx_t_6 = 0; } __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_CallOneArg(((PyObject *)(&PyString_Type)), __pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 73, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF_SET(__pyx_v_gid, __pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":74 * elif name and type == 'group': * gid = str(group2gid(name, name)) * entries.append(':'.join([type, gid, permission, gid])) # <<<<<<<<<<<<<< * else: * entries.append(entry) */ __pyx_t_3 = PyList_New(4); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 74, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_INCREF(__pyx_v_type); __Pyx_GIVEREF(__pyx_v_type); PyList_SET_ITEM(__pyx_t_3, 0, __pyx_v_type); __Pyx_INCREF(__pyx_v_gid); __Pyx_GIVEREF(__pyx_v_gid); PyList_SET_ITEM(__pyx_t_3, 1, __pyx_v_gid); __Pyx_INCREF(__pyx_v_permission); __Pyx_GIVEREF(__pyx_v_permission); PyList_SET_ITEM(__pyx_t_3, 2, __pyx_v_permission); __Pyx_INCREF(__pyx_v_gid); __Pyx_GIVEREF(__pyx_v_gid); PyList_SET_ITEM(__pyx_t_3, 3, __pyx_v_gid); __pyx_t_1 = __Pyx_PyString_Join(__pyx_kp_s__2, __pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 74, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_t_1); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 74, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":72 * uid = str(user2uid(name, name)) * entries.append(':'.join([type, uid, permission, uid])) * elif name and type == 'group': # <<<<<<<<<<<<<< * gid = str(group2gid(name, name)) * entries.append(':'.join([type, gid, permission, gid])) */ goto __pyx_L8; } /* "borg/platform_linux.pyx":76 * entries.append(':'.join([type, gid, permission, gid])) * else: * entries.append(entry) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ /*else*/ { __pyx_t_13 = __Pyx_PyList_Append(__pyx_v_entries, __pyx_v_entry); if (unlikely(__pyx_t_13 == ((int)-1))) __PYX_ERR(0, 76, __pyx_L1_error) } __pyx_L8:; /* "borg/platform_linux.pyx":67 * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): * if entry: # <<<<<<<<<<<<<< * type, name, permission = entry.split(':') * if name and type == 'user': */ } /* "borg/platform_linux.pyx":66 * """ * entries = [] * for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): # <<<<<<<<<<<<<< * if entry: * type, name, permission = entry.split(':') */ } __Pyx_DECREF(__pyx_t_4); __pyx_t_4 = 0; /* "borg/platform_linux.pyx":77 * else: * entries.append(entry) * return safe_encode('\n'.join(entries)) # <<<<<<<<<<<<<< * * */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 77, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __Pyx_PyString_Join(__pyx_kp_s_, __pyx_v_entries); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 77, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_6 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_6 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_6)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_6); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_4 = (__pyx_t_6) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_6, __pyx_t_3) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_t_3); __Pyx_XDECREF(__pyx_t_6); __pyx_t_6 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 77, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L0; /* "borg/platform_linux.pyx":62 * * * cdef acl_numeric_ids(acl): # <<<<<<<<<<<<<< * """Replace the "POSIX 1003.1e draft standard 17" user/group field with uid/gid * """ */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_XDECREF(__pyx_t_5); __Pyx_XDECREF(__pyx_t_6); __Pyx_AddTraceback("borg.platform_linux.acl_numeric_ids", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = 0; __pyx_L0:; __Pyx_XDECREF(__pyx_v_entries); __Pyx_XDECREF(__pyx_v_entry); __Pyx_XDECREF(__pyx_v_type); __Pyx_XDECREF(__pyx_v_name); __Pyx_XDECREF(__pyx_v_permission); __Pyx_XDECREF(__pyx_v_uid); __Pyx_XDECREF(__pyx_v_gid); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_linux.pyx":80 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_14platform_linux_3acl_get(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_14platform_linux_2acl_get[] = "Saves ACL Entries\n\n If `numeric_owner` is True the user/group field is not preserved only uid/gid\n "; static PyMethodDef __pyx_mdef_4borg_14platform_linux_3acl_get = {"acl_get", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_14platform_linux_3acl_get, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_14platform_linux_2acl_get}; static PyObject *__pyx_pw_4borg_14platform_linux_3acl_get(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_path = 0; PyObject *__pyx_v_item = 0; PyObject *__pyx_v_st = 0; PyObject *__pyx_v_numeric_owner = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_get (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_path,&__pyx_n_s_item,&__pyx_n_s_st,&__pyx_n_s_numeric_owner,0}; PyObject* values[4] = {0,0,0,0}; values[3] = ((PyObject *)Py_False); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_item)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, 1); __PYX_ERR(0, 80, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (likely((values[2] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_st)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, 2); __PYX_ERR(0, 80, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 3: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_numeric_owner); if (value) { values[3] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "acl_get") < 0)) __PYX_ERR(0, 80, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 4: values[3] = PyTuple_GET_ITEM(__pyx_args, 3); CYTHON_FALLTHROUGH; case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_path = values[0]; __pyx_v_item = values[1]; __pyx_v_st = values[2]; __pyx_v_numeric_owner = values[3]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("acl_get", 0, 3, 4, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 80, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.platform_linux.acl_get", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_14platform_linux_2acl_get(__pyx_self, __pyx_v_path, __pyx_v_item, __pyx_v_st, __pyx_v_numeric_owner); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_14platform_linux_2acl_get(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_st, PyObject *__pyx_v_numeric_owner) { acl_t __pyx_v_default_acl; acl_t __pyx_v_access_acl; char *__pyx_v_default_text; char *__pyx_v_access_text; PyObject *__pyx_v_p = NULL; PyObject *(*__pyx_v_converter)(PyObject *); PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; PyObject *__pyx_t_5 = NULL; int __pyx_t_6; char const *__pyx_t_7; char const *__pyx_t_8; int __pyx_t_9; int __pyx_t_10; char const *__pyx_t_11; PyObject *__pyx_t_12 = NULL; PyObject *__pyx_t_13 = NULL; PyObject *__pyx_t_14 = NULL; PyObject *__pyx_t_15 = NULL; PyObject *__pyx_t_16 = NULL; PyObject *__pyx_t_17 = NULL; __Pyx_RefNannySetupContext("acl_get", 0); /* "borg/platform_linux.pyx":85 * If `numeric_owner` is True the user/group field is not preserved only uid/gid * """ * cdef acl_t default_acl = NULL # <<<<<<<<<<<<<< * cdef acl_t access_acl = NULL * cdef char *default_text = NULL */ __pyx_v_default_acl = NULL; /* "borg/platform_linux.pyx":86 * """ * cdef acl_t default_acl = NULL * cdef acl_t access_acl = NULL # <<<<<<<<<<<<<< * cdef char *default_text = NULL * cdef char *access_text = NULL */ __pyx_v_access_acl = NULL; /* "borg/platform_linux.pyx":87 * cdef acl_t default_acl = NULL * cdef acl_t access_acl = NULL * cdef char *default_text = NULL # <<<<<<<<<<<<<< * cdef char *access_text = NULL * */ __pyx_v_default_text = NULL; /* "borg/platform_linux.pyx":88 * cdef acl_t access_acl = NULL * cdef char *default_text = NULL * cdef char *access_text = NULL # <<<<<<<<<<<<<< * * p = os.fsencode(path) */ __pyx_v_access_text = NULL; /* "borg/platform_linux.pyx":90 * cdef char *access_text = NULL * * p = os.fsencode(path) # <<<<<<<<<<<<<< * if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: * return */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_os); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 90, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 90, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_path); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 90, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __pyx_t_1; __Pyx_INCREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_p = ((PyObject*)__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":91 * * p = os.fsencode(path) * if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: # <<<<<<<<<<<<<< * return * if numeric_owner: */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_S_ISLNK); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_v_st, __pyx_n_s_st_mode); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_5 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_5 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_5)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_5); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_5) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_5, __pyx_t_2) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_t_2); __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_6 = __Pyx_PyObject_IsTrue(__pyx_t_3); if (unlikely(__pyx_t_6 < 0)) __PYX_ERR(0, 91, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (!__pyx_t_6) { } else { __pyx_t_4 = __pyx_t_6; goto __pyx_L4_bool_binop_done; } if (unlikely(__pyx_v_p == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 91, __pyx_L1_error) } __pyx_t_7 = __Pyx_PyBytes_AsString(__pyx_v_p); if (unlikely((!__pyx_t_7) && PyErr_Occurred())) __PYX_ERR(0, 91, __pyx_L1_error) __pyx_t_6 = ((acl_extended_file(__pyx_t_7) <= 0) != 0); __pyx_t_4 = __pyx_t_6; __pyx_L4_bool_binop_done:; if (__pyx_t_4) { /* "borg/platform_linux.pyx":92 * p = os.fsencode(path) * if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: * return # <<<<<<<<<<<<<< * if numeric_owner: * converter = acl_numeric_ids */ __Pyx_XDECREF(__pyx_r); __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; /* "borg/platform_linux.pyx":91 * * p = os.fsencode(path) * if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: # <<<<<<<<<<<<<< * return * if numeric_owner: */ } /* "borg/platform_linux.pyx":93 * if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: * return * if numeric_owner: # <<<<<<<<<<<<<< * converter = acl_numeric_ids * else: */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 93, __pyx_L1_error) if (__pyx_t_4) { /* "borg/platform_linux.pyx":94 * return * if numeric_owner: * converter = acl_numeric_ids # <<<<<<<<<<<<<< * else: * converter = acl_append_numeric_ids */ __pyx_v_converter = __pyx_f_4borg_14platform_linux_acl_numeric_ids; /* "borg/platform_linux.pyx":93 * if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: * return * if numeric_owner: # <<<<<<<<<<<<<< * converter = acl_numeric_ids * else: */ goto __pyx_L6; } /* "borg/platform_linux.pyx":96 * converter = acl_numeric_ids * else: * converter = acl_append_numeric_ids # <<<<<<<<<<<<<< * try: * access_acl = acl_get_file(p, ACL_TYPE_ACCESS) */ /*else*/ { __pyx_v_converter = __pyx_f_4borg_14platform_linux_acl_append_numeric_ids; } __pyx_L6:; /* "borg/platform_linux.pyx":97 * else: * converter = acl_append_numeric_ids * try: # <<<<<<<<<<<<<< * access_acl = acl_get_file(p, ACL_TYPE_ACCESS) * if access_acl: */ /*try:*/ { /* "borg/platform_linux.pyx":98 * converter = acl_append_numeric_ids * try: * access_acl = acl_get_file(p, ACL_TYPE_ACCESS) # <<<<<<<<<<<<<< * if access_acl: * access_text = acl_to_text(access_acl, NULL) */ if (unlikely(__pyx_v_p == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 98, __pyx_L8_error) } __pyx_t_8 = __Pyx_PyBytes_AsString(__pyx_v_p); if (unlikely((!__pyx_t_8) && PyErr_Occurred())) __PYX_ERR(0, 98, __pyx_L8_error) __pyx_v_access_acl = acl_get_file(__pyx_t_8, ACL_TYPE_ACCESS); /* "borg/platform_linux.pyx":99 * try: * access_acl = acl_get_file(p, ACL_TYPE_ACCESS) * if access_acl: # <<<<<<<<<<<<<< * access_text = acl_to_text(access_acl, NULL) * if access_text: */ __pyx_t_4 = (__pyx_v_access_acl != 0); if (__pyx_t_4) { /* "borg/platform_linux.pyx":100 * access_acl = acl_get_file(p, ACL_TYPE_ACCESS) * if access_acl: * access_text = acl_to_text(access_acl, NULL) # <<<<<<<<<<<<<< * if access_text: * item[b'acl_access'] = converter(access_text) */ __pyx_v_access_text = acl_to_text(__pyx_v_access_acl, NULL); /* "borg/platform_linux.pyx":101 * if access_acl: * access_text = acl_to_text(access_acl, NULL) * if access_text: # <<<<<<<<<<<<<< * item[b'acl_access'] = converter(access_text) * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) */ __pyx_t_4 = (__pyx_v_access_text != 0); if (__pyx_t_4) { /* "borg/platform_linux.pyx":102 * access_text = acl_to_text(access_acl, NULL) * if access_text: * item[b'acl_access'] = converter(access_text) # <<<<<<<<<<<<<< * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) * if default_acl: */ __pyx_t_3 = __Pyx_PyBytes_FromString(__pyx_v_access_text); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 102, __pyx_L8_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_1 = __pyx_v_converter(__pyx_t_3); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 102, __pyx_L8_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; if (unlikely(PyObject_SetItem(__pyx_v_item, __pyx_n_b_acl_access, __pyx_t_1) < 0)) __PYX_ERR(0, 102, __pyx_L8_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":101 * if access_acl: * access_text = acl_to_text(access_acl, NULL) * if access_text: # <<<<<<<<<<<<<< * item[b'acl_access'] = converter(access_text) * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) */ } /* "borg/platform_linux.pyx":99 * try: * access_acl = acl_get_file(p, ACL_TYPE_ACCESS) * if access_acl: # <<<<<<<<<<<<<< * access_text = acl_to_text(access_acl, NULL) * if access_text: */ } /* "borg/platform_linux.pyx":103 * if access_text: * item[b'acl_access'] = converter(access_text) * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) # <<<<<<<<<<<<<< * if default_acl: * default_text = acl_to_text(default_acl, NULL) */ if (unlikely(__pyx_v_p == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 103, __pyx_L8_error) } __pyx_t_8 = __Pyx_PyBytes_AsString(__pyx_v_p); if (unlikely((!__pyx_t_8) && PyErr_Occurred())) __PYX_ERR(0, 103, __pyx_L8_error) __pyx_v_default_acl = acl_get_file(__pyx_t_8, ACL_TYPE_DEFAULT); /* "borg/platform_linux.pyx":104 * item[b'acl_access'] = converter(access_text) * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) * if default_acl: # <<<<<<<<<<<<<< * default_text = acl_to_text(default_acl, NULL) * if default_text: */ __pyx_t_4 = (__pyx_v_default_acl != 0); if (__pyx_t_4) { /* "borg/platform_linux.pyx":105 * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) * if default_acl: * default_text = acl_to_text(default_acl, NULL) # <<<<<<<<<<<<<< * if default_text: * item[b'acl_default'] = converter(default_text) */ __pyx_v_default_text = acl_to_text(__pyx_v_default_acl, NULL); /* "borg/platform_linux.pyx":106 * if default_acl: * default_text = acl_to_text(default_acl, NULL) * if default_text: # <<<<<<<<<<<<<< * item[b'acl_default'] = converter(default_text) * finally: */ __pyx_t_4 = (__pyx_v_default_text != 0); if (__pyx_t_4) { /* "borg/platform_linux.pyx":107 * default_text = acl_to_text(default_acl, NULL) * if default_text: * item[b'acl_default'] = converter(default_text) # <<<<<<<<<<<<<< * finally: * acl_free(default_text) */ __pyx_t_1 = __Pyx_PyBytes_FromString(__pyx_v_default_text); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 107, __pyx_L8_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __pyx_v_converter(__pyx_t_1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 107, __pyx_L8_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(PyObject_SetItem(__pyx_v_item, __pyx_n_b_acl_default, __pyx_t_3) < 0)) __PYX_ERR(0, 107, __pyx_L8_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":106 * if default_acl: * default_text = acl_to_text(default_acl, NULL) * if default_text: # <<<<<<<<<<<<<< * item[b'acl_default'] = converter(default_text) * finally: */ } /* "borg/platform_linux.pyx":104 * item[b'acl_access'] = converter(access_text) * default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) * if default_acl: # <<<<<<<<<<<<<< * default_text = acl_to_text(default_acl, NULL) * if default_text: */ } } /* "borg/platform_linux.pyx":109 * item[b'acl_default'] = converter(default_text) * finally: * acl_free(default_text) # <<<<<<<<<<<<<< * acl_free(default_acl) * acl_free(access_text) */ /*finally:*/ { /*normal exit:*/{ (void)(acl_free(__pyx_v_default_text)); /* "borg/platform_linux.pyx":110 * finally: * acl_free(default_text) * acl_free(default_acl) # <<<<<<<<<<<<<< * acl_free(access_text) * acl_free(access_acl) */ (void)(acl_free(__pyx_v_default_acl)); /* "borg/platform_linux.pyx":111 * acl_free(default_text) * acl_free(default_acl) * acl_free(access_text) # <<<<<<<<<<<<<< * acl_free(access_acl) * */ (void)(acl_free(__pyx_v_access_text)); /* "borg/platform_linux.pyx":112 * acl_free(default_acl) * acl_free(access_text) * acl_free(access_acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_access_acl)); goto __pyx_L9; } __pyx_L8_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_12 = 0; __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_t_17 = 0; __Pyx_XDECREF(__pyx_t_5); __pyx_t_5 = 0; __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_15, &__pyx_t_16, &__pyx_t_17); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_12, &__pyx_t_13, &__pyx_t_14) < 0)) __Pyx_ErrFetch(&__pyx_t_12, &__pyx_t_13, &__pyx_t_14); __Pyx_XGOTREF(__pyx_t_12); __Pyx_XGOTREF(__pyx_t_13); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_15); __Pyx_XGOTREF(__pyx_t_16); __Pyx_XGOTREF(__pyx_t_17); __pyx_t_9 = __pyx_lineno; __pyx_t_10 = __pyx_clineno; __pyx_t_11 = __pyx_filename; { /* "borg/platform_linux.pyx":109 * item[b'acl_default'] = converter(default_text) * finally: * acl_free(default_text) # <<<<<<<<<<<<<< * acl_free(default_acl) * acl_free(access_text) */ (void)(acl_free(__pyx_v_default_text)); /* "borg/platform_linux.pyx":110 * finally: * acl_free(default_text) * acl_free(default_acl) # <<<<<<<<<<<<<< * acl_free(access_text) * acl_free(access_acl) */ (void)(acl_free(__pyx_v_default_acl)); /* "borg/platform_linux.pyx":111 * acl_free(default_text) * acl_free(default_acl) * acl_free(access_text) # <<<<<<<<<<<<<< * acl_free(access_acl) * */ (void)(acl_free(__pyx_v_access_text)); /* "borg/platform_linux.pyx":112 * acl_free(default_acl) * acl_free(access_text) * acl_free(access_acl) # <<<<<<<<<<<<<< * * */ (void)(acl_free(__pyx_v_access_acl)); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_15); __Pyx_XGIVEREF(__pyx_t_16); __Pyx_XGIVEREF(__pyx_t_17); __Pyx_ExceptionReset(__pyx_t_15, __pyx_t_16, __pyx_t_17); } __Pyx_XGIVEREF(__pyx_t_12); __Pyx_XGIVEREF(__pyx_t_13); __Pyx_XGIVEREF(__pyx_t_14); __Pyx_ErrRestore(__pyx_t_12, __pyx_t_13, __pyx_t_14); __pyx_t_12 = 0; __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_t_16 = 0; __pyx_t_17 = 0; __pyx_lineno = __pyx_t_9; __pyx_clineno = __pyx_t_10; __pyx_filename = __pyx_t_11; goto __pyx_L1_error; } __pyx_L9:; } /* "borg/platform_linux.pyx":80 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_5); __Pyx_AddTraceback("borg.platform_linux.acl_get", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_p); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_linux.pyx":115 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_14platform_linux_5acl_set(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds); /*proto*/ static char __pyx_doc_4borg_14platform_linux_4acl_set[] = "Restore ACL Entries\n\n If `numeric_owner` is True the stored uid/gid is used instead\n of the user/group names\n "; static PyMethodDef __pyx_mdef_4borg_14platform_linux_5acl_set = {"acl_set", (PyCFunction)(void*)(PyCFunctionWithKeywords)__pyx_pw_4borg_14platform_linux_5acl_set, METH_VARARGS|METH_KEYWORDS, __pyx_doc_4borg_14platform_linux_4acl_set}; static PyObject *__pyx_pw_4borg_14platform_linux_5acl_set(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) { PyObject *__pyx_v_path = 0; PyObject *__pyx_v_item = 0; PyObject *__pyx_v_numeric_owner = 0; PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("acl_set (wrapper)", 0); { static PyObject **__pyx_pyargnames[] = {&__pyx_n_s_path,&__pyx_n_s_item,&__pyx_n_s_numeric_owner,0}; PyObject* values[3] = {0,0,0}; values[2] = ((PyObject *)Py_False); if (unlikely(__pyx_kwds)) { Py_ssize_t kw_args; const Py_ssize_t pos_args = PyTuple_GET_SIZE(__pyx_args); switch (pos_args) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); CYTHON_FALLTHROUGH; case 1: values[0] = PyTuple_GET_ITEM(__pyx_args, 0); CYTHON_FALLTHROUGH; case 0: break; default: goto __pyx_L5_argtuple_error; } kw_args = PyDict_Size(__pyx_kwds); switch (pos_args) { case 0: if (likely((values[0] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_path)) != 0)) kw_args--; else goto __pyx_L5_argtuple_error; CYTHON_FALLTHROUGH; case 1: if (likely((values[1] = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_item)) != 0)) kw_args--; else { __Pyx_RaiseArgtupleInvalid("acl_set", 0, 2, 3, 1); __PYX_ERR(0, 115, __pyx_L3_error) } CYTHON_FALLTHROUGH; case 2: if (kw_args > 0) { PyObject* value = __Pyx_PyDict_GetItemStr(__pyx_kwds, __pyx_n_s_numeric_owner); if (value) { values[2] = value; kw_args--; } } } if (unlikely(kw_args > 0)) { if (unlikely(__Pyx_ParseOptionalKeywords(__pyx_kwds, __pyx_pyargnames, 0, values, pos_args, "acl_set") < 0)) __PYX_ERR(0, 115, __pyx_L3_error) } } else { switch (PyTuple_GET_SIZE(__pyx_args)) { case 3: values[2] = PyTuple_GET_ITEM(__pyx_args, 2); CYTHON_FALLTHROUGH; case 2: values[1] = PyTuple_GET_ITEM(__pyx_args, 1); values[0] = PyTuple_GET_ITEM(__pyx_args, 0); break; default: goto __pyx_L5_argtuple_error; } } __pyx_v_path = values[0]; __pyx_v_item = values[1]; __pyx_v_numeric_owner = values[2]; } goto __pyx_L4_argument_unpacking_done; __pyx_L5_argtuple_error:; __Pyx_RaiseArgtupleInvalid("acl_set", 0, 2, 3, PyTuple_GET_SIZE(__pyx_args)); __PYX_ERR(0, 115, __pyx_L3_error) __pyx_L3_error:; __Pyx_AddTraceback("borg.platform_linux.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); __Pyx_RefNannyFinishContext(); return NULL; __pyx_L4_argument_unpacking_done:; __pyx_r = __pyx_pf_4borg_14platform_linux_4acl_set(__pyx_self, __pyx_v_path, __pyx_v_item, __pyx_v_numeric_owner); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_14platform_linux_4acl_set(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_path, PyObject *__pyx_v_item, PyObject *__pyx_v_numeric_owner) { acl_t __pyx_v_access_acl; acl_t __pyx_v_default_acl; PyObject *__pyx_v_p = NULL; PyObject *__pyx_v_converter = NULL; PyObject *__pyx_v_access_text = NULL; PyObject *__pyx_v_default_text = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; int __pyx_t_4; char const *__pyx_t_5; char const *__pyx_t_6; int __pyx_t_7; int __pyx_t_8; char const *__pyx_t_9; PyObject *__pyx_t_10 = NULL; PyObject *__pyx_t_11 = NULL; PyObject *__pyx_t_12 = NULL; PyObject *__pyx_t_13 = NULL; PyObject *__pyx_t_14 = NULL; PyObject *__pyx_t_15 = NULL; char const *__pyx_t_16; __Pyx_RefNannySetupContext("acl_set", 0); /* "borg/platform_linux.pyx":121 * of the user/group names * """ * cdef acl_t access_acl = NULL # <<<<<<<<<<<<<< * cdef acl_t default_acl = NULL * */ __pyx_v_access_acl = NULL; /* "borg/platform_linux.pyx":122 * """ * cdef acl_t access_acl = NULL * cdef acl_t default_acl = NULL # <<<<<<<<<<<<<< * * p = os.fsencode(path) */ __pyx_v_default_acl = NULL; /* "borg/platform_linux.pyx":124 * cdef acl_t default_acl = NULL * * p = os.fsencode(path) # <<<<<<<<<<<<<< * if numeric_owner: * converter = posix_acl_use_stored_uid_gid */ __Pyx_GetModuleGlobalName(__pyx_t_2, __pyx_n_s_os); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 124, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_2, __pyx_n_s_fsencode); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 124, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_3))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_3); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_3); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_3, function); } } __pyx_t_1 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_3, __pyx_t_2, __pyx_v_path) : __Pyx_PyObject_CallOneArg(__pyx_t_3, __pyx_v_path); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 124, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __pyx_t_1; __Pyx_INCREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_p = ((PyObject*)__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":125 * * p = os.fsencode(path) * if numeric_owner: # <<<<<<<<<<<<<< * converter = posix_acl_use_stored_uid_gid * else: */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_numeric_owner); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 125, __pyx_L1_error) if (__pyx_t_4) { /* "borg/platform_linux.pyx":126 * p = os.fsencode(path) * if numeric_owner: * converter = posix_acl_use_stored_uid_gid # <<<<<<<<<<<<<< * else: * converter = acl_use_local_uid_gid */ __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_posix_acl_use_stored_uid_gid); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 126, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_v_converter = __pyx_t_3; __pyx_t_3 = 0; /* "borg/platform_linux.pyx":125 * * p = os.fsencode(path) * if numeric_owner: # <<<<<<<<<<<<<< * converter = posix_acl_use_stored_uid_gid * else: */ goto __pyx_L3; } /* "borg/platform_linux.pyx":128 * converter = posix_acl_use_stored_uid_gid * else: * converter = acl_use_local_uid_gid # <<<<<<<<<<<<<< * access_text = item.get(b'acl_access') * default_text = item.get(b'acl_default') */ /*else*/ { __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_acl_use_local_uid_gid); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 128, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_v_converter = __pyx_t_3; __pyx_t_3 = 0; } __pyx_L3:; /* "borg/platform_linux.pyx":129 * else: * converter = acl_use_local_uid_gid * access_text = item.get(b'acl_access') # <<<<<<<<<<<<<< * default_text = item.get(b'acl_default') * if access_text: */ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_v_item, __pyx_n_s_get); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_1))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_2, __pyx_n_b_acl_access) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_n_b_acl_access); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 129, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_access_text = __pyx_t_3; __pyx_t_3 = 0; /* "borg/platform_linux.pyx":130 * converter = acl_use_local_uid_gid * access_text = item.get(b'acl_access') * default_text = item.get(b'acl_default') # <<<<<<<<<<<<<< * if access_text: * try: */ __pyx_t_1 = __Pyx_PyObject_GetAttrStr(__pyx_v_item, __pyx_n_s_get); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 130, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && likely(PyMethod_Check(__pyx_t_1))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_2, __pyx_n_b_acl_default) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_n_b_acl_default); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 130, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_v_default_text = __pyx_t_3; __pyx_t_3 = 0; /* "borg/platform_linux.pyx":131 * access_text = item.get(b'acl_access') * default_text = item.get(b'acl_default') * if access_text: # <<<<<<<<<<<<<< * try: * access_acl = acl_from_text(converter(access_text)) */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_access_text); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 131, __pyx_L1_error) if (__pyx_t_4) { /* "borg/platform_linux.pyx":132 * default_text = item.get(b'acl_default') * if access_text: * try: # <<<<<<<<<<<<<< * access_acl = acl_from_text(converter(access_text)) * if access_acl: */ /*try:*/ { /* "borg/platform_linux.pyx":133 * if access_text: * try: * access_acl = acl_from_text(converter(access_text)) # <<<<<<<<<<<<<< * if access_acl: * acl_set_file(p, ACL_TYPE_ACCESS, access_acl) */ __Pyx_INCREF(__pyx_v_converter); __pyx_t_1 = __pyx_v_converter; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_2, __pyx_v_access_text) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_v_access_text); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 133, __pyx_L6_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(__pyx_t_3 == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 133, __pyx_L6_error) } __pyx_t_5 = __Pyx_PyBytes_AsString(__pyx_t_3); if (unlikely((!__pyx_t_5) && PyErr_Occurred())) __PYX_ERR(0, 133, __pyx_L6_error) __pyx_v_access_acl = acl_from_text(__pyx_t_5); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":134 * try: * access_acl = acl_from_text(converter(access_text)) * if access_acl: # <<<<<<<<<<<<<< * acl_set_file(p, ACL_TYPE_ACCESS, access_acl) * finally: */ __pyx_t_4 = (__pyx_v_access_acl != 0); if (__pyx_t_4) { /* "borg/platform_linux.pyx":135 * access_acl = acl_from_text(converter(access_text)) * if access_acl: * acl_set_file(p, ACL_TYPE_ACCESS, access_acl) # <<<<<<<<<<<<<< * finally: * acl_free(access_acl) */ if (unlikely(__pyx_v_p == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 135, __pyx_L6_error) } __pyx_t_6 = __Pyx_PyBytes_AsString(__pyx_v_p); if (unlikely((!__pyx_t_6) && PyErr_Occurred())) __PYX_ERR(0, 135, __pyx_L6_error) (void)(acl_set_file(__pyx_t_6, ACL_TYPE_ACCESS, __pyx_v_access_acl)); /* "borg/platform_linux.pyx":134 * try: * access_acl = acl_from_text(converter(access_text)) * if access_acl: # <<<<<<<<<<<<<< * acl_set_file(p, ACL_TYPE_ACCESS, access_acl) * finally: */ } } /* "borg/platform_linux.pyx":137 * acl_set_file(p, ACL_TYPE_ACCESS, access_acl) * finally: * acl_free(access_acl) # <<<<<<<<<<<<<< * if default_text: * try: */ /*finally:*/ { /*normal exit:*/{ (void)(acl_free(__pyx_v_access_acl)); goto __pyx_L7; } __pyx_L6_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_10 = 0; __pyx_t_11 = 0; __pyx_t_12 = 0; __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_13, &__pyx_t_14, &__pyx_t_15); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_10, &__pyx_t_11, &__pyx_t_12) < 0)) __Pyx_ErrFetch(&__pyx_t_10, &__pyx_t_11, &__pyx_t_12); __Pyx_XGOTREF(__pyx_t_10); __Pyx_XGOTREF(__pyx_t_11); __Pyx_XGOTREF(__pyx_t_12); __Pyx_XGOTREF(__pyx_t_13); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_15); __pyx_t_7 = __pyx_lineno; __pyx_t_8 = __pyx_clineno; __pyx_t_9 = __pyx_filename; { (void)(acl_free(__pyx_v_access_acl)); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_13); __Pyx_XGIVEREF(__pyx_t_14); __Pyx_XGIVEREF(__pyx_t_15); __Pyx_ExceptionReset(__pyx_t_13, __pyx_t_14, __pyx_t_15); } __Pyx_XGIVEREF(__pyx_t_10); __Pyx_XGIVEREF(__pyx_t_11); __Pyx_XGIVEREF(__pyx_t_12); __Pyx_ErrRestore(__pyx_t_10, __pyx_t_11, __pyx_t_12); __pyx_t_10 = 0; __pyx_t_11 = 0; __pyx_t_12 = 0; __pyx_t_13 = 0; __pyx_t_14 = 0; __pyx_t_15 = 0; __pyx_lineno = __pyx_t_7; __pyx_clineno = __pyx_t_8; __pyx_filename = __pyx_t_9; goto __pyx_L1_error; } __pyx_L7:; } /* "borg/platform_linux.pyx":131 * access_text = item.get(b'acl_access') * default_text = item.get(b'acl_default') * if access_text: # <<<<<<<<<<<<<< * try: * access_acl = acl_from_text(converter(access_text)) */ } /* "borg/platform_linux.pyx":138 * finally: * acl_free(access_acl) * if default_text: # <<<<<<<<<<<<<< * try: * default_acl = acl_from_text(converter(default_text)) */ __pyx_t_4 = __Pyx_PyObject_IsTrue(__pyx_v_default_text); if (unlikely(__pyx_t_4 < 0)) __PYX_ERR(0, 138, __pyx_L1_error) if (__pyx_t_4) { /* "borg/platform_linux.pyx":139 * acl_free(access_acl) * if default_text: * try: # <<<<<<<<<<<<<< * default_acl = acl_from_text(converter(default_text)) * if default_acl: */ /*try:*/ { /* "borg/platform_linux.pyx":140 * if default_text: * try: * default_acl = acl_from_text(converter(default_text)) # <<<<<<<<<<<<<< * if default_acl: * acl_set_file(p, ACL_TYPE_DEFAULT, default_acl) */ __Pyx_INCREF(__pyx_v_converter); __pyx_t_1 = __pyx_v_converter; __pyx_t_2 = NULL; if (CYTHON_UNPACK_METHODS && unlikely(PyMethod_Check(__pyx_t_1))) { __pyx_t_2 = PyMethod_GET_SELF(__pyx_t_1); if (likely(__pyx_t_2)) { PyObject* function = PyMethod_GET_FUNCTION(__pyx_t_1); __Pyx_INCREF(__pyx_t_2); __Pyx_INCREF(function); __Pyx_DECREF_SET(__pyx_t_1, function); } } __pyx_t_3 = (__pyx_t_2) ? __Pyx_PyObject_Call2Args(__pyx_t_1, __pyx_t_2, __pyx_v_default_text) : __Pyx_PyObject_CallOneArg(__pyx_t_1, __pyx_v_default_text); __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 140, __pyx_L13_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (unlikely(__pyx_t_3 == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 140, __pyx_L13_error) } __pyx_t_5 = __Pyx_PyBytes_AsString(__pyx_t_3); if (unlikely((!__pyx_t_5) && PyErr_Occurred())) __PYX_ERR(0, 140, __pyx_L13_error) __pyx_v_default_acl = acl_from_text(__pyx_t_5); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":141 * try: * default_acl = acl_from_text(converter(default_text)) * if default_acl: # <<<<<<<<<<<<<< * acl_set_file(p, ACL_TYPE_DEFAULT, default_acl) * finally: */ __pyx_t_4 = (__pyx_v_default_acl != 0); if (__pyx_t_4) { /* "borg/platform_linux.pyx":142 * default_acl = acl_from_text(converter(default_text)) * if default_acl: * acl_set_file(p, ACL_TYPE_DEFAULT, default_acl) # <<<<<<<<<<<<<< * finally: * acl_free(default_acl) */ if (unlikely(__pyx_v_p == Py_None)) { PyErr_SetString(PyExc_TypeError, "expected bytes, NoneType found"); __PYX_ERR(0, 142, __pyx_L13_error) } __pyx_t_6 = __Pyx_PyBytes_AsString(__pyx_v_p); if (unlikely((!__pyx_t_6) && PyErr_Occurred())) __PYX_ERR(0, 142, __pyx_L13_error) (void)(acl_set_file(__pyx_t_6, ACL_TYPE_DEFAULT, __pyx_v_default_acl)); /* "borg/platform_linux.pyx":141 * try: * default_acl = acl_from_text(converter(default_text)) * if default_acl: # <<<<<<<<<<<<<< * acl_set_file(p, ACL_TYPE_DEFAULT, default_acl) * finally: */ } } /* "borg/platform_linux.pyx":144 * acl_set_file(p, ACL_TYPE_DEFAULT, default_acl) * finally: * acl_free(default_acl) # <<<<<<<<<<<<<< * * */ /*finally:*/ { /*normal exit:*/{ (void)(acl_free(__pyx_v_default_acl)); goto __pyx_L14; } __pyx_L13_error:; /*exception exit:*/{ __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __pyx_t_15 = 0; __pyx_t_14 = 0; __pyx_t_13 = 0; __pyx_t_12 = 0; __pyx_t_11 = 0; __pyx_t_10 = 0; __Pyx_XDECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_XDECREF(__pyx_t_3); __pyx_t_3 = 0; if (PY_MAJOR_VERSION >= 3) __Pyx_ExceptionSwap(&__pyx_t_12, &__pyx_t_11, &__pyx_t_10); if ((PY_MAJOR_VERSION < 3) || unlikely(__Pyx_GetException(&__pyx_t_15, &__pyx_t_14, &__pyx_t_13) < 0)) __Pyx_ErrFetch(&__pyx_t_15, &__pyx_t_14, &__pyx_t_13); __Pyx_XGOTREF(__pyx_t_15); __Pyx_XGOTREF(__pyx_t_14); __Pyx_XGOTREF(__pyx_t_13); __Pyx_XGOTREF(__pyx_t_12); __Pyx_XGOTREF(__pyx_t_11); __Pyx_XGOTREF(__pyx_t_10); __pyx_t_8 = __pyx_lineno; __pyx_t_7 = __pyx_clineno; __pyx_t_16 = __pyx_filename; { (void)(acl_free(__pyx_v_default_acl)); } if (PY_MAJOR_VERSION >= 3) { __Pyx_XGIVEREF(__pyx_t_12); __Pyx_XGIVEREF(__pyx_t_11); __Pyx_XGIVEREF(__pyx_t_10); __Pyx_ExceptionReset(__pyx_t_12, __pyx_t_11, __pyx_t_10); } __Pyx_XGIVEREF(__pyx_t_15); __Pyx_XGIVEREF(__pyx_t_14); __Pyx_XGIVEREF(__pyx_t_13); __Pyx_ErrRestore(__pyx_t_15, __pyx_t_14, __pyx_t_13); __pyx_t_15 = 0; __pyx_t_14 = 0; __pyx_t_13 = 0; __pyx_t_12 = 0; __pyx_t_11 = 0; __pyx_t_10 = 0; __pyx_lineno = __pyx_t_8; __pyx_clineno = __pyx_t_7; __pyx_filename = __pyx_t_16; goto __pyx_L1_error; } __pyx_L14:; } /* "borg/platform_linux.pyx":138 * finally: * acl_free(access_acl) * if default_text: # <<<<<<<<<<<<<< * try: * default_acl = acl_from_text(converter(default_text)) */ } /* "borg/platform_linux.pyx":115 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ /* function exit code */ __pyx_r = Py_None; __Pyx_INCREF(Py_None); goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_AddTraceback("borg.platform_linux.acl_set", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_p); __Pyx_XDECREF(__pyx_v_converter); __Pyx_XDECREF(__pyx_v_access_text); __Pyx_XDECREF(__pyx_v_default_text); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } /* "borg/platform_linux.pyx":147 * * * def umount(mountpoint): # <<<<<<<<<<<<<< * env = prepare_subprocess_env(system=True) * return subprocess.call(['fusermount', '-u', mountpoint], env=env) */ /* Python wrapper */ static PyObject *__pyx_pw_4borg_14platform_linux_7umount(PyObject *__pyx_self, PyObject *__pyx_v_mountpoint); /*proto*/ static PyMethodDef __pyx_mdef_4borg_14platform_linux_7umount = {"umount", (PyCFunction)__pyx_pw_4borg_14platform_linux_7umount, METH_O, 0}; static PyObject *__pyx_pw_4borg_14platform_linux_7umount(PyObject *__pyx_self, PyObject *__pyx_v_mountpoint) { PyObject *__pyx_r = 0; __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("umount (wrapper)", 0); __pyx_r = __pyx_pf_4borg_14platform_linux_6umount(__pyx_self, ((PyObject *)__pyx_v_mountpoint)); /* function exit code */ __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyObject *__pyx_pf_4borg_14platform_linux_6umount(CYTHON_UNUSED PyObject *__pyx_self, PyObject *__pyx_v_mountpoint) { PyObject *__pyx_v_env = NULL; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclarations PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; PyObject *__pyx_t_4 = NULL; __Pyx_RefNannySetupContext("umount", 0); /* "borg/platform_linux.pyx":148 * * def umount(mountpoint): * env = prepare_subprocess_env(system=True) # <<<<<<<<<<<<<< * return subprocess.call(['fusermount', '-u', mountpoint], env=env) */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_prepare_subprocess_env); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 148, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyDict_NewPresized(1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 148, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_t_2, __pyx_n_s_system, Py_True) < 0) __PYX_ERR(0, 148, __pyx_L1_error) __pyx_t_3 = __Pyx_PyObject_Call(__pyx_t_1, __pyx_empty_tuple, __pyx_t_2); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 148, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_v_env = __pyx_t_3; __pyx_t_3 = 0; /* "borg/platform_linux.pyx":149 * def umount(mountpoint): * env = prepare_subprocess_env(system=True) * return subprocess.call(['fusermount', '-u', mountpoint], env=env) # <<<<<<<<<<<<<< */ __Pyx_XDECREF(__pyx_r); __Pyx_GetModuleGlobalName(__pyx_t_3, __pyx_n_s_subprocess); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 149, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_3, __pyx_n_s_call); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 149, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = PyList_New(3); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 149, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_INCREF(__pyx_n_s_fusermount); __Pyx_GIVEREF(__pyx_n_s_fusermount); PyList_SET_ITEM(__pyx_t_3, 0, __pyx_n_s_fusermount); __Pyx_INCREF(__pyx_kp_s_u); __Pyx_GIVEREF(__pyx_kp_s_u); PyList_SET_ITEM(__pyx_t_3, 1, __pyx_kp_s_u); __Pyx_INCREF(__pyx_v_mountpoint); __Pyx_GIVEREF(__pyx_v_mountpoint); PyList_SET_ITEM(__pyx_t_3, 2, __pyx_v_mountpoint); __pyx_t_1 = PyTuple_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 149, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyDict_NewPresized(1); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 149, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); if (PyDict_SetItem(__pyx_t_3, __pyx_n_s_env, __pyx_v_env) < 0) __PYX_ERR(0, 149, __pyx_L1_error) __pyx_t_4 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_t_1, __pyx_t_3); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 149, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_4); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; __pyx_r = __pyx_t_4; __pyx_t_4 = 0; goto __pyx_L0; /* "borg/platform_linux.pyx":147 * * * def umount(mountpoint): # <<<<<<<<<<<<<< * env = prepare_subprocess_env(system=True) * return subprocess.call(['fusermount', '-u', mountpoint], env=env) */ /* function exit code */ __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); __Pyx_XDECREF(__pyx_t_4); __Pyx_AddTraceback("borg.platform_linux.umount", __pyx_clineno, __pyx_lineno, __pyx_filename); __pyx_r = NULL; __pyx_L0:; __Pyx_XDECREF(__pyx_v_env); __Pyx_XGIVEREF(__pyx_r); __Pyx_RefNannyFinishContext(); return __pyx_r; } static PyMethodDef __pyx_methods[] = { {0, 0, 0, 0} }; #if PY_MAJOR_VERSION >= 3 #if CYTHON_PEP489_MULTI_PHASE_INIT static PyObject* __pyx_pymod_create(PyObject *spec, PyModuleDef *def); /*proto*/ static int __pyx_pymod_exec_platform_linux(PyObject* module); /*proto*/ static PyModuleDef_Slot __pyx_moduledef_slots[] = { {Py_mod_create, (void*)__pyx_pymod_create}, {Py_mod_exec, (void*)__pyx_pymod_exec_platform_linux}, {0, NULL} }; #endif static struct PyModuleDef __pyx_moduledef = { PyModuleDef_HEAD_INIT, "platform_linux", 0, /* m_doc */ #if CYTHON_PEP489_MULTI_PHASE_INIT 0, /* m_size */ #else -1, /* m_size */ #endif __pyx_methods /* m_methods */, #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_moduledef_slots, /* m_slots */ #else NULL, /* m_reload */ #endif NULL, /* m_traverse */ NULL, /* m_clear */ NULL /* m_free */ }; #endif #ifndef CYTHON_SMALL_CODE #if defined(__clang__) #define CYTHON_SMALL_CODE #elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 3)) #define CYTHON_SMALL_CODE __attribute__((cold)) #else #define CYTHON_SMALL_CODE #endif #endif static __Pyx_StringTabEntry __pyx_string_tab[] = { {&__pyx_kp_s_, __pyx_k_, sizeof(__pyx_k_), 0, 0, 1, 0}, {&__pyx_kp_s_1_0_01, __pyx_k_1_0_01, sizeof(__pyx_k_1_0_01), 0, 0, 1, 0}, {&__pyx_n_s_API_VERSION, __pyx_k_API_VERSION, sizeof(__pyx_k_API_VERSION), 0, 0, 1, 1}, {&__pyx_n_s_M, __pyx_k_M, sizeof(__pyx_k_M), 0, 0, 1, 1}, {&__pyx_n_s_S_ISLNK, __pyx_k_S_ISLNK, sizeof(__pyx_k_S_ISLNK), 0, 0, 1, 1}, {&__pyx_kp_s__2, __pyx_k__2, sizeof(__pyx_k__2), 0, 0, 1, 0}, {&__pyx_kp_s__4, __pyx_k__4, sizeof(__pyx_k__4), 0, 0, 1, 0}, {&__pyx_kp_s__5, __pyx_k__5, sizeof(__pyx_k__5), 0, 0, 1, 0}, {&__pyx_n_s_access_acl, __pyx_k_access_acl, sizeof(__pyx_k_access_acl), 0, 0, 1, 1}, {&__pyx_n_s_access_text, __pyx_k_access_text, sizeof(__pyx_k_access_text), 0, 0, 1, 1}, {&__pyx_n_s_acl, __pyx_k_acl, sizeof(__pyx_k_acl), 0, 0, 1, 1}, {&__pyx_n_b_acl_access, __pyx_k_acl_access, sizeof(__pyx_k_acl_access), 0, 0, 0, 1}, {&__pyx_n_b_acl_default, __pyx_k_acl_default, sizeof(__pyx_k_acl_default), 0, 0, 0, 1}, {&__pyx_n_s_acl_get, __pyx_k_acl_get, sizeof(__pyx_k_acl_get), 0, 0, 1, 1}, {&__pyx_n_s_acl_set, __pyx_k_acl_set, sizeof(__pyx_k_acl_set), 0, 0, 1, 1}, {&__pyx_n_s_acl_use_local_uid_gid, __pyx_k_acl_use_local_uid_gid, sizeof(__pyx_k_acl_use_local_uid_gid), 0, 0, 1, 1}, {&__pyx_n_s_borg_platform_linux, __pyx_k_borg_platform_linux, sizeof(__pyx_k_borg_platform_linux), 0, 0, 1, 1}, {&__pyx_kp_s_borg_platform_linux_pyx, __pyx_k_borg_platform_linux_pyx, sizeof(__pyx_k_borg_platform_linux_pyx), 0, 0, 1, 0}, {&__pyx_n_s_call, __pyx_k_call, sizeof(__pyx_k_call), 0, 0, 1, 1}, {&__pyx_n_s_cline_in_traceback, __pyx_k_cline_in_traceback, sizeof(__pyx_k_cline_in_traceback), 0, 0, 1, 1}, {&__pyx_n_s_comment_re, __pyx_k_comment_re, sizeof(__pyx_k_comment_re), 0, 0, 1, 1}, {&__pyx_n_s_compile, __pyx_k_compile, sizeof(__pyx_k_compile), 0, 0, 1, 1}, {&__pyx_n_s_converter, __pyx_k_converter, sizeof(__pyx_k_converter), 0, 0, 1, 1}, {&__pyx_n_s_default_acl, __pyx_k_default_acl, sizeof(__pyx_k_default_acl), 0, 0, 1, 1}, {&__pyx_n_s_default_text, __pyx_k_default_text, sizeof(__pyx_k_default_text), 0, 0, 1, 1}, {&__pyx_n_s_entries, __pyx_k_entries, sizeof(__pyx_k_entries), 0, 0, 1, 1}, {&__pyx_n_s_entry, __pyx_k_entry, sizeof(__pyx_k_entry), 0, 0, 1, 1}, {&__pyx_n_s_env, __pyx_k_env, sizeof(__pyx_k_env), 0, 0, 1, 1}, {&__pyx_n_s_fields, __pyx_k_fields, sizeof(__pyx_k_fields), 0, 0, 1, 1}, {&__pyx_n_s_fsencode, __pyx_k_fsencode, sizeof(__pyx_k_fsencode), 0, 0, 1, 1}, {&__pyx_n_s_fusermount, __pyx_k_fusermount, sizeof(__pyx_k_fusermount), 0, 0, 1, 1}, {&__pyx_n_s_get, __pyx_k_get, sizeof(__pyx_k_get), 0, 0, 1, 1}, {&__pyx_n_s_group, __pyx_k_group, sizeof(__pyx_k_group), 0, 0, 1, 1}, {&__pyx_n_s_group2gid, __pyx_k_group2gid, sizeof(__pyx_k_group2gid), 0, 0, 1, 1}, {&__pyx_n_s_helpers, __pyx_k_helpers, sizeof(__pyx_k_helpers), 0, 0, 1, 1}, {&__pyx_n_s_import, __pyx_k_import, sizeof(__pyx_k_import), 0, 0, 1, 1}, {&__pyx_n_s_item, __pyx_k_item, sizeof(__pyx_k_item), 0, 0, 1, 1}, {&__pyx_n_s_join, __pyx_k_join, sizeof(__pyx_k_join), 0, 0, 1, 1}, {&__pyx_n_s_main, __pyx_k_main, sizeof(__pyx_k_main), 0, 0, 1, 1}, {&__pyx_n_s_mountpoint, __pyx_k_mountpoint, sizeof(__pyx_k_mountpoint), 0, 0, 1, 1}, {&__pyx_n_s_name, __pyx_k_name, sizeof(__pyx_k_name), 0, 0, 1, 1}, {&__pyx_n_s_numeric_owner, __pyx_k_numeric_owner, sizeof(__pyx_k_numeric_owner), 0, 0, 1, 1}, {&__pyx_n_s_os, __pyx_k_os, sizeof(__pyx_k_os), 0, 0, 1, 1}, {&__pyx_n_s_p, __pyx_k_p, sizeof(__pyx_k_p), 0, 0, 1, 1}, {&__pyx_n_s_path, __pyx_k_path, sizeof(__pyx_k_path), 0, 0, 1, 1}, {&__pyx_n_s_posix_acl_use_stored_uid_gid, __pyx_k_posix_acl_use_stored_uid_gid, sizeof(__pyx_k_posix_acl_use_stored_uid_gid), 0, 0, 1, 1}, {&__pyx_n_s_prepare_subprocess_env, __pyx_k_prepare_subprocess_env, sizeof(__pyx_k_prepare_subprocess_env), 0, 0, 1, 1}, {&__pyx_n_s_re, __pyx_k_re, sizeof(__pyx_k_re), 0, 0, 1, 1}, {&__pyx_n_s_safe_decode, __pyx_k_safe_decode, sizeof(__pyx_k_safe_decode), 0, 0, 1, 1}, {&__pyx_n_s_safe_encode, __pyx_k_safe_encode, sizeof(__pyx_k_safe_encode), 0, 0, 1, 1}, {&__pyx_n_s_split, __pyx_k_split, sizeof(__pyx_k_split), 0, 0, 1, 1}, {&__pyx_n_s_st, __pyx_k_st, sizeof(__pyx_k_st), 0, 0, 1, 1}, {&__pyx_n_s_st_mode, __pyx_k_st_mode, sizeof(__pyx_k_st_mode), 0, 0, 1, 1}, {&__pyx_n_s_stat, __pyx_k_stat, sizeof(__pyx_k_stat), 0, 0, 1, 1}, {&__pyx_n_s_sub, __pyx_k_sub, sizeof(__pyx_k_sub), 0, 0, 1, 1}, {&__pyx_n_s_subprocess, __pyx_k_subprocess, sizeof(__pyx_k_subprocess), 0, 0, 1, 1}, {&__pyx_n_s_system, __pyx_k_system, sizeof(__pyx_k_system), 0, 0, 1, 1}, {&__pyx_n_s_test, __pyx_k_test, sizeof(__pyx_k_test), 0, 0, 1, 1}, {&__pyx_kp_s_u, __pyx_k_u, sizeof(__pyx_k_u), 0, 0, 1, 0}, {&__pyx_n_s_umount, __pyx_k_umount, sizeof(__pyx_k_umount), 0, 0, 1, 1}, {&__pyx_n_s_user, __pyx_k_user, sizeof(__pyx_k_user), 0, 0, 1, 1}, {&__pyx_n_s_user2uid, __pyx_k_user2uid, sizeof(__pyx_k_user2uid), 0, 0, 1, 1}, {0, 0, 0, 0, 0, 0, 0} }; static CYTHON_SMALL_CODE int __Pyx_InitCachedBuiltins(void) { return 0; } static CYTHON_SMALL_CODE int __Pyx_InitCachedConstants(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_InitCachedConstants", 0); /* "borg/platform_linux.pyx":42 * elif fields[0] == 'group' and fields[1]: * fields[1] = str(group2gid(fields[1], fields[3])) * entries.append(':'.join(fields[:3])) # <<<<<<<<<<<<<< * return safe_encode('\n'.join(entries)) * */ __pyx_slice__3 = PySlice_New(Py_None, __pyx_int_3, Py_None); if (unlikely(!__pyx_slice__3)) __PYX_ERR(0, 42, __pyx_L1_error) __Pyx_GOTREF(__pyx_slice__3); __Pyx_GIVEREF(__pyx_slice__3); /* "borg/platform_linux.pyx":31 * * * def acl_use_local_uid_gid(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ __pyx_tuple__6 = PyTuple_Pack(4, __pyx_n_s_acl, __pyx_n_s_entries, __pyx_n_s_entry, __pyx_n_s_fields); if (unlikely(!__pyx_tuple__6)) __PYX_ERR(0, 31, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__6); __Pyx_GIVEREF(__pyx_tuple__6); __pyx_codeobj__7 = (PyObject*)__Pyx_PyCode_New(1, 0, 4, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__6, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_linux_pyx, __pyx_n_s_acl_use_local_uid_gid, 31, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__7)) __PYX_ERR(0, 31, __pyx_L1_error) /* "borg/platform_linux.pyx":80 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ __pyx_tuple__8 = PyTuple_Pack(10, __pyx_n_s_path, __pyx_n_s_item, __pyx_n_s_st, __pyx_n_s_numeric_owner, __pyx_n_s_default_acl, __pyx_n_s_access_acl, __pyx_n_s_default_text, __pyx_n_s_access_text, __pyx_n_s_p, __pyx_n_s_converter); if (unlikely(!__pyx_tuple__8)) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__8); __Pyx_GIVEREF(__pyx_tuple__8); __pyx_codeobj__9 = (PyObject*)__Pyx_PyCode_New(4, 0, 10, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__8, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_linux_pyx, __pyx_n_s_acl_get, 80, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__9)) __PYX_ERR(0, 80, __pyx_L1_error) /* "borg/platform_linux.pyx":115 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ __pyx_tuple__10 = PyTuple_Pack(9, __pyx_n_s_path, __pyx_n_s_item, __pyx_n_s_numeric_owner, __pyx_n_s_access_acl, __pyx_n_s_default_acl, __pyx_n_s_p, __pyx_n_s_converter, __pyx_n_s_access_text, __pyx_n_s_default_text); if (unlikely(!__pyx_tuple__10)) __PYX_ERR(0, 115, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__10); __Pyx_GIVEREF(__pyx_tuple__10); __pyx_codeobj__11 = (PyObject*)__Pyx_PyCode_New(3, 0, 9, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__10, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_linux_pyx, __pyx_n_s_acl_set, 115, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__11)) __PYX_ERR(0, 115, __pyx_L1_error) /* "borg/platform_linux.pyx":147 * * * def umount(mountpoint): # <<<<<<<<<<<<<< * env = prepare_subprocess_env(system=True) * return subprocess.call(['fusermount', '-u', mountpoint], env=env) */ __pyx_tuple__12 = PyTuple_Pack(2, __pyx_n_s_mountpoint, __pyx_n_s_env); if (unlikely(!__pyx_tuple__12)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_tuple__12); __Pyx_GIVEREF(__pyx_tuple__12); __pyx_codeobj__13 = (PyObject*)__Pyx_PyCode_New(1, 0, 2, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__12, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_borg_platform_linux_pyx, __pyx_n_s_umount, 147, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__13)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_RefNannyFinishContext(); return 0; __pyx_L1_error:; __Pyx_RefNannyFinishContext(); return -1; } static CYTHON_SMALL_CODE int __Pyx_InitGlobals(void) { if (__Pyx_InitStrings(__pyx_string_tab) < 0) __PYX_ERR(0, 1, __pyx_L1_error); __pyx_int_3 = PyInt_FromLong(3); if (unlikely(!__pyx_int_3)) __PYX_ERR(0, 1, __pyx_L1_error) return 0; __pyx_L1_error:; return -1; } static CYTHON_SMALL_CODE int __Pyx_modinit_global_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_export_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_init_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_type_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_variable_import_code(void); /*proto*/ static CYTHON_SMALL_CODE int __Pyx_modinit_function_import_code(void); /*proto*/ static int __Pyx_modinit_global_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_global_init_code", 0); /*--- Global init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_export_code", 0); /*--- Variable export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_export_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_export_code", 0); /*--- Function export code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_init_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_init_code", 0); /*--- Type init code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_type_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_type_import_code", 0); /*--- Type import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_variable_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_variable_import_code", 0); /*--- Variable import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } static int __Pyx_modinit_function_import_code(void) { __Pyx_RefNannyDeclarations __Pyx_RefNannySetupContext("__Pyx_modinit_function_import_code", 0); /*--- Function import code ---*/ __Pyx_RefNannyFinishContext(); return 0; } #if PY_MAJOR_VERSION < 3 #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC void #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #else #ifdef CYTHON_NO_PYINIT_EXPORT #define __Pyx_PyMODINIT_FUNC PyObject * #else #define __Pyx_PyMODINIT_FUNC PyMODINIT_FUNC #endif #endif #if PY_MAJOR_VERSION < 3 __Pyx_PyMODINIT_FUNC initplatform_linux(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC initplatform_linux(void) #else __Pyx_PyMODINIT_FUNC PyInit_platform_linux(void) CYTHON_SMALL_CODE; /*proto*/ __Pyx_PyMODINIT_FUNC PyInit_platform_linux(void) #if CYTHON_PEP489_MULTI_PHASE_INIT { return PyModuleDef_Init(&__pyx_moduledef); } static CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) { #if PY_VERSION_HEX >= 0x030700A1 static PY_INT64_T main_interpreter_id = -1; PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()->interp); if (main_interpreter_id == -1) { main_interpreter_id = current_id; return (unlikely(current_id == -1)) ? -1 : 0; } else if (unlikely(main_interpreter_id != current_id)) #else static PyInterpreterState *main_interpreter = NULL; PyInterpreterState *current_interpreter = PyThreadState_Get()->interp; if (!main_interpreter) { main_interpreter = current_interpreter; } else if (unlikely(main_interpreter != current_interpreter)) #endif { PyErr_SetString( PyExc_ImportError, "Interpreter change detected - this module can only be loaded into one interpreter per process."); return -1; } return 0; } static CYTHON_SMALL_CODE int __Pyx_copy_spec_to_module(PyObject *spec, PyObject *moddict, const char* from_name, const char* to_name) { PyObject *value = PyObject_GetAttrString(spec, from_name); int result = 0; if (likely(value)) { result = PyDict_SetItemString(moddict, to_name, value); Py_DECREF(value); } else if (PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Clear(); } else { result = -1; } return result; } static CYTHON_SMALL_CODE PyObject* __pyx_pymod_create(PyObject *spec, CYTHON_UNUSED PyModuleDef *def) { PyObject *module = NULL, *moddict, *modname; if (__Pyx_check_single_interpreter()) return NULL; if (__pyx_m) return __Pyx_NewRef(__pyx_m); modname = PyObject_GetAttrString(spec, "name"); if (unlikely(!modname)) goto bad; module = PyModule_NewObject(modname); Py_DECREF(modname); if (unlikely(!module)) goto bad; moddict = PyModule_GetDict(module); if (unlikely(!moddict)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "loader", "__loader__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "origin", "__file__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "parent", "__package__") < 0)) goto bad; if (unlikely(__Pyx_copy_spec_to_module(spec, moddict, "submodule_search_locations", "__path__") < 0)) goto bad; return module; bad: Py_XDECREF(module); return NULL; } static CYTHON_SMALL_CODE int __pyx_pymod_exec_platform_linux(PyObject *__pyx_pyinit_module) #endif #endif { PyObject *__pyx_t_1 = NULL; PyObject *__pyx_t_2 = NULL; PyObject *__pyx_t_3 = NULL; __Pyx_RefNannyDeclarations #if CYTHON_PEP489_MULTI_PHASE_INIT if (__pyx_m) { if (__pyx_m == __pyx_pyinit_module) return 0; PyErr_SetString(PyExc_RuntimeError, "Module 'platform_linux' has already been imported. Re-initialisation is not supported."); return -1; } #elif PY_MAJOR_VERSION >= 3 if (__pyx_m) return __Pyx_NewRef(__pyx_m); #endif #if CYTHON_REFNANNY __Pyx_RefNanny = __Pyx_RefNannyImportAPI("refnanny"); if (!__Pyx_RefNanny) { PyErr_Clear(); __Pyx_RefNanny = __Pyx_RefNannyImportAPI("Cython.Runtime.refnanny"); if (!__Pyx_RefNanny) Py_FatalError("failed to import 'refnanny' module"); } #endif __Pyx_RefNannySetupContext("__Pyx_PyMODINIT_FUNC PyInit_platform_linux(void)", 0); if (__Pyx_check_binary_version() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pxy_PyFrame_Initialize_Offsets __Pxy_PyFrame_Initialize_Offsets(); #endif __pyx_empty_tuple = PyTuple_New(0); if (unlikely(!__pyx_empty_tuple)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_bytes = PyBytes_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_bytes)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_empty_unicode = PyUnicode_FromStringAndSize("", 0); if (unlikely(!__pyx_empty_unicode)) __PYX_ERR(0, 1, __pyx_L1_error) #ifdef __Pyx_CyFunction_USED if (__pyx_CyFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_FusedFunction_USED if (__pyx_FusedFunction_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Coroutine_USED if (__pyx_Coroutine_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_Generator_USED if (__pyx_Generator_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_AsyncGen_USED if (__pyx_AsyncGen_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif #ifdef __Pyx_StopAsyncIteration_USED if (__pyx_StopAsyncIteration_init() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /*--- Library function declarations ---*/ /*--- Threads initialization code ---*/ #if defined(__PYX_FORCE_INIT_THREADS) && __PYX_FORCE_INIT_THREADS #ifdef WITH_THREAD /* Python build with threading support? */ PyEval_InitThreads(); #endif #endif /*--- Module creation code ---*/ #if CYTHON_PEP489_MULTI_PHASE_INIT __pyx_m = __pyx_pyinit_module; Py_INCREF(__pyx_m); #else #if PY_MAJOR_VERSION < 3 __pyx_m = Py_InitModule4("platform_linux", __pyx_methods, 0, 0, PYTHON_API_VERSION); Py_XINCREF(__pyx_m); #else __pyx_m = PyModule_Create(&__pyx_moduledef); #endif if (unlikely(!__pyx_m)) __PYX_ERR(0, 1, __pyx_L1_error) #endif __pyx_d = PyModule_GetDict(__pyx_m); if (unlikely(!__pyx_d)) __PYX_ERR(0, 1, __pyx_L1_error) Py_INCREF(__pyx_d); __pyx_b = PyImport_AddModule(__Pyx_BUILTIN_MODULE_NAME); if (unlikely(!__pyx_b)) __PYX_ERR(0, 1, __pyx_L1_error) __pyx_cython_runtime = PyImport_AddModule((char *) "cython_runtime"); if (unlikely(!__pyx_cython_runtime)) __PYX_ERR(0, 1, __pyx_L1_error) #if CYTHON_COMPILING_IN_PYPY Py_INCREF(__pyx_b); #endif if (PyObject_SetAttrString(__pyx_m, "__builtins__", __pyx_b) < 0) __PYX_ERR(0, 1, __pyx_L1_error); /*--- Initialize various global constants etc. ---*/ if (__Pyx_InitGlobals() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #if PY_MAJOR_VERSION < 3 && (__PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT) if (__Pyx_init_sys_getdefaultencoding_params() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif if (__pyx_module_is_main_borg__platform_linux) { if (PyObject_SetAttr(__pyx_m, __pyx_n_s_name, __pyx_n_s_main) < 0) __PYX_ERR(0, 1, __pyx_L1_error) } #if PY_MAJOR_VERSION >= 3 { PyObject *modules = PyImport_GetModuleDict(); if (unlikely(!modules)) __PYX_ERR(0, 1, __pyx_L1_error) if (!PyDict_GetItemString(modules, "borg.platform_linux")) { if (unlikely(PyDict_SetItemString(modules, "borg.platform_linux", __pyx_m) < 0)) __PYX_ERR(0, 1, __pyx_L1_error) } } #endif /*--- Builtin init code ---*/ if (__Pyx_InitCachedBuiltins() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Constants init code ---*/ if (__Pyx_InitCachedConstants() < 0) __PYX_ERR(0, 1, __pyx_L1_error) /*--- Global type/function init code ---*/ (void)__Pyx_modinit_global_init_code(); (void)__Pyx_modinit_variable_export_code(); (void)__Pyx_modinit_function_export_code(); (void)__Pyx_modinit_type_init_code(); (void)__Pyx_modinit_type_import_code(); (void)__Pyx_modinit_variable_import_code(); (void)__Pyx_modinit_function_import_code(); /*--- Execution code ---*/ #if defined(__Pyx_Generator_USED) || defined(__Pyx_Coroutine_USED) if (__Pyx_patch_abc() < 0) __PYX_ERR(0, 1, __pyx_L1_error) #endif /* "borg/platform_linux.pyx":1 * import os # <<<<<<<<<<<<<< * import re * import subprocess */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_os, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_os, __pyx_t_1) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":2 * import os * import re # <<<<<<<<<<<<<< * import subprocess * from stat import S_ISLNK */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_re, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_re, __pyx_t_1) < 0) __PYX_ERR(0, 2, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":3 * import os * import re * import subprocess # <<<<<<<<<<<<<< * from stat import S_ISLNK * from .helpers import posix_acl_use_stored_uid_gid, user2uid, group2gid, safe_decode, safe_encode, prepare_subprocess_env */ __pyx_t_1 = __Pyx_Import(__pyx_n_s_subprocess, 0, -1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 3, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_subprocess, __pyx_t_1) < 0) __PYX_ERR(0, 3, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":4 * import re * import subprocess * from stat import S_ISLNK # <<<<<<<<<<<<<< * from .helpers import posix_acl_use_stored_uid_gid, user2uid, group2gid, safe_decode, safe_encode, prepare_subprocess_env * */ __pyx_t_1 = PyList_New(1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_n_s_S_ISLNK); __Pyx_GIVEREF(__pyx_n_s_S_ISLNK); PyList_SET_ITEM(__pyx_t_1, 0, __pyx_n_s_S_ISLNK); __pyx_t_2 = __Pyx_Import(__pyx_n_s_stat, __pyx_t_1, -1); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = __Pyx_ImportFrom(__pyx_t_2, __pyx_n_s_S_ISLNK); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 4, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); if (PyDict_SetItem(__pyx_d, __pyx_n_s_S_ISLNK, __pyx_t_1) < 0) __PYX_ERR(0, 4, __pyx_L1_error) __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; /* "borg/platform_linux.pyx":5 * import subprocess * from stat import S_ISLNK * from .helpers import posix_acl_use_stored_uid_gid, user2uid, group2gid, safe_decode, safe_encode, prepare_subprocess_env # <<<<<<<<<<<<<< * * API_VERSION = '1.0_01' */ __pyx_t_2 = PyList_New(6); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_INCREF(__pyx_n_s_posix_acl_use_stored_uid_gid); __Pyx_GIVEREF(__pyx_n_s_posix_acl_use_stored_uid_gid); PyList_SET_ITEM(__pyx_t_2, 0, __pyx_n_s_posix_acl_use_stored_uid_gid); __Pyx_INCREF(__pyx_n_s_user2uid); __Pyx_GIVEREF(__pyx_n_s_user2uid); PyList_SET_ITEM(__pyx_t_2, 1, __pyx_n_s_user2uid); __Pyx_INCREF(__pyx_n_s_group2gid); __Pyx_GIVEREF(__pyx_n_s_group2gid); PyList_SET_ITEM(__pyx_t_2, 2, __pyx_n_s_group2gid); __Pyx_INCREF(__pyx_n_s_safe_decode); __Pyx_GIVEREF(__pyx_n_s_safe_decode); PyList_SET_ITEM(__pyx_t_2, 3, __pyx_n_s_safe_decode); __Pyx_INCREF(__pyx_n_s_safe_encode); __Pyx_GIVEREF(__pyx_n_s_safe_encode); PyList_SET_ITEM(__pyx_t_2, 4, __pyx_n_s_safe_encode); __Pyx_INCREF(__pyx_n_s_prepare_subprocess_env); __Pyx_GIVEREF(__pyx_n_s_prepare_subprocess_env); PyList_SET_ITEM(__pyx_t_2, 5, __pyx_n_s_prepare_subprocess_env); __pyx_t_1 = __Pyx_Import(__pyx_n_s_helpers, __pyx_t_2, 1); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_1, __pyx_n_s_posix_acl_use_stored_uid_gid); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_posix_acl_use_stored_uid_gid, __pyx_t_2) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_1, __pyx_n_s_user2uid); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_user2uid, __pyx_t_2) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_1, __pyx_n_s_group2gid); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_group2gid, __pyx_t_2) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_1, __pyx_n_s_safe_decode); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_safe_decode, __pyx_t_2) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_1, __pyx_n_s_safe_encode); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_safe_encode, __pyx_t_2) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __pyx_t_2 = __Pyx_ImportFrom(__pyx_t_1, __pyx_n_s_prepare_subprocess_env); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); if (PyDict_SetItem(__pyx_d, __pyx_n_s_prepare_subprocess_env, __pyx_t_2) < 0) __PYX_ERR(0, 5, __pyx_L1_error) __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; /* "borg/platform_linux.pyx":7 * from .helpers import posix_acl_use_stored_uid_gid, user2uid, group2gid, safe_decode, safe_encode, prepare_subprocess_env * * API_VERSION = '1.0_01' # <<<<<<<<<<<<<< * * cdef extern from "sys/types.h": */ if (PyDict_SetItem(__pyx_d, __pyx_n_s_API_VERSION, __pyx_kp_s_1_0_01) < 0) __PYX_ERR(0, 7, __pyx_L1_error) /* "borg/platform_linux.pyx":28 * * * _comment_re = re.compile(' *#.*', re.M) # <<<<<<<<<<<<<< * * */ __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_re); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_2 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_compile); if (unlikely(!__pyx_t_2)) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_2); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __Pyx_GetModuleGlobalName(__pyx_t_1, __pyx_n_s_re); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __pyx_t_3 = __Pyx_PyObject_GetAttrStr(__pyx_t_1, __pyx_n_s_M); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; __pyx_t_1 = PyTuple_New(2); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_1); __Pyx_INCREF(__pyx_kp_s__5); __Pyx_GIVEREF(__pyx_kp_s__5); PyTuple_SET_ITEM(__pyx_t_1, 0, __pyx_kp_s__5); __Pyx_GIVEREF(__pyx_t_3); PyTuple_SET_ITEM(__pyx_t_1, 1, __pyx_t_3); __pyx_t_3 = 0; __pyx_t_3 = __Pyx_PyObject_Call(__pyx_t_2, __pyx_t_1, NULL); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0; if (PyDict_SetItem(__pyx_d, __pyx_n_s_comment_re, __pyx_t_3) < 0) __PYX_ERR(0, 28, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":31 * * * def acl_use_local_uid_gid(acl): # <<<<<<<<<<<<<< * """Replace the user/group field with the local uid/gid if possible * """ */ __pyx_t_3 = PyCFunction_NewEx(&__pyx_mdef_4borg_14platform_linux_1acl_use_local_uid_gid, NULL, __pyx_n_s_borg_platform_linux); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 31, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_use_local_uid_gid, __pyx_t_3) < 0) __PYX_ERR(0, 31, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":80 * * * def acl_get(path, item, st, numeric_owner=False): # <<<<<<<<<<<<<< * """Saves ACL Entries * */ __pyx_t_3 = PyCFunction_NewEx(&__pyx_mdef_4borg_14platform_linux_3acl_get, NULL, __pyx_n_s_borg_platform_linux); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_get, __pyx_t_3) < 0) __PYX_ERR(0, 80, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":115 * * * def acl_set(path, item, numeric_owner=False): # <<<<<<<<<<<<<< * """Restore ACL Entries * */ __pyx_t_3 = PyCFunction_NewEx(&__pyx_mdef_4borg_14platform_linux_5acl_set, NULL, __pyx_n_s_borg_platform_linux); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 115, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); if (PyDict_SetItem(__pyx_d, __pyx_n_s_acl_set, __pyx_t_3) < 0) __PYX_ERR(0, 115, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":147 * * * def umount(mountpoint): # <<<<<<<<<<<<<< * env = prepare_subprocess_env(system=True) * return subprocess.call(['fusermount', '-u', mountpoint], env=env) */ __pyx_t_3 = PyCFunction_NewEx(&__pyx_mdef_4borg_14platform_linux_7umount, NULL, __pyx_n_s_borg_platform_linux); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); if (PyDict_SetItem(__pyx_d, __pyx_n_s_umount, __pyx_t_3) < 0) __PYX_ERR(0, 147, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /* "borg/platform_linux.pyx":1 * import os # <<<<<<<<<<<<<< * import re * import subprocess */ __pyx_t_3 = __Pyx_PyDict_NewPresized(0); if (unlikely(!__pyx_t_3)) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_GOTREF(__pyx_t_3); if (PyDict_SetItem(__pyx_d, __pyx_n_s_test, __pyx_t_3) < 0) __PYX_ERR(0, 1, __pyx_L1_error) __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; /*--- Wrapped vars code ---*/ goto __pyx_L0; __pyx_L1_error:; __Pyx_XDECREF(__pyx_t_1); __Pyx_XDECREF(__pyx_t_2); __Pyx_XDECREF(__pyx_t_3); if (__pyx_m) { if (__pyx_d) { __Pyx_AddTraceback("init borg.platform_linux", __pyx_clineno, __pyx_lineno, __pyx_filename); } Py_CLEAR(__pyx_m); } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_ImportError, "init borg.platform_linux"); } __pyx_L0:; __Pyx_RefNannyFinishContext(); #if CYTHON_PEP489_MULTI_PHASE_INIT return (__pyx_m != NULL) ? 0 : -1; #elif PY_MAJOR_VERSION >= 3 return __pyx_m; #else return; #endif } /* --- Runtime support code --- */ /* Refnanny */ #if CYTHON_REFNANNY static __Pyx_RefNannyAPIStruct *__Pyx_RefNannyImportAPI(const char *modname) { PyObject *m = NULL, *p = NULL; void *r = NULL; m = PyImport_ImportModule(modname); if (!m) goto end; p = PyObject_GetAttrString(m, "RefNannyAPI"); if (!p) goto end; r = PyLong_AsVoidPtr(p); end: Py_XDECREF(p); Py_XDECREF(m); return (__Pyx_RefNannyAPIStruct *)r; } #endif /* PyObjectGetAttrStr */ #if CYTHON_USE_TYPE_SLOTS static CYTHON_INLINE PyObject* __Pyx_PyObject_GetAttrStr(PyObject* obj, PyObject* attr_name) { PyTypeObject* tp = Py_TYPE(obj); if (likely(tp->tp_getattro)) return tp->tp_getattro(obj, attr_name); #if PY_MAJOR_VERSION < 3 if (likely(tp->tp_getattr)) return tp->tp_getattr(obj, PyString_AS_STRING(attr_name)); #endif return PyObject_GetAttr(obj, attr_name); } #endif /* GetBuiltinName */ static PyObject *__Pyx_GetBuiltinName(PyObject *name) { PyObject* result = __Pyx_PyObject_GetAttrStr(__pyx_b, name); if (unlikely(!result)) { PyErr_Format(PyExc_NameError, #if PY_MAJOR_VERSION >= 3 "name '%U' is not defined", name); #else "name '%.200s' is not defined", PyString_AS_STRING(name)); #endif } return result; } /* GetModuleGlobalName */ #if CYTHON_USE_DICT_VERSIONS static PyObject *__Pyx__GetModuleGlobalName(PyObject *name, PY_UINT64_T *dict_version, PyObject **dict_cached_value) #else static CYTHON_INLINE PyObject *__Pyx__GetModuleGlobalName(PyObject *name) #endif { PyObject *result; #if !CYTHON_AVOID_BORROWED_REFS #if CYTHON_COMPILING_IN_CPYTHON && PY_VERSION_HEX >= 0x030500A1 result = _PyDict_GetItem_KnownHash(__pyx_d, name, ((PyASCIIObject *) name)->hash); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } else if (unlikely(PyErr_Occurred())) { return NULL; } #else result = PyDict_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } #endif #else result = PyObject_GetItem(__pyx_d, name); __PYX_UPDATE_DICT_CACHE(__pyx_d, result, *dict_cached_value, *dict_version) if (likely(result)) { return __Pyx_NewRef(result); } PyErr_Clear(); #endif return __Pyx_GetBuiltinName(name); } /* PyCFunctionFastCall */ #if CYTHON_FAST_PYCCALL static CYTHON_INLINE PyObject * __Pyx_PyCFunction_FastCall(PyObject *func_obj, PyObject **args, Py_ssize_t nargs) { PyCFunctionObject *func = (PyCFunctionObject*)func_obj; PyCFunction meth = PyCFunction_GET_FUNCTION(func); PyObject *self = PyCFunction_GET_SELF(func); int flags = PyCFunction_GET_FLAGS(func); assert(PyCFunction_Check(func)); assert(METH_FASTCALL == (flags & ~(METH_CLASS | METH_STATIC | METH_COEXIST | METH_KEYWORDS | METH_STACKLESS))); assert(nargs >= 0); assert(nargs == 0 || args != NULL); /* _PyCFunction_FastCallDict() must not be called with an exception set, because it may clear it (directly or indirectly) and so the caller loses its exception */ assert(!PyErr_Occurred()); if ((PY_VERSION_HEX < 0x030700A0) || unlikely(flags & METH_KEYWORDS)) { return (*((__Pyx_PyCFunctionFastWithKeywords)(void*)meth)) (self, args, nargs, NULL); } else { return (*((__Pyx_PyCFunctionFast)(void*)meth)) (self, args, nargs); } } #endif /* PyFunctionFastCall */ #if CYTHON_FAST_PYCALL static PyObject* __Pyx_PyFunction_FastCallNoKw(PyCodeObject *co, PyObject **args, Py_ssize_t na, PyObject *globals) { PyFrameObject *f; PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject **fastlocals; Py_ssize_t i; PyObject *result; assert(globals != NULL); /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) { return NULL; } fastlocals = __Pyx_PyFrame_GetLocalsplus(f); for (i = 0; i < na; i++) { Py_INCREF(*args); fastlocals[i] = *args++; } result = PyEval_EvalFrameEx(f,0); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; return result; } #if 1 || PY_VERSION_HEX < 0x030600B1 static PyObject *__Pyx_PyFunction_FastCallDict(PyObject *func, PyObject **args, int nargs, PyObject *kwargs) { PyCodeObject *co = (PyCodeObject *)PyFunction_GET_CODE(func); PyObject *globals = PyFunction_GET_GLOBALS(func); PyObject *argdefs = PyFunction_GET_DEFAULTS(func); PyObject *closure; #if PY_MAJOR_VERSION >= 3 PyObject *kwdefs; #endif PyObject *kwtuple, **k; PyObject **d; Py_ssize_t nd; Py_ssize_t nk; PyObject *result; assert(kwargs == NULL || PyDict_Check(kwargs)); nk = kwargs ? PyDict_Size(kwargs) : 0; if (Py_EnterRecursiveCall((char*)" while calling a Python object")) { return NULL; } if ( #if PY_MAJOR_VERSION >= 3 co->co_kwonlyargcount == 0 && #endif likely(kwargs == NULL || nk == 0) && co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) { if (argdefs == NULL && co->co_argcount == nargs) { result = __Pyx_PyFunction_FastCallNoKw(co, args, nargs, globals); goto done; } else if (nargs == 0 && argdefs != NULL && co->co_argcount == Py_SIZE(argdefs)) { /* function called with no arguments, but all parameters have a default value: use default values as arguments .*/ args = &PyTuple_GET_ITEM(argdefs, 0); result =__Pyx_PyFunction_FastCallNoKw(co, args, Py_SIZE(argdefs), globals); goto done; } } if (kwargs != NULL) { Py_ssize_t pos, i; kwtuple = PyTuple_New(2 * nk); if (kwtuple == NULL) { result = NULL; goto done; } k = &PyTuple_GET_ITEM(kwtuple, 0); pos = i = 0; while (PyDict_Next(kwargs, &pos, &k[i], &k[i+1])) { Py_INCREF(k[i]); Py_INCREF(k[i+1]); i += 2; } nk = i / 2; } else { kwtuple = NULL; k = NULL; } closure = PyFunction_GET_CLOSURE(func); #if PY_MAJOR_VERSION >= 3 kwdefs = PyFunction_GET_KW_DEFAULTS(func); #endif if (argdefs != NULL) { d = &PyTuple_GET_ITEM(argdefs, 0); nd = Py_SIZE(argdefs); } else { d = NULL; nd = 0; } #if PY_MAJOR_VERSION >= 3 result = PyEval_EvalCodeEx((PyObject*)co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, kwdefs, closure); #else result = PyEval_EvalCodeEx(co, globals, (PyObject *)NULL, args, nargs, k, (int)nk, d, (int)nd, closure); #endif Py_XDECREF(kwtuple); done: Py_LeaveRecursiveCall(); return result; } #endif #endif /* PyObjectCall */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) { PyObject *result; ternaryfunc call = func->ob_type->tp_call; if (unlikely(!call)) return PyObject_Call(func, arg, kw); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = (*call)(func, arg, kw); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCall2Args */ static CYTHON_UNUSED PyObject* __Pyx_PyObject_Call2Args(PyObject* function, PyObject* arg1, PyObject* arg2) { PyObject *args, *result = NULL; #if CYTHON_FAST_PYCALL if (PyFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyFunction_FastCall(function, args, 2); } #endif #if CYTHON_FAST_PYCCALL if (__Pyx_PyFastCFunction_Check(function)) { PyObject *args[2] = {arg1, arg2}; return __Pyx_PyCFunction_FastCall(function, args, 2); } #endif args = PyTuple_New(2); if (unlikely(!args)) goto done; Py_INCREF(arg1); PyTuple_SET_ITEM(args, 0, arg1); Py_INCREF(arg2); PyTuple_SET_ITEM(args, 1, arg2); Py_INCREF(function); result = __Pyx_PyObject_Call(function, args, NULL); Py_DECREF(args); Py_DECREF(function); done: return result; } /* PyObjectCallMethO */ #if CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyObject_CallMethO(PyObject *func, PyObject *arg) { PyObject *self, *result; PyCFunction cfunc; cfunc = PyCFunction_GET_FUNCTION(func); self = PyCFunction_GET_SELF(func); if (unlikely(Py_EnterRecursiveCall((char*)" while calling a Python object"))) return NULL; result = cfunc(self, arg); Py_LeaveRecursiveCall(); if (unlikely(!result) && unlikely(!PyErr_Occurred())) { PyErr_SetString( PyExc_SystemError, "NULL result without error in PyObject_Call"); } return result; } #endif /* PyObjectCallOneArg */ #if CYTHON_COMPILING_IN_CPYTHON static PyObject* __Pyx__PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_New(1); if (unlikely(!args)) return NULL; Py_INCREF(arg); PyTuple_SET_ITEM(args, 0, arg); result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { #if CYTHON_FAST_PYCALL if (PyFunction_Check(func)) { return __Pyx_PyFunction_FastCall(func, &arg, 1); } #endif if (likely(PyCFunction_Check(func))) { if (likely(PyCFunction_GET_FLAGS(func) & METH_O)) { return __Pyx_PyObject_CallMethO(func, arg); #if CYTHON_FAST_PYCCALL } else if (PyCFunction_GET_FLAGS(func) & METH_FASTCALL) { return __Pyx_PyCFunction_FastCall(func, &arg, 1); #endif } } return __Pyx__PyObject_CallOneArg(func, arg); } #else static CYTHON_INLINE PyObject* __Pyx_PyObject_CallOneArg(PyObject *func, PyObject *arg) { PyObject *result; PyObject *args = PyTuple_Pack(1, arg); if (unlikely(!args)) return NULL; result = __Pyx_PyObject_Call(func, args, NULL); Py_DECREF(args); return result; } #endif /* GetItemInt */ static PyObject *__Pyx_GetItemInt_Generic(PyObject *o, PyObject* j) { PyObject *r; if (!j) return NULL; r = PyObject_GetItem(o, j); Py_DECREF(j); return r; } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyList_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyList_GET_SIZE(o)))) { PyObject *r = PyList_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS Py_ssize_t wrapped_i = i; if (wraparound & unlikely(i < 0)) { wrapped_i += PyTuple_GET_SIZE(o); } if ((!boundscheck) || likely(__Pyx_is_valid_index(wrapped_i, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, wrapped_i); Py_INCREF(r); return r; } return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); #else return PySequence_GetItem(o, i); #endif } static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyList_GET_SIZE(o); if ((!boundscheck) || (likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o))))) { PyObject *r = PyList_GET_ITEM(o, n); Py_INCREF(r); return r; } } else if (PyTuple_CheckExact(o)) { Py_ssize_t n = ((!wraparound) | likely(i >= 0)) ? i : i + PyTuple_GET_SIZE(o); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyTuple_GET_SIZE(o)))) { PyObject *r = PyTuple_GET_ITEM(o, n); Py_INCREF(r); return r; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return NULL; PyErr_Clear(); } } return m->sq_item(o, i); } } #else if (is_list || PySequence_Check(o)) { return PySequence_GetItem(o, i); } #endif return __Pyx_GetItemInt_Generic(o, PyInt_FromSsize_t(i)); } /* BytesEquals */ static CYTHON_INLINE int __Pyx_PyBytes_Equals(PyObject* s1, PyObject* s2, int equals) { #if CYTHON_COMPILING_IN_PYPY return PyObject_RichCompareBool(s1, s2, equals); #else if (s1 == s2) { return (equals == Py_EQ); } else if (PyBytes_CheckExact(s1) & PyBytes_CheckExact(s2)) { const char *ps1, *ps2; Py_ssize_t length = PyBytes_GET_SIZE(s1); if (length != PyBytes_GET_SIZE(s2)) return (equals == Py_NE); ps1 = PyBytes_AS_STRING(s1); ps2 = PyBytes_AS_STRING(s2); if (ps1[0] != ps2[0]) { return (equals == Py_NE); } else if (length == 1) { return (equals == Py_EQ); } else { int result; #if CYTHON_USE_UNICODE_INTERNALS Py_hash_t hash1, hash2; hash1 = ((PyBytesObject*)s1)->ob_shash; hash2 = ((PyBytesObject*)s2)->ob_shash; if (hash1 != hash2 && hash1 != -1 && hash2 != -1) { return (equals == Py_NE); } #endif result = memcmp(ps1, ps2, (size_t)length); return (equals == Py_EQ) ? (result == 0) : (result != 0); } } else if ((s1 == Py_None) & PyBytes_CheckExact(s2)) { return (equals == Py_NE); } else if ((s2 == Py_None) & PyBytes_CheckExact(s1)) { return (equals == Py_NE); } else { int result; PyObject* py_result = PyObject_RichCompare(s1, s2, equals); if (!py_result) return -1; result = __Pyx_PyObject_IsTrue(py_result); Py_DECREF(py_result); return result; } #endif } /* UnicodeEquals */ static CYTHON_INLINE int __Pyx_PyUnicode_Equals(PyObject* s1, PyObject* s2, int equals) { #if CYTHON_COMPILING_IN_PYPY return PyObject_RichCompareBool(s1, s2, equals); #else #if PY_MAJOR_VERSION < 3 PyObject* owned_ref = NULL; #endif int s1_is_unicode, s2_is_unicode; if (s1 == s2) { goto return_eq; } s1_is_unicode = PyUnicode_CheckExact(s1); s2_is_unicode = PyUnicode_CheckExact(s2); #if PY_MAJOR_VERSION < 3 if ((s1_is_unicode & (!s2_is_unicode)) && PyString_CheckExact(s2)) { owned_ref = PyUnicode_FromObject(s2); if (unlikely(!owned_ref)) return -1; s2 = owned_ref; s2_is_unicode = 1; } else if ((s2_is_unicode & (!s1_is_unicode)) && PyString_CheckExact(s1)) { owned_ref = PyUnicode_FromObject(s1); if (unlikely(!owned_ref)) return -1; s1 = owned_ref; s1_is_unicode = 1; } else if (((!s2_is_unicode) & (!s1_is_unicode))) { return __Pyx_PyBytes_Equals(s1, s2, equals); } #endif if (s1_is_unicode & s2_is_unicode) { Py_ssize_t length; int kind; void *data1, *data2; if (unlikely(__Pyx_PyUnicode_READY(s1) < 0) || unlikely(__Pyx_PyUnicode_READY(s2) < 0)) return -1; length = __Pyx_PyUnicode_GET_LENGTH(s1); if (length != __Pyx_PyUnicode_GET_LENGTH(s2)) { goto return_ne; } #if CYTHON_USE_UNICODE_INTERNALS { Py_hash_t hash1, hash2; #if CYTHON_PEP393_ENABLED hash1 = ((PyASCIIObject*)s1)->hash; hash2 = ((PyASCIIObject*)s2)->hash; #else hash1 = ((PyUnicodeObject*)s1)->hash; hash2 = ((PyUnicodeObject*)s2)->hash; #endif if (hash1 != hash2 && hash1 != -1 && hash2 != -1) { goto return_ne; } } #endif kind = __Pyx_PyUnicode_KIND(s1); if (kind != __Pyx_PyUnicode_KIND(s2)) { goto return_ne; } data1 = __Pyx_PyUnicode_DATA(s1); data2 = __Pyx_PyUnicode_DATA(s2); if (__Pyx_PyUnicode_READ(kind, data1, 0) != __Pyx_PyUnicode_READ(kind, data2, 0)) { goto return_ne; } else if (length == 1) { goto return_eq; } else { int result = memcmp(data1, data2, (size_t)(length * kind)); #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif return (equals == Py_EQ) ? (result == 0) : (result != 0); } } else if ((s1 == Py_None) & s2_is_unicode) { goto return_ne; } else if ((s2 == Py_None) & s1_is_unicode) { goto return_ne; } else { int result; PyObject* py_result = PyObject_RichCompare(s1, s2, equals); #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif if (!py_result) return -1; result = __Pyx_PyObject_IsTrue(py_result); Py_DECREF(py_result); return result; } return_eq: #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif return (equals == Py_EQ); return_ne: #if PY_MAJOR_VERSION < 3 Py_XDECREF(owned_ref); #endif return (equals == Py_NE); #endif } /* SetItemInt */ static int __Pyx_SetItemInt_Generic(PyObject *o, PyObject *j, PyObject *v) { int r; if (!j) return -1; r = PyObject_SetItem(o, j, v); Py_DECREF(j); return r; } static CYTHON_INLINE int __Pyx_SetItemInt_Fast(PyObject *o, Py_ssize_t i, PyObject *v, int is_list, CYTHON_NCP_UNUSED int wraparound, CYTHON_NCP_UNUSED int boundscheck) { #if CYTHON_ASSUME_SAFE_MACROS && !CYTHON_AVOID_BORROWED_REFS && CYTHON_USE_TYPE_SLOTS if (is_list || PyList_CheckExact(o)) { Py_ssize_t n = (!wraparound) ? i : ((likely(i >= 0)) ? i : i + PyList_GET_SIZE(o)); if ((!boundscheck) || likely(__Pyx_is_valid_index(n, PyList_GET_SIZE(o)))) { PyObject* old = PyList_GET_ITEM(o, n); Py_INCREF(v); PyList_SET_ITEM(o, n, v); Py_DECREF(old); return 1; } } else { PySequenceMethods *m = Py_TYPE(o)->tp_as_sequence; if (likely(m && m->sq_ass_item)) { if (wraparound && unlikely(i < 0) && likely(m->sq_length)) { Py_ssize_t l = m->sq_length(o); if (likely(l >= 0)) { i += l; } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) return -1; PyErr_Clear(); } } return m->sq_ass_item(o, i, v); } } #else #if CYTHON_COMPILING_IN_PYPY if (is_list || (PySequence_Check(o) && !PyDict_Check(o))) #else if (is_list || PySequence_Check(o)) #endif { return PySequence_SetItem(o, i, v); } #endif return __Pyx_SetItemInt_Generic(o, PyInt_FromSsize_t(i), v); } /* StringJoin */ #if !CYTHON_COMPILING_IN_CPYTHON static CYTHON_INLINE PyObject* __Pyx_PyBytes_Join(PyObject* sep, PyObject* values) { return PyObject_CallMethodObjArgs(sep, __pyx_n_s_join, values, NULL); } #endif /* SliceObject */ static CYTHON_INLINE PyObject* __Pyx_PyObject_GetSlice(PyObject* obj, Py_ssize_t cstart, Py_ssize_t cstop, PyObject** _py_start, PyObject** _py_stop, PyObject** _py_slice, int has_cstart, int has_cstop, CYTHON_UNUSED int wraparound) { #if CYTHON_USE_TYPE_SLOTS PyMappingMethods* mp; #if PY_MAJOR_VERSION < 3 PySequenceMethods* ms = Py_TYPE(obj)->tp_as_sequence; if (likely(ms && ms->sq_slice)) { if (!has_cstart) { if (_py_start && (*_py_start != Py_None)) { cstart = __Pyx_PyIndex_AsSsize_t(*_py_start); if ((cstart == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstart = 0; } if (!has_cstop) { if (_py_stop && (*_py_stop != Py_None)) { cstop = __Pyx_PyIndex_AsSsize_t(*_py_stop); if ((cstop == (Py_ssize_t)-1) && PyErr_Occurred()) goto bad; } else cstop = PY_SSIZE_T_MAX; } if (wraparound && unlikely((cstart < 0) | (cstop < 0)) && likely(ms->sq_length)) { Py_ssize_t l = ms->sq_length(obj); if (likely(l >= 0)) { if (cstop < 0) { cstop += l; if (cstop < 0) cstop = 0; } if (cstart < 0) { cstart += l; if (cstart < 0) cstart = 0; } } else { if (!PyErr_ExceptionMatches(PyExc_OverflowError)) goto bad; PyErr_Clear(); } } return ms->sq_slice(obj, cstart, cstop); } #endif mp = Py_TYPE(obj)->tp_as_mapping; if (likely(mp && mp->mp_subscript)) #endif { PyObject* result; PyObject *py_slice, *py_start, *py_stop; if (_py_slice) { py_slice = *_py_slice; } else { PyObject* owned_start = NULL; PyObject* owned_stop = NULL; if (_py_start) { py_start = *_py_start; } else { if (has_cstart) { owned_start = py_start = PyInt_FromSsize_t(cstart); if (unlikely(!py_start)) goto bad; } else py_start = Py_None; } if (_py_stop) { py_stop = *_py_stop; } else { if (has_cstop) { owned_stop = py_stop = PyInt_FromSsize_t(cstop); if (unlikely(!py_stop)) { Py_XDECREF(owned_start); goto bad; } } else py_stop = Py_None; } py_slice = PySlice_New(py_start, py_stop, Py_None); Py_XDECREF(owned_start); Py_XDECREF(owned_stop); if (unlikely(!py_slice)) goto bad; } #if CYTHON_USE_TYPE_SLOTS result = mp->mp_subscript(obj, py_slice); #else result = PyObject_GetItem(obj, py_slice); #endif if (!_py_slice) { Py_DECREF(py_slice); } return result; } PyErr_Format(PyExc_TypeError, "'%.200s' object is unsliceable", Py_TYPE(obj)->tp_name); bad: return NULL; } /* RaiseTooManyValuesToUnpack */ static CYTHON_INLINE void __Pyx_RaiseTooManyValuesError(Py_ssize_t expected) { PyErr_Format(PyExc_ValueError, "too many values to unpack (expected %" CYTHON_FORMAT_SSIZE_T "d)", expected); } /* RaiseNeedMoreValuesToUnpack */ static CYTHON_INLINE void __Pyx_RaiseNeedMoreValuesError(Py_ssize_t index) { PyErr_Format(PyExc_ValueError, "need more than %" CYTHON_FORMAT_SSIZE_T "d value%.1s to unpack", index, (index == 1) ? "" : "s"); } /* IterFinish */ static CYTHON_INLINE int __Pyx_IterFinish(void) { #if CYTHON_FAST_THREAD_STATE PyThreadState *tstate = __Pyx_PyThreadState_Current; PyObject* exc_type = tstate->curexc_type; if (unlikely(exc_type)) { if (likely(__Pyx_PyErr_GivenExceptionMatches(exc_type, PyExc_StopIteration))) { PyObject *exc_value, *exc_tb; exc_value = tstate->curexc_value; exc_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; Py_DECREF(exc_type); Py_XDECREF(exc_value); Py_XDECREF(exc_tb); return 0; } else { return -1; } } return 0; #else if (unlikely(PyErr_Occurred())) { if (likely(PyErr_ExceptionMatches(PyExc_StopIteration))) { PyErr_Clear(); return 0; } else { return -1; } } return 0; #endif } /* UnpackItemEndCheck */ static int __Pyx_IternextUnpackEndCheck(PyObject *retval, Py_ssize_t expected) { if (unlikely(retval)) { Py_DECREF(retval); __Pyx_RaiseTooManyValuesError(expected); return -1; } else { return __Pyx_IterFinish(); } return 0; } /* RaiseArgTupleInvalid */ static void __Pyx_RaiseArgtupleInvalid( const char* func_name, int exact, Py_ssize_t num_min, Py_ssize_t num_max, Py_ssize_t num_found) { Py_ssize_t num_expected; const char *more_or_less; if (num_found < num_min) { num_expected = num_min; more_or_less = "at least"; } else { num_expected = num_max; more_or_less = "at most"; } if (exact) { more_or_less = "exactly"; } PyErr_Format(PyExc_TypeError, "%.200s() takes %.8s %" CYTHON_FORMAT_SSIZE_T "d positional argument%.1s (%" CYTHON_FORMAT_SSIZE_T "d given)", func_name, more_or_less, num_expected, (num_expected == 1) ? "" : "s", num_found); } /* RaiseDoubleKeywords */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AsString(kw_name)); #endif } /* ParseKeywords */ static int __Pyx_ParseOptionalKeywords( PyObject *kwds, PyObject **argnames[], PyObject *kwds2, PyObject *values[], Py_ssize_t num_pos_args, const char* function_name) { PyObject *key = 0, *value = 0; Py_ssize_t pos = 0; PyObject*** name; PyObject*** first_kw_arg = argnames + num_pos_args; while (PyDict_Next(kwds, &pos, &key, &value)) { name = first_kw_arg; while (*name && (**name != key)) name++; if (*name) { values[name-argnames] = value; continue; } name = first_kw_arg; #if PY_MAJOR_VERSION < 3 if (likely(PyString_CheckExact(key)) || likely(PyString_Check(key))) { while (*name) { if ((CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**name) == PyString_GET_SIZE(key)) && _PyString_Eq(**name, key)) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { if ((**argname == key) || ( (CYTHON_COMPILING_IN_PYPY || PyString_GET_SIZE(**argname) == PyString_GET_SIZE(key)) && _PyString_Eq(**argname, key))) { goto arg_passed_twice; } argname++; } } } else #endif if (likely(PyUnicode_Check(key))) { while (*name) { int cmp = (**name == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**name, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) { values[name-argnames] = value; break; } name++; } if (*name) continue; else { PyObject*** argname = argnames; while (argname != first_kw_arg) { int cmp = (**argname == key) ? 0 : #if !CYTHON_COMPILING_IN_PYPY && PY_MAJOR_VERSION >= 3 (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : #endif PyUnicode_Compare(**argname, key); if (cmp < 0 && unlikely(PyErr_Occurred())) goto bad; if (cmp == 0) goto arg_passed_twice; argname++; } } } else goto invalid_keyword_type; if (kwds2) { if (unlikely(PyDict_SetItem(kwds2, key, value))) goto bad; } else { goto invalid_keyword; } } return 0; arg_passed_twice: __Pyx_RaiseDoubleKeywordsError(function_name, key); goto bad; invalid_keyword_type: PyErr_Format(PyExc_TypeError, "%.200s() keywords must be strings", function_name); goto bad; invalid_keyword: PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION < 3 "%.200s() got an unexpected keyword argument '%.200s'", function_name, PyString_AsString(key)); #else "%s() got an unexpected keyword argument '%U'", function_name, key); #endif bad: return -1; } /* PyErrFetchRestore */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx_ErrRestoreInState(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; tmp_type = tstate->curexc_type; tmp_value = tstate->curexc_value; tmp_tb = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = tb; Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { *type = tstate->curexc_type; *value = tstate->curexc_value; *tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; } #endif /* GetException */ #if CYTHON_FAST_THREAD_STATE static int __Pyx__GetException(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) #else static int __Pyx_GetException(PyObject **type, PyObject **value, PyObject **tb) #endif { PyObject *local_type, *local_value, *local_tb; #if CYTHON_FAST_THREAD_STATE PyObject *tmp_type, *tmp_value, *tmp_tb; local_type = tstate->curexc_type; local_value = tstate->curexc_value; local_tb = tstate->curexc_traceback; tstate->curexc_type = 0; tstate->curexc_value = 0; tstate->curexc_traceback = 0; #else PyErr_Fetch(&local_type, &local_value, &local_tb); #endif PyErr_NormalizeException(&local_type, &local_value, &local_tb); #if CYTHON_FAST_THREAD_STATE if (unlikely(tstate->curexc_type)) #else if (unlikely(PyErr_Occurred())) #endif goto bad; #if PY_MAJOR_VERSION >= 3 if (local_tb) { if (unlikely(PyException_SetTraceback(local_value, local_tb) < 0)) goto bad; } #endif Py_XINCREF(local_tb); Py_XINCREF(local_type); Py_XINCREF(local_value); *type = local_type; *value = local_value; *tb = local_tb; #if CYTHON_FAST_THREAD_STATE #if CYTHON_USE_EXC_INFO_STACK { _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = local_type; exc_info->exc_value = local_value; exc_info->exc_traceback = local_tb; } #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = local_type; tstate->exc_value = local_value; tstate->exc_traceback = local_tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); #else PyErr_SetExcInfo(local_type, local_value, local_tb); #endif return 0; bad: *type = 0; *value = 0; *tb = 0; Py_XDECREF(local_type); Py_XDECREF(local_value); Py_XDECREF(local_tb); return -1; } /* SwapException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSwap(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = *type; exc_info->exc_value = *value; exc_info->exc_traceback = *tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = *type; tstate->exc_value = *value; tstate->exc_traceback = *tb; #endif *type = tmp_type; *value = tmp_value; *tb = tmp_tb; } #else static CYTHON_INLINE void __Pyx_ExceptionSwap(PyObject **type, PyObject **value, PyObject **tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; PyErr_GetExcInfo(&tmp_type, &tmp_value, &tmp_tb); PyErr_SetExcInfo(*type, *value, *tb); *type = tmp_type; *value = tmp_value; *tb = tmp_tb; } #endif /* GetTopmostException */ #if CYTHON_USE_EXC_INFO_STACK static _PyErr_StackItem * __Pyx_PyErr_GetTopmostException(PyThreadState *tstate) { _PyErr_StackItem *exc_info = tstate->exc_info; while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) && exc_info->previous_item != NULL) { exc_info = exc_info->previous_item; } return exc_info; } #endif /* SaveResetException */ #if CYTHON_FAST_THREAD_STATE static CYTHON_INLINE void __Pyx__ExceptionSave(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) { #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = __Pyx_PyErr_GetTopmostException(tstate); *type = exc_info->exc_type; *value = exc_info->exc_value; *tb = exc_info->exc_traceback; #else *type = tstate->exc_type; *value = tstate->exc_value; *tb = tstate->exc_traceback; #endif Py_XINCREF(*type); Py_XINCREF(*value); Py_XINCREF(*tb); } static CYTHON_INLINE void __Pyx__ExceptionReset(PyThreadState *tstate, PyObject *type, PyObject *value, PyObject *tb) { PyObject *tmp_type, *tmp_value, *tmp_tb; #if CYTHON_USE_EXC_INFO_STACK _PyErr_StackItem *exc_info = tstate->exc_info; tmp_type = exc_info->exc_type; tmp_value = exc_info->exc_value; tmp_tb = exc_info->exc_traceback; exc_info->exc_type = type; exc_info->exc_value = value; exc_info->exc_traceback = tb; #else tmp_type = tstate->exc_type; tmp_value = tstate->exc_value; tmp_tb = tstate->exc_traceback; tstate->exc_type = type; tstate->exc_value = value; tstate->exc_traceback = tb; #endif Py_XDECREF(tmp_type); Py_XDECREF(tmp_value); Py_XDECREF(tmp_tb); } #endif /* Import */ static PyObject *__Pyx_Import(PyObject *name, PyObject *from_list, int level) { PyObject *empty_list = 0; PyObject *module = 0; PyObject *global_dict = 0; PyObject *empty_dict = 0; PyObject *list; #if PY_MAJOR_VERSION < 3 PyObject *py_import; py_import = __Pyx_PyObject_GetAttrStr(__pyx_b, __pyx_n_s_import); if (!py_import) goto bad; #endif if (from_list) list = from_list; else { empty_list = PyList_New(0); if (!empty_list) goto bad; list = empty_list; } global_dict = PyModule_GetDict(__pyx_m); if (!global_dict) goto bad; empty_dict = PyDict_New(); if (!empty_dict) goto bad; { #if PY_MAJOR_VERSION >= 3 if (level == -1) { if (strchr(__Pyx_MODULE_NAME, '.')) { module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, 1); if (!module) { if (!PyErr_ExceptionMatches(PyExc_ImportError)) goto bad; PyErr_Clear(); } } level = 0; } #endif if (!module) { #if PY_MAJOR_VERSION < 3 PyObject *py_level = PyInt_FromLong(level); if (!py_level) goto bad; module = PyObject_CallFunctionObjArgs(py_import, name, global_dict, empty_dict, list, py_level, (PyObject *)NULL); Py_DECREF(py_level); #else module = PyImport_ImportModuleLevelObject( name, global_dict, empty_dict, list, level); #endif } } bad: #if PY_MAJOR_VERSION < 3 Py_XDECREF(py_import); #endif Py_XDECREF(empty_list); Py_XDECREF(empty_dict); return module; } /* ImportFrom */ static PyObject* __Pyx_ImportFrom(PyObject* module, PyObject* name) { PyObject* value = __Pyx_PyObject_GetAttrStr(module, name); if (unlikely(!value) && PyErr_ExceptionMatches(PyExc_AttributeError)) { PyErr_Format(PyExc_ImportError, #if PY_MAJOR_VERSION < 3 "cannot import name %.230s", PyString_AS_STRING(name)); #else "cannot import name %S", name); #endif } return value; } /* CLineInTraceback */ #ifndef CYTHON_CLINE_IN_TRACEBACK static int __Pyx_CLineForTraceback(PyThreadState *tstate, int c_line) { PyObject *use_cline; PyObject *ptype, *pvalue, *ptraceback; #if CYTHON_COMPILING_IN_CPYTHON PyObject **cython_runtime_dict; #endif if (unlikely(!__pyx_cython_runtime)) { return c_line; } __Pyx_ErrFetchInState(tstate, &ptype, &pvalue, &ptraceback); #if CYTHON_COMPILING_IN_CPYTHON cython_runtime_dict = _PyObject_GetDictPtr(__pyx_cython_runtime); if (likely(cython_runtime_dict)) { __PYX_PY_DICT_LOOKUP_IF_MODIFIED( use_cline, *cython_runtime_dict, __Pyx_PyDict_GetItemStr(*cython_runtime_dict, __pyx_n_s_cline_in_traceback)) } else #endif { PyObject *use_cline_obj = __Pyx_PyObject_GetAttrStr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback); if (use_cline_obj) { use_cline = PyObject_Not(use_cline_obj) ? Py_False : Py_True; Py_DECREF(use_cline_obj); } else { PyErr_Clear(); use_cline = NULL; } } if (!use_cline) { c_line = 0; PyObject_SetAttr(__pyx_cython_runtime, __pyx_n_s_cline_in_traceback, Py_False); } else if (use_cline == Py_False || (use_cline != Py_True && PyObject_Not(use_cline) != 0)) { c_line = 0; } __Pyx_ErrRestoreInState(tstate, ptype, pvalue, ptraceback); return c_line; } #endif /* CodeObjectCache */ static int __pyx_bisect_code_objects(__Pyx_CodeObjectCacheEntry* entries, int count, int code_line) { int start = 0, mid = 0, end = count - 1; if (end >= 0 && code_line > entries[end].code_line) { return count; } while (start < end) { mid = start + (end - start) / 2; if (code_line < entries[mid].code_line) { end = mid; } else if (code_line > entries[mid].code_line) { start = mid + 1; } else { return mid; } } if (code_line <= entries[mid].code_line) { return mid; } else { return mid + 1; } } static PyCodeObject *__pyx_find_code_object(int code_line) { PyCodeObject* code_object; int pos; if (unlikely(!code_line) || unlikely(!__pyx_code_cache.entries)) { return NULL; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if (unlikely(pos >= __pyx_code_cache.count) || unlikely(__pyx_code_cache.entries[pos].code_line != code_line)) { return NULL; } code_object = __pyx_code_cache.entries[pos].code_object; Py_INCREF(code_object); return code_object; } static void __pyx_insert_code_object(int code_line, PyCodeObject* code_object) { int pos, i; __Pyx_CodeObjectCacheEntry* entries = __pyx_code_cache.entries; if (unlikely(!code_line)) { return; } if (unlikely(!entries)) { entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Malloc(64*sizeof(__Pyx_CodeObjectCacheEntry)); if (likely(entries)) { __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = 64; __pyx_code_cache.count = 1; entries[0].code_line = code_line; entries[0].code_object = code_object; Py_INCREF(code_object); } return; } pos = __pyx_bisect_code_objects(__pyx_code_cache.entries, __pyx_code_cache.count, code_line); if ((pos < __pyx_code_cache.count) && unlikely(__pyx_code_cache.entries[pos].code_line == code_line)) { PyCodeObject* tmp = entries[pos].code_object; entries[pos].code_object = code_object; Py_DECREF(tmp); return; } if (__pyx_code_cache.count == __pyx_code_cache.max_count) { int new_max = __pyx_code_cache.max_count + 64; entries = (__Pyx_CodeObjectCacheEntry*)PyMem_Realloc( __pyx_code_cache.entries, (size_t)new_max*sizeof(__Pyx_CodeObjectCacheEntry)); if (unlikely(!entries)) { return; } __pyx_code_cache.entries = entries; __pyx_code_cache.max_count = new_max; } for (i=__pyx_code_cache.count; i>pos; i--) { entries[i] = entries[i-1]; } entries[pos].code_line = code_line; entries[pos].code_object = code_object; __pyx_code_cache.count++; Py_INCREF(code_object); } /* AddTraceback */ #include "compile.h" #include "frameobject.h" #include "traceback.h" static PyCodeObject* __Pyx_CreateCodeObjectForTraceback( const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyObject *py_srcfile = 0; PyObject *py_funcname = 0; #if PY_MAJOR_VERSION < 3 py_srcfile = PyString_FromString(filename); #else py_srcfile = PyUnicode_FromString(filename); #endif if (!py_srcfile) goto bad; if (c_line) { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #else py_funcname = PyUnicode_FromFormat( "%s (%s:%d)", funcname, __pyx_cfilenm, c_line); #endif } else { #if PY_MAJOR_VERSION < 3 py_funcname = PyString_FromString(funcname); #else py_funcname = PyUnicode_FromString(funcname); #endif } if (!py_funcname) goto bad; py_code = __Pyx_PyCode_New( 0, 0, 0, 0, 0, __pyx_empty_bytes, /*PyObject *code,*/ __pyx_empty_tuple, /*PyObject *consts,*/ __pyx_empty_tuple, /*PyObject *names,*/ __pyx_empty_tuple, /*PyObject *varnames,*/ __pyx_empty_tuple, /*PyObject *freevars,*/ __pyx_empty_tuple, /*PyObject *cellvars,*/ py_srcfile, /*PyObject *filename,*/ py_funcname, /*PyObject *name,*/ py_line, __pyx_empty_bytes /*PyObject *lnotab*/ ); Py_DECREF(py_srcfile); Py_DECREF(py_funcname); return py_code; bad: Py_XDECREF(py_srcfile); Py_XDECREF(py_funcname); return NULL; } static void __Pyx_AddTraceback(const char *funcname, int c_line, int py_line, const char *filename) { PyCodeObject *py_code = 0; PyFrameObject *py_frame = 0; PyThreadState *tstate = __Pyx_PyThreadState_Current; if (c_line) { c_line = __Pyx_CLineForTraceback(tstate, c_line); } py_code = __pyx_find_code_object(c_line ? -c_line : py_line); if (!py_code) { py_code = __Pyx_CreateCodeObjectForTraceback( funcname, c_line, py_line, filename); if (!py_code) goto bad; __pyx_insert_code_object(c_line ? -c_line : py_line, py_code); } py_frame = PyFrame_New( tstate, /*PyThreadState *tstate,*/ py_code, /*PyCodeObject *code,*/ __pyx_d, /*PyObject *globals,*/ 0 /*PyObject *locals*/ ); if (!py_frame) goto bad; __Pyx_PyFrame_SetLineNumber(py_frame, py_line); PyTraceBack_Here(py_frame); bad: Py_XDECREF(py_code); Py_XDECREF(py_frame); } /* CIntToPy */ static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; if (is_unsigned) { if (sizeof(long) < sizeof(long)) { return PyInt_FromLong((long) value); } else if (sizeof(long) <= sizeof(unsigned long)) { return PyLong_FromUnsignedLong((unsigned long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { return PyLong_FromUnsignedLongLong((unsigned PY_LONG_LONG) value); #endif } } else { if (sizeof(long) <= sizeof(long)) { return PyInt_FromLong((long) value); #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { return PyLong_FromLongLong((PY_LONG_LONG) value); #endif } } { int one = 1; int little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&value; return _PyLong_FromByteArray(bytes, sizeof(long), little, !is_unsigned); } } /* CIntFromPyVerify */ #define __PYX_VERIFY_RETURN_INT(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 0) #define __PYX_VERIFY_RETURN_INT_EXC(target_type, func_type, func_value)\ __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, 1) #define __PYX__VERIFY_RETURN_INT(target_type, func_type, func_value, exc)\ {\ func_type value = func_value;\ if (sizeof(target_type) < sizeof(func_type)) {\ if (unlikely(value != (func_type) (target_type) value)) {\ func_type zero = 0;\ if (exc && unlikely(value == (func_type)-1 && PyErr_Occurred()))\ return (target_type) -1;\ if (is_unsigned && unlikely(value < zero))\ goto raise_neg_overflow;\ else\ goto raise_overflow;\ }\ }\ return (target_type) value;\ } /* CIntFromPy */ static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) { const long neg_one = (long) ((long) 0 - (long) 1), const_zero = (long) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(long) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(long, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (long) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case 1: __PYX_VERIFY_RETURN_INT(long, digit, digits[0]) case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 2 * PyLong_SHIFT) { return (long) (((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 3 * PyLong_SHIFT) { return (long) (((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) >= 4 * PyLong_SHIFT) { return (long) (((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (long) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(long) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (long) 0; case -1: __PYX_VERIFY_RETURN_INT(long, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(long, digit, +digits[0]) case -2: if (8 * sizeof(long) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) (((long)-1)*(((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 2: if (8 * sizeof(long) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { return (long) ((((((long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -3: if (8 * sizeof(long) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 3: if (8 * sizeof(long) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { return (long) ((((((((long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case -4: if (8 * sizeof(long) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) (((long)-1)*(((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; case 4: if (8 * sizeof(long) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(long, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(long) - 1 > 4 * PyLong_SHIFT) { return (long) ((((((((((long)digits[3]) << PyLong_SHIFT) | (long)digits[2]) << PyLong_SHIFT) | (long)digits[1]) << PyLong_SHIFT) | (long)digits[0]))); } } break; } #endif if (sizeof(long) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(long, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(long) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(long, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else long val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (long) -1; } } else { long val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (long) -1; val = __Pyx_PyInt_As_long(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to long"); return (long) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to long"); return (long) -1; } /* CIntFromPy */ static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) { const int neg_one = (int) ((int) 0 - (int) 1), const_zero = (int) 0; const int is_unsigned = neg_one > const_zero; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x))) { if (sizeof(int) < sizeof(long)) { __PYX_VERIFY_RETURN_INT(int, long, PyInt_AS_LONG(x)) } else { long val = PyInt_AS_LONG(x); if (is_unsigned && unlikely(val < 0)) { goto raise_neg_overflow; } return (int) val; } } else #endif if (likely(PyLong_Check(x))) { if (is_unsigned) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case 1: __PYX_VERIFY_RETURN_INT(int, digit, digits[0]) case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 2 * PyLong_SHIFT) { return (int) (((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 3 * PyLong_SHIFT) { return (int) (((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) >= 4 * PyLong_SHIFT) { return (int) (((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0])); } } break; } #endif #if CYTHON_COMPILING_IN_CPYTHON if (unlikely(Py_SIZE(x) < 0)) { goto raise_neg_overflow; } #else { int result = PyObject_RichCompareBool(x, Py_False, Py_LT); if (unlikely(result < 0)) return (int) -1; if (unlikely(result == 1)) goto raise_neg_overflow; } #endif if (sizeof(int) <= sizeof(unsigned long)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned long, PyLong_AsUnsignedLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(unsigned PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, unsigned PY_LONG_LONG, PyLong_AsUnsignedLongLong(x)) #endif } } else { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)x)->ob_digit; switch (Py_SIZE(x)) { case 0: return (int) 0; case -1: __PYX_VERIFY_RETURN_INT(int, sdigit, (sdigit) (-(sdigit)digits[0])) case 1: __PYX_VERIFY_RETURN_INT(int, digit, +digits[0]) case -2: if (8 * sizeof(int) - 1 > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) (((int)-1)*(((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 2: if (8 * sizeof(int) > 1 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 2 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { return (int) ((((((int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -3: if (8 * sizeof(int) - 1 > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 3: if (8 * sizeof(int) > 2 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 3 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { return (int) ((((((((int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case -4: if (8 * sizeof(int) - 1 > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, long, -(long) (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) (((int)-1)*(((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; case 4: if (8 * sizeof(int) > 3 * PyLong_SHIFT) { if (8 * sizeof(unsigned long) > 4 * PyLong_SHIFT) { __PYX_VERIFY_RETURN_INT(int, unsigned long, (((((((((unsigned long)digits[3]) << PyLong_SHIFT) | (unsigned long)digits[2]) << PyLong_SHIFT) | (unsigned long)digits[1]) << PyLong_SHIFT) | (unsigned long)digits[0]))) } else if (8 * sizeof(int) - 1 > 4 * PyLong_SHIFT) { return (int) ((((((((((int)digits[3]) << PyLong_SHIFT) | (int)digits[2]) << PyLong_SHIFT) | (int)digits[1]) << PyLong_SHIFT) | (int)digits[0]))); } } break; } #endif if (sizeof(int) <= sizeof(long)) { __PYX_VERIFY_RETURN_INT_EXC(int, long, PyLong_AsLong(x)) #ifdef HAVE_LONG_LONG } else if (sizeof(int) <= sizeof(PY_LONG_LONG)) { __PYX_VERIFY_RETURN_INT_EXC(int, PY_LONG_LONG, PyLong_AsLongLong(x)) #endif } } { #if CYTHON_COMPILING_IN_PYPY && !defined(_PyLong_AsByteArray) PyErr_SetString(PyExc_RuntimeError, "_PyLong_AsByteArray() not available in PyPy, cannot convert large numbers"); #else int val; PyObject *v = __Pyx_PyNumber_IntOrLong(x); #if PY_MAJOR_VERSION < 3 if (likely(v) && !PyLong_Check(v)) { PyObject *tmp = v; v = PyNumber_Long(tmp); Py_DECREF(tmp); } #endif if (likely(v)) { int one = 1; int is_little = (int)*(unsigned char *)&one; unsigned char *bytes = (unsigned char *)&val; int ret = _PyLong_AsByteArray((PyLongObject *)v, bytes, sizeof(val), is_little, !is_unsigned); Py_DECREF(v); if (likely(!ret)) return val; } #endif return (int) -1; } } else { int val; PyObject *tmp = __Pyx_PyNumber_IntOrLong(x); if (!tmp) return (int) -1; val = __Pyx_PyInt_As_int(tmp); Py_DECREF(tmp); return val; } raise_overflow: PyErr_SetString(PyExc_OverflowError, "value too large to convert to int"); return (int) -1; raise_neg_overflow: PyErr_SetString(PyExc_OverflowError, "can't convert negative value to int"); return (int) -1; } /* FastTypeChecks */ #if CYTHON_COMPILING_IN_CPYTHON static int __Pyx_InBases(PyTypeObject *a, PyTypeObject *b) { while (a) { a = a->tp_base; if (a == b) return 1; } return b == &PyBaseObject_Type; } static CYTHON_INLINE int __Pyx_IsSubtype(PyTypeObject *a, PyTypeObject *b) { PyObject *mro; if (a == b) return 1; mro = a->tp_mro; if (likely(mro)) { Py_ssize_t i, n; n = PyTuple_GET_SIZE(mro); for (i = 0; i < n; i++) { if (PyTuple_GET_ITEM(mro, i) == (PyObject *)b) return 1; } return 0; } return __Pyx_InBases(a, b); } #if PY_MAJOR_VERSION == 2 static int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject* exc_type2) { PyObject *exception, *value, *tb; int res; __Pyx_PyThreadState_declare __Pyx_PyThreadState_assign __Pyx_ErrFetch(&exception, &value, &tb); res = exc_type1 ? PyObject_IsSubclass(err, exc_type1) : 0; if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } if (!res) { res = PyObject_IsSubclass(err, exc_type2); if (unlikely(res == -1)) { PyErr_WriteUnraisable(err); res = 0; } } __Pyx_ErrRestore(exception, value, tb); return res; } #else static CYTHON_INLINE int __Pyx_inner_PyErr_GivenExceptionMatches2(PyObject *err, PyObject* exc_type1, PyObject *exc_type2) { int res = exc_type1 ? __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type1) : 0; if (!res) { res = __Pyx_IsSubtype((PyTypeObject*)err, (PyTypeObject*)exc_type2); } return res; } #endif static int __Pyx_PyErr_GivenExceptionMatchesTuple(PyObject *exc_type, PyObject *tuple) { Py_ssize_t i, n; assert(PyExceptionClass_Check(exc_type)); n = PyTuple_GET_SIZE(tuple); #if PY_MAJOR_VERSION >= 3 for (i=0; ip) { #if PY_MAJOR_VERSION < 3 if (t->is_unicode) { *t->p = PyUnicode_DecodeUTF8(t->s, t->n - 1, NULL); } else if (t->intern) { *t->p = PyString_InternFromString(t->s); } else { *t->p = PyString_FromStringAndSize(t->s, t->n - 1); } #else if (t->is_unicode | t->is_str) { if (t->intern) { *t->p = PyUnicode_InternFromString(t->s); } else if (t->encoding) { *t->p = PyUnicode_Decode(t->s, t->n - 1, t->encoding, NULL); } else { *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1); } } else { *t->p = PyBytes_FromStringAndSize(t->s, t->n - 1); } #endif if (!*t->p) return -1; if (PyObject_Hash(*t->p) == -1) return -1; ++t; } return 0; } static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) { return __Pyx_PyUnicode_FromStringAndSize(c_str, (Py_ssize_t)strlen(c_str)); } static CYTHON_INLINE const char* __Pyx_PyObject_AsString(PyObject* o) { Py_ssize_t ignore; return __Pyx_PyObject_AsStringAndSize(o, &ignore); } #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT #if !CYTHON_PEP393_ENABLED static const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { char* defenc_c; PyObject* defenc = _PyUnicode_AsDefaultEncodedString(o, NULL); if (!defenc) return NULL; defenc_c = PyBytes_AS_STRING(defenc); #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII { char* end = defenc_c + PyBytes_GET_SIZE(defenc); char* c; for (c = defenc_c; c < end; c++) { if ((unsigned char) (*c) >= 128) { PyUnicode_AsASCIIString(o); return NULL; } } } #endif *length = PyBytes_GET_SIZE(defenc); return defenc_c; } #else static CYTHON_INLINE const char* __Pyx_PyUnicode_AsStringAndSize(PyObject* o, Py_ssize_t *length) { if (unlikely(__Pyx_PyUnicode_READY(o) == -1)) return NULL; #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII if (likely(PyUnicode_IS_ASCII(o))) { *length = PyUnicode_GET_LENGTH(o); return PyUnicode_AsUTF8(o); } else { PyUnicode_AsASCIIString(o); return NULL; } #else return PyUnicode_AsUTF8AndSize(o, length); #endif } #endif #endif static CYTHON_INLINE const char* __Pyx_PyObject_AsStringAndSize(PyObject* o, Py_ssize_t *length) { #if __PYX_DEFAULT_STRING_ENCODING_IS_ASCII || __PYX_DEFAULT_STRING_ENCODING_IS_DEFAULT if ( #if PY_MAJOR_VERSION < 3 && __PYX_DEFAULT_STRING_ENCODING_IS_ASCII __Pyx_sys_getdefaultencoding_not_ascii && #endif PyUnicode_Check(o)) { return __Pyx_PyUnicode_AsStringAndSize(o, length); } else #endif #if (!CYTHON_COMPILING_IN_PYPY) || (defined(PyByteArray_AS_STRING) && defined(PyByteArray_GET_SIZE)) if (PyByteArray_Check(o)) { *length = PyByteArray_GET_SIZE(o); return PyByteArray_AS_STRING(o); } else #endif { char* result; int r = PyBytes_AsStringAndSize(o, &result, length); if (unlikely(r < 0)) { return NULL; } else { return result; } } } static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) { int is_true = x == Py_True; if (is_true | (x == Py_False) | (x == Py_None)) return is_true; else return PyObject_IsTrue(x); } static CYTHON_INLINE int __Pyx_PyObject_IsTrueAndDecref(PyObject* x) { int retval; if (unlikely(!x)) return -1; retval = __Pyx_PyObject_IsTrue(x); Py_DECREF(x); return retval; } static PyObject* __Pyx_PyNumber_IntOrLongWrongResultType(PyObject* result, const char* type_name) { #if PY_MAJOR_VERSION >= 3 if (PyLong_Check(result)) { if (PyErr_WarnFormat(PyExc_DeprecationWarning, 1, "__int__ returned non-int (type %.200s). " "The ability to return an instance of a strict subclass of int " "is deprecated, and may be removed in a future version of Python.", Py_TYPE(result)->tp_name)) { Py_DECREF(result); return NULL; } return result; } #endif PyErr_Format(PyExc_TypeError, "__%.4s__ returned non-%.4s (type %.200s)", type_name, type_name, Py_TYPE(result)->tp_name); Py_DECREF(result); return NULL; } static CYTHON_INLINE PyObject* __Pyx_PyNumber_IntOrLong(PyObject* x) { #if CYTHON_USE_TYPE_SLOTS PyNumberMethods *m; #endif const char *name = NULL; PyObject *res = NULL; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_Check(x) || PyLong_Check(x))) #else if (likely(PyLong_Check(x))) #endif return __Pyx_NewRef(x); #if CYTHON_USE_TYPE_SLOTS m = Py_TYPE(x)->tp_as_number; #if PY_MAJOR_VERSION < 3 if (m && m->nb_int) { name = "int"; res = m->nb_int(x); } else if (m && m->nb_long) { name = "long"; res = m->nb_long(x); } #else if (likely(m && m->nb_int)) { name = "int"; res = m->nb_int(x); } #endif #else if (!PyBytes_CheckExact(x) && !PyUnicode_CheckExact(x)) { res = PyNumber_Int(x); } #endif if (likely(res)) { #if PY_MAJOR_VERSION < 3 if (unlikely(!PyInt_Check(res) && !PyLong_Check(res))) { #else if (unlikely(!PyLong_CheckExact(res))) { #endif return __Pyx_PyNumber_IntOrLongWrongResultType(res, name); } } else if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, "an integer is required"); } return res; } static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) { Py_ssize_t ival; PyObject *x; #if PY_MAJOR_VERSION < 3 if (likely(PyInt_CheckExact(b))) { if (sizeof(Py_ssize_t) >= sizeof(long)) return PyInt_AS_LONG(b); else return PyInt_AsSsize_t(b); } #endif if (likely(PyLong_CheckExact(b))) { #if CYTHON_USE_PYLONG_INTERNALS const digit* digits = ((PyLongObject*)b)->ob_digit; const Py_ssize_t size = Py_SIZE(b); if (likely(__Pyx_sst_abs(size) <= 1)) { ival = likely(size) ? digits[0] : 0; if (size == -1) ival = -ival; return ival; } else { switch (size) { case 2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return (Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -2: if (8 * sizeof(Py_ssize_t) > 2 * PyLong_SHIFT) { return -(Py_ssize_t) (((((size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return (Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -3: if (8 * sizeof(Py_ssize_t) > 3 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case 4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return (Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; case -4: if (8 * sizeof(Py_ssize_t) > 4 * PyLong_SHIFT) { return -(Py_ssize_t) (((((((((size_t)digits[3]) << PyLong_SHIFT) | (size_t)digits[2]) << PyLong_SHIFT) | (size_t)digits[1]) << PyLong_SHIFT) | (size_t)digits[0])); } break; } } #endif return PyLong_AsSsize_t(b); } x = PyNumber_Index(b); if (!x) return -1; ival = PyInt_AsSsize_t(x); Py_DECREF(x); return ival; } static CYTHON_INLINE PyObject * __Pyx_PyBool_FromLong(long b) { return b ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False); } static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) { return PyInt_FromSize_t(ival); } #endif /* Py_PYTHON_H */ borgbackup-1.0.13/borg/platform_linux.pyx0000644000175000017500000001143213431513133020425 0ustar useruser00000000000000import os import re import subprocess from stat import S_ISLNK from .helpers import posix_acl_use_stored_uid_gid, user2uid, group2gid, safe_decode, safe_encode, prepare_subprocess_env API_VERSION = '1.0_01' cdef extern from "sys/types.h": int ACL_TYPE_ACCESS int ACL_TYPE_DEFAULT cdef extern from "sys/acl.h": ctypedef struct _acl_t: pass ctypedef _acl_t *acl_t int acl_free(void *obj) acl_t acl_get_file(const char *path, int type) int acl_set_file(const char *path, int type, acl_t acl) acl_t acl_from_text(const char *buf) char *acl_to_text(acl_t acl, ssize_t *len) cdef extern from "acl/libacl.h": int acl_extended_file(const char *path) _comment_re = re.compile(' *#.*', re.M) def acl_use_local_uid_gid(acl): """Replace the user/group field with the local uid/gid if possible """ entries = [] for entry in safe_decode(acl).split('\n'): if entry: fields = entry.split(':') if fields[0] == 'user' and fields[1]: fields[1] = str(user2uid(fields[1], fields[3])) elif fields[0] == 'group' and fields[1]: fields[1] = str(group2gid(fields[1], fields[3])) entries.append(':'.join(fields[:3])) return safe_encode('\n'.join(entries)) cdef acl_append_numeric_ids(acl): """Extend the "POSIX 1003.1e draft standard 17" format with an additional uid/gid field """ entries = [] for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): if entry: type, name, permission = entry.split(':') if name and type == 'user': entries.append(':'.join([type, name, permission, str(user2uid(name, name))])) elif name and type == 'group': entries.append(':'.join([type, name, permission, str(group2gid(name, name))])) else: entries.append(entry) return safe_encode('\n'.join(entries)) cdef acl_numeric_ids(acl): """Replace the "POSIX 1003.1e draft standard 17" user/group field with uid/gid """ entries = [] for entry in _comment_re.sub('', safe_decode(acl)).split('\n'): if entry: type, name, permission = entry.split(':') if name and type == 'user': uid = str(user2uid(name, name)) entries.append(':'.join([type, uid, permission, uid])) elif name and type == 'group': gid = str(group2gid(name, name)) entries.append(':'.join([type, gid, permission, gid])) else: entries.append(entry) return safe_encode('\n'.join(entries)) def acl_get(path, item, st, numeric_owner=False): """Saves ACL Entries If `numeric_owner` is True the user/group field is not preserved only uid/gid """ cdef acl_t default_acl = NULL cdef acl_t access_acl = NULL cdef char *default_text = NULL cdef char *access_text = NULL p = os.fsencode(path) if S_ISLNK(st.st_mode) or acl_extended_file(p) <= 0: return if numeric_owner: converter = acl_numeric_ids else: converter = acl_append_numeric_ids try: access_acl = acl_get_file(p, ACL_TYPE_ACCESS) if access_acl: access_text = acl_to_text(access_acl, NULL) if access_text: item[b'acl_access'] = converter(access_text) default_acl = acl_get_file(p, ACL_TYPE_DEFAULT) if default_acl: default_text = acl_to_text(default_acl, NULL) if default_text: item[b'acl_default'] = converter(default_text) finally: acl_free(default_text) acl_free(default_acl) acl_free(access_text) acl_free(access_acl) def acl_set(path, item, numeric_owner=False): """Restore ACL Entries If `numeric_owner` is True the stored uid/gid is used instead of the user/group names """ cdef acl_t access_acl = NULL cdef acl_t default_acl = NULL p = os.fsencode(path) if numeric_owner: converter = posix_acl_use_stored_uid_gid else: converter = acl_use_local_uid_gid access_text = item.get(b'acl_access') default_text = item.get(b'acl_default') if access_text: try: access_acl = acl_from_text(converter(access_text)) if access_acl: acl_set_file(p, ACL_TYPE_ACCESS, access_acl) finally: acl_free(access_acl) if default_text: try: default_acl = acl_from_text(converter(default_text)) if default_acl: acl_set_file(p, ACL_TYPE_DEFAULT, default_acl) finally: acl_free(default_acl) def umount(mountpoint): env = prepare_subprocess_env(system=True) return subprocess.call(['fusermount', '-u', mountpoint], env=env) borgbackup-1.0.13/borg/remote.py0000644000175000017500000005400313431513133016466 0ustar useruser00000000000000import errno import fcntl import logging import os import select import shlex import sys import tempfile import textwrap import time from subprocess import Popen, PIPE from . import __version__ from .helpers import Error, IntegrityError, sysinfo from .helpers import replace_placeholders from .helpers import BUFSIZE from .helpers import get_limited_unpacker from .helpers import prepare_subprocess_env from .repository import Repository from .logger import create_logger import msgpack logger = create_logger(__name__) RPC_PROTOCOL_VERSION = 2 MAX_INFLIGHT = 100 def os_write(fd, data): """os.write wrapper so we do not lose data for partial writes.""" # TODO: this issue is fixed in cygwin since at least 2.8.0, remove this # wrapper / workaround when this version is considered ancient. # This is happening frequently on cygwin due to its small pipe buffer size of only 64kiB # and also due to its different blocking pipe behaviour compared to Linux/*BSD. # Neither Linux nor *BSD ever do partial writes on blocking pipes, unless interrupted by a # signal, in which case serve() would terminate. amount = remaining = len(data) while remaining: count = os.write(fd, data) remaining -= count if not remaining: break data = data[count:] time.sleep(count * 1e-09) return amount class ConnectionClosed(Error): """Connection closed by remote host""" class ConnectionClosedWithHint(ConnectionClosed): """Connection closed by remote host. {}""" class PathNotAllowed(Error): """Repository path not allowed""" class InvalidRPCMethod(Error): """RPC method {} is not valid""" class UnexpectedRPCDataFormatFromClient(Error): """Borg {}: Got unexpected RPC data format from client.""" class UnexpectedRPCDataFormatFromServer(Error): """Got unexpected RPC data format from server:\n{}""" def __init__(self, data): try: data = data.decode()[:128] except UnicodeDecodeError: data = data[:128] data = ['%02X' % byte for byte in data] data = textwrap.fill(' '.join(data), 16 * 3) super().__init__(data) class RepositoryServer: # pragma: no cover rpc_methods = ( '__len__', 'check', 'commit', 'delete', 'destroy', 'get', 'list', 'negotiate', 'open', 'put', 'rollback', 'save_key', 'load_key', 'break_lock', ) def __init__(self, restrict_to_paths, append_only): self.repository = None self.restrict_to_paths = restrict_to_paths self.append_only = append_only def serve(self): stdin_fd = sys.stdin.fileno() stdout_fd = sys.stdout.fileno() stderr_fd = sys.stdout.fileno() # Make stdin non-blocking fl = fcntl.fcntl(stdin_fd, fcntl.F_GETFL) fcntl.fcntl(stdin_fd, fcntl.F_SETFL, fl | os.O_NONBLOCK) # Make stdout blocking fl = fcntl.fcntl(stdout_fd, fcntl.F_GETFL) fcntl.fcntl(stdout_fd, fcntl.F_SETFL, fl & ~os.O_NONBLOCK) # Make stderr blocking fl = fcntl.fcntl(stderr_fd, fcntl.F_GETFL) fcntl.fcntl(stderr_fd, fcntl.F_SETFL, fl & ~os.O_NONBLOCK) unpacker = get_limited_unpacker('server') while True: r, w, es = select.select([stdin_fd], [], [], 10) if r: data = os.read(stdin_fd, BUFSIZE) if not data: if self.repository is not None: self.repository.close() else: os_write(stderr_fd, "Borg {}: Got connection close before repository was opened.\n" .format(__version__).encode()) return unpacker.feed(data) for unpacked in unpacker: if not (isinstance(unpacked, tuple) and len(unpacked) == 4): if self.repository is not None: self.repository.close() raise UnexpectedRPCDataFormatFromClient(__version__) type, msgid, method, args = unpacked method = method.decode('ascii') try: if method not in self.rpc_methods: raise InvalidRPCMethod(method) try: f = getattr(self, method) except AttributeError: f = getattr(self.repository, method) res = f(*args) except BaseException as e: # These exceptions are reconstructed on the client end in RemoteRepository.call_many(), # and will be handled just like locally raised exceptions. Suppress the remote traceback # for these, except ErrorWithTraceback, which should always display a traceback. if not isinstance(e, (Repository.DoesNotExist, Repository.AlreadyExists, PathNotAllowed)): logging.exception('Borg %s: exception in RPC call:', __version__) logging.error(sysinfo()) exc = "Remote Exception (see remote log for the traceback)" os_write(stdout_fd, msgpack.packb((1, msgid, e.__class__.__name__, exc))) else: os_write(stdout_fd, msgpack.packb((1, msgid, None, res))) if es: self.repository.close() return def negotiate(self, versions): return RPC_PROTOCOL_VERSION def open(self, path, create=False, lock_wait=None, lock=True, exclusive=None, append_only=False): path = os.fsdecode(path) if path.startswith('/~'): # /~/x = path x relative to home dir, /~username/x = relative to "user" home dir path = path[1:] elif path.startswith('/./'): # /./x = path x relative to cwd path = path[3:] path = os.path.realpath(os.path.expanduser(path)) if self.restrict_to_paths: # if --restrict-to-path P is given, we make sure that we only operate in/below path P. # for the prefix check, it is important that the compared pathes both have trailing slashes, # so that a path /foobar will NOT be accepted with --restrict-to-path /foo option. path_with_sep = os.path.join(path, '') # make sure there is a trailing slash (os.sep) for restrict_to_path in self.restrict_to_paths: restrict_to_path_with_sep = os.path.join(os.path.realpath(restrict_to_path), '') # trailing slash if path_with_sep.startswith(restrict_to_path_with_sep): break else: raise PathNotAllowed(path) self.repository = Repository(path, create, lock_wait=lock_wait, lock=lock, append_only=self.append_only or append_only, exclusive=exclusive) self.repository.__enter__() # clean exit handled by serve() method return self.repository.id class RemoteRepository: extra_test_args = [] class RPCError(Exception): def __init__(self, name, remote_type): self.name = name self.remote_type = remote_type class NoAppendOnlyOnServer(Error): """Server does not support --append-only.""" def __init__(self, location, create=False, exclusive=False, lock_wait=None, lock=True, append_only=False, args=None): self.location = self._location = location self.preload_ids = [] self.msgid = 0 self.to_send = b'' self.cache = {} self.stderr_received = b'' # incomplete stderr line bytes received (no \n yet) self.ignore_responses = set() self.responses = {} self.unpacker = get_limited_unpacker('client') self.p = None testing = location.host == '__testsuite__' # when testing, we invoke and talk to a borg process directly (no ssh). # when not testing, we invoke the system-installed ssh binary to talk to a remote borg. env = prepare_subprocess_env(system=not testing) borg_cmd = self.borg_cmd(args, testing) if not testing: borg_cmd = self.ssh_cmd(location) + borg_cmd logger.debug('SSH command line: %s', borg_cmd) self.p = Popen(borg_cmd, bufsize=0, stdin=PIPE, stdout=PIPE, stderr=PIPE, env=env) self.stdin_fd = self.p.stdin.fileno() self.stdout_fd = self.p.stdout.fileno() self.stderr_fd = self.p.stderr.fileno() fcntl.fcntl(self.stdin_fd, fcntl.F_SETFL, fcntl.fcntl(self.stdin_fd, fcntl.F_GETFL) | os.O_NONBLOCK) fcntl.fcntl(self.stdout_fd, fcntl.F_SETFL, fcntl.fcntl(self.stdout_fd, fcntl.F_GETFL) | os.O_NONBLOCK) fcntl.fcntl(self.stderr_fd, fcntl.F_SETFL, fcntl.fcntl(self.stderr_fd, fcntl.F_GETFL) | os.O_NONBLOCK) self.r_fds = [self.stdout_fd, self.stderr_fd] self.x_fds = [self.stdin_fd, self.stdout_fd, self.stderr_fd] try: try: version = self.call('negotiate', RPC_PROTOCOL_VERSION) except ConnectionClosed: raise ConnectionClosedWithHint('Is borg working on the server?') from None if version != RPC_PROTOCOL_VERSION: raise Exception('Server insisted on using unsupported protocol version %d' % version) try: self.id = self.call('open', self.location.path, create, lock_wait, lock, exclusive, append_only) except self.RPCError as err: if err.remote_type != 'TypeError': raise msg = """\ Please note: If you see a TypeError complaining about the number of positional arguments given to open(), you can ignore it if it comes from a borg version < 1.0.7. This TypeError is a cosmetic side effect of the compatibility code borg clients >= 1.0.7 have to support older borg servers. This problem will go away as soon as the server has been upgraded to 1.0.7+. """ # emit this msg in the same way as the "Remote: ..." lines that show the remote TypeError sys.stderr.write(msg) if append_only: raise self.NoAppendOnlyOnServer() self.id = self.call('open', self.location.path, create, lock_wait, lock) except Exception: self.close() raise def __del__(self): if len(self.responses): logging.debug("still %d cached responses left in RemoteRepository" % (len(self.responses),)) if self.p: self.close() assert False, "cleanup happened in Repository.__del__" def __repr__(self): return '<%s %s>' % (self.__class__.__name__, self.location.canonical_path()) def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): try: if exc_type is not None: self.rollback() finally: # in any case, we want to cleanly close the repo, even if the # rollback can not succeed (e.g. because the connection was # already closed) and raised another exception: self.close() def borg_cmd(self, args, testing): """return a borg serve command line""" # give some args/options to "borg serve" process as they were given to us opts = [] if args is not None: opts.append('--umask=%03o' % args.umask) root_logger = logging.getLogger() if root_logger.isEnabledFor(logging.DEBUG): opts.append('--debug') elif root_logger.isEnabledFor(logging.INFO): opts.append('--info') elif root_logger.isEnabledFor(logging.WARNING): pass # warning is default elif root_logger.isEnabledFor(logging.ERROR): opts.append('--error') elif root_logger.isEnabledFor(logging.CRITICAL): opts.append('--critical') else: raise ValueError('log level missing, fix this code') if testing: return [sys.executable, '-m', 'borg.archiver', 'serve'] + opts + self.extra_test_args else: # pragma: no cover remote_path = args.remote_path or os.environ.get('BORG_REMOTE_PATH', 'borg') remote_path = replace_placeholders(remote_path) return [remote_path, 'serve'] + opts def ssh_cmd(self, location): """return a ssh command line that can be prefixed to a borg command line""" args = shlex.split(os.environ.get('BORG_RSH', 'ssh')) if location.port: args += ['-p', str(location.port)] if location.user: args.append('%s@%s' % (location.user, location.host)) else: args.append('%s' % location.host) return args def call(self, cmd, *args, **kw): for resp in self.call_many(cmd, [args], **kw): return resp def call_many(self, cmd, calls, wait=True, is_preloaded=False): if not calls: return def fetch_from_cache(args): msgid = self.cache[args].pop(0) if not self.cache[args]: del self.cache[args] return msgid def handle_error(error, res): if error == b'DoesNotExist': raise Repository.DoesNotExist(self.location.orig) elif error == b'AlreadyExists': raise Repository.AlreadyExists(self.location.orig) elif error == b'CheckNeeded': raise Repository.CheckNeeded(self.location.orig) elif error == b'IntegrityError': raise IntegrityError(res) elif error == b'PathNotAllowed': raise PathNotAllowed(*res) elif error == b'ObjectNotFound': raise Repository.ObjectNotFound(res[0], self.location.orig) elif error == b'InvalidRPCMethod': raise InvalidRPCMethod(*res) else: raise self.RPCError(res.decode('utf-8'), error.decode('utf-8')) calls = list(calls) waiting_for = [] while wait or calls: while waiting_for: try: error, res = self.responses.pop(waiting_for[0]) waiting_for.pop(0) if error: handle_error(error, res) else: yield res if not waiting_for and not calls: return except KeyError: break if self.to_send or ((calls or self.preload_ids) and len(waiting_for) < MAX_INFLIGHT): w_fds = [self.stdin_fd] else: w_fds = [] r, w, x = select.select(self.r_fds, w_fds, self.x_fds, 1) if x: raise Exception('FD exception occurred') for fd in r: if fd is self.stdout_fd: data = os.read(fd, BUFSIZE) if not data: raise ConnectionClosed() self.unpacker.feed(data) for unpacked in self.unpacker: if not (isinstance(unpacked, tuple) and len(unpacked) == 4): raise UnexpectedRPCDataFormatFromServer(data) type, msgid, error, res = unpacked if msgid in self.ignore_responses: self.ignore_responses.remove(msgid) if error: handle_error(error, res) else: self.responses[msgid] = error, res elif fd is self.stderr_fd: data = os.read(fd, 32768) if not data: raise ConnectionClosed() # deal with incomplete lines (may appear due to block buffering) if self.stderr_received: data = self.stderr_received + data self.stderr_received = b'' lines = data.splitlines(keepends=True) if lines and not lines[-1].endswith((b'\r', b'\n')): self.stderr_received = lines.pop() # now we have complete lines in and any partial line in self.stderr_received. for line in lines: handle_remote_line(line.decode('utf-8')) # decode late, avoid partial utf-8 sequences if w: while not self.to_send and (calls or self.preload_ids) and len(waiting_for) < MAX_INFLIGHT: if calls: if is_preloaded: if calls[0] in self.cache: waiting_for.append(fetch_from_cache(calls.pop(0))) else: args = calls.pop(0) if cmd == 'get' and args in self.cache: waiting_for.append(fetch_from_cache(args)) else: self.msgid += 1 waiting_for.append(self.msgid) self.to_send = msgpack.packb((1, self.msgid, cmd, args)) if not self.to_send and self.preload_ids: args = (self.preload_ids.pop(0),) self.msgid += 1 self.cache.setdefault(args, []).append(self.msgid) self.to_send = msgpack.packb((1, self.msgid, cmd, args)) if self.to_send: try: self.to_send = self.to_send[os.write(self.stdin_fd, self.to_send):] except OSError as e: # io.write might raise EAGAIN even though select indicates # that the fd should be writable if e.errno != errno.EAGAIN: raise self.ignore_responses |= set(waiting_for) def check(self, repair=False, save_space=False): return self.call('check', repair, save_space) def commit(self, save_space=False): return self.call('commit', save_space) def rollback(self, *args): return self.call('rollback') def destroy(self): return self.call('destroy') def __len__(self): return self.call('__len__') def list(self, limit=None, marker=None): return self.call('list', limit, marker) def get(self, id_): for resp in self.get_many([id_]): return resp def get_many(self, ids, is_preloaded=False): for resp in self.call_many('get', [(id_,) for id_ in ids], is_preloaded=is_preloaded): yield resp def put(self, id_, data, wait=True): return self.call('put', id_, data, wait=wait) def delete(self, id_, wait=True): return self.call('delete', id_, wait=wait) def save_key(self, keydata): return self.call('save_key', keydata) def load_key(self): return self.call('load_key') def break_lock(self): return self.call('break_lock') def close(self): if self.p: self.p.stdin.close() self.p.stdout.close() self.p.wait() self.p = None def preload(self, ids): self.preload_ids += ids def handle_remote_line(line): """Handle a remote log line.""" assert line.endswith(('\r', '\n')) if line.startswith('$LOG '): _, level, msg = line.split(' ', 2) level = getattr(logging, level, logging.CRITICAL) # str -> int # server format: '$LOG Remote: ' logging.log(level, msg.rstrip()) else: # We circumvent logging to preserve carriage returns (\r) which are generated by remote progress displays. sys.stderr.write('Remote: ' + line) class RepositoryNoCache: """A not caching Repository wrapper, passes through to repository. Just to have same API (including the context manager) as RepositoryCache. """ def __init__(self, repository): self.repository = repository def close(self): pass def __enter__(self): return self def __exit__(self, exc_type, exc_val, exc_tb): self.close() def get(self, key): return next(self.get_many([key])) def get_many(self, keys): for data in self.repository.get_many(keys): yield data class RepositoryCache(RepositoryNoCache): """A caching Repository wrapper Caches Repository GET operations using a local temporary Repository. """ # maximum object size that will be cached, 64 kiB. THRESHOLD = 2**16 def __init__(self, repository): super().__init__(repository) tmppath = tempfile.mkdtemp(prefix='borg-tmp') self.caching_repo = Repository(tmppath, create=True, exclusive=True) self.caching_repo.__enter__() # handled by context manager in base class def close(self): if self.caching_repo is not None: self.caching_repo.destroy() self.caching_repo = None def get_many(self, keys): unknown_keys = [key for key in keys if key not in self.caching_repo] repository_iterator = zip(unknown_keys, self.repository.get_many(unknown_keys)) for key in keys: try: yield self.caching_repo.get(key) except Repository.ObjectNotFound: for key_, data in repository_iterator: if key_ == key: if len(data) <= self.THRESHOLD: self.caching_repo.put(key, data) yield data break # Consume any pending requests for _ in repository_iterator: pass def cache_if_remote(repository): if isinstance(repository, RemoteRepository): return RepositoryCache(repository) else: return RepositoryNoCache(repository) borgbackup-1.0.13/borg/repository.py0000644000175000017500000011422413431513133017414 0ustar useruser00000000000000from configparser import ConfigParser from binascii import unhexlify from datetime import datetime from itertools import islice import errno import os import shutil import struct from zlib import crc32 import msgpack from .logger import create_logger logger = create_logger() from .helpers import Error, ErrorWithTraceback, IntegrityError, Location, ProgressIndicatorPercent, bin_to_hex from .helpers import LIST_SCAN_LIMIT, MAX_OBJECT_SIZE, MAX_DATA_SIZE, ISO_FORMAT from .hashindex import NSIndex from .locking import Lock, LockError, LockErrorT from .lrucache import LRUCache from .platform import sync_dir MAGIC = b'BORG_SEG' MAGIC_LEN = len(MAGIC) TAG_PUT = 0 TAG_DELETE = 1 TAG_COMMIT = 2 class Repository: """Filesystem based transactional key value store On disk layout: dir/README dir/config dir/data// dir/index.X dir/hints.X """ DEFAULT_MAX_SEGMENT_SIZE = 5 * 1024 * 1024 DEFAULT_SEGMENTS_PER_DIR = 10000 class DoesNotExist(Error): """Repository {} does not exist.""" class AlreadyExists(Error): """Repository {} already exists.""" class InvalidRepository(Error): """{} is not a valid repository. Check repo config.""" class CheckNeeded(ErrorWithTraceback): """Inconsistency detected. Please run "borg check {}".""" class ObjectNotFound(ErrorWithTraceback): """Object with key {} not found in repository {}.""" def __init__(self, path, create=False, exclusive=False, lock_wait=None, lock=True, append_only=False): self.path = os.path.abspath(path) self._location = Location('file://%s' % self.path) self.io = None self.lock = None self.index = None self._active_txn = False self.lock_wait = lock_wait self.do_lock = lock self.do_create = create self.exclusive = exclusive self.append_only = append_only def __del__(self): if self.lock: self.close() assert False, "cleanup happened in Repository.__del__" def __repr__(self): return '<%s %s>' % (self.__class__.__name__, self.path) def __enter__(self): if self.do_create: self.do_create = False self.create(self.path) self.open(self.path, bool(self.exclusive), lock_wait=self.lock_wait, lock=self.do_lock) return self def __exit__(self, exc_type, exc_val, exc_tb): if exc_type is not None: no_space_left_on_device = exc_type is OSError and exc_val.errno == errno.ENOSPC # The ENOSPC could have originated somewhere else besides the Repository. The cleanup is always safe, unless # EIO or FS corruption ensues, which is why we specifically check for ENOSPC. if self._active_txn and no_space_left_on_device: logger.warning('No space left on device, cleaning up partial transaction to free space.') self.io.cleanup(self.io.get_segments_transaction_id()) self.rollback() self.close() def create(self, path): """Create a new empty repository at `path` """ if os.path.exists(path) and (not os.path.isdir(path) or os.listdir(path)): raise self.AlreadyExists(path) if not os.path.exists(path): os.mkdir(path) with open(os.path.join(path, 'README'), 'w') as fd: fd.write('This is a Borg repository\n') os.mkdir(os.path.join(path, 'data')) config = ConfigParser(interpolation=None) config.add_section('repository') config.set('repository', 'version', '1') config.set('repository', 'segments_per_dir', str(self.DEFAULT_SEGMENTS_PER_DIR)) config.set('repository', 'max_segment_size', str(self.DEFAULT_MAX_SEGMENT_SIZE)) config.set('repository', 'append_only', str(int(self.append_only))) config.set('repository', 'id', bin_to_hex(os.urandom(32))) self.save_config(path, config) def save_config(self, path, config): config_path = os.path.join(path, 'config') with open(config_path, 'w') as fd: config.write(fd) def save_key(self, keydata): assert self.config keydata = keydata.decode('utf-8') # remote repo: msgpack issue #99, getting bytes self.config.set('repository', 'key', keydata) self.save_config(self.path, self.config) def load_key(self): keydata = self.config.get('repository', 'key') return keydata.encode('utf-8') # remote repo: msgpack issue #99, returning bytes def destroy(self): """Destroy the repository at `self.path` """ if self.append_only: raise ValueError(self.path + " is in append-only mode") self.close() os.remove(os.path.join(self.path, 'config')) # kill config first shutil.rmtree(self.path) def get_index_transaction_id(self): indices = sorted(int(fn[6:]) for fn in os.listdir(self.path) if fn.startswith('index.') and fn[6:].isdigit() and os.stat(os.path.join(self.path, fn)).st_size != 0) if indices: return indices[-1] else: return None def get_transaction_id(self): index_transaction_id = self.get_index_transaction_id() segments_transaction_id = self.io.get_segments_transaction_id() if index_transaction_id is not None and segments_transaction_id is None: # we have a transaction id from the index, but we did not find *any* # commit in the segment files (thus no segments transaction id). # this can happen if a lot of segment files are lost, e.g. due to a # filesystem or hardware malfunction. it means we have no identifiable # valid (committed) state of the repo which we could use. msg = '%s" - although likely this is "beyond repair' % self.path # dirty hack raise self.CheckNeeded(msg) # Attempt to automatically rebuild index if we crashed between commit # tag write and index save if index_transaction_id != segments_transaction_id: if index_transaction_id is not None and index_transaction_id > segments_transaction_id: replay_from = None else: replay_from = index_transaction_id self.replay_segments(replay_from, segments_transaction_id) return self.get_index_transaction_id() def break_lock(self): Lock(os.path.join(self.path, 'lock')).break_lock() def migrate_lock(self, old_id, new_id): # note: only needed for local repos if self.lock is not None: self.lock.migrate_lock(old_id, new_id) def open(self, path, exclusive, lock_wait=None, lock=True): self.path = path if not os.path.isdir(path): raise self.DoesNotExist(path) if lock: self.lock = Lock(os.path.join(path, 'lock'), exclusive, timeout=lock_wait).acquire() else: self.lock = None self.config = ConfigParser(interpolation=None) self.config.read(os.path.join(self.path, 'config')) if 'repository' not in self.config.sections() or self.config.getint('repository', 'version') != 1: self.close() raise self.InvalidRepository(path) self.max_segment_size = self.config.getint('repository', 'max_segment_size') self.segments_per_dir = self.config.getint('repository', 'segments_per_dir') # append_only can be set in the constructor # it shouldn't be overridden (True -> False) here self.append_only = self.append_only or self.config.getboolean('repository', 'append_only', fallback=False) self.id = unhexlify(self.config.get('repository', 'id').strip()) self.io = LoggedIO(self.path, self.max_segment_size, self.segments_per_dir) def close(self): if self.lock: if self.io: self.io.close() self.io = None self.lock.release() self.lock = None def commit(self, save_space=False): """Commit transaction """ self.io.write_commit() if not self.append_only: self.compact_segments(save_space=save_space) self.write_index() self.rollback() def open_index(self, transaction_id): if transaction_id is None: return NSIndex() return NSIndex.read((os.path.join(self.path, 'index.%d') % transaction_id).encode('utf-8')) def prepare_txn(self, transaction_id, do_cleanup=True): self._active_txn = True if not self.lock.got_exclusive_lock(): if self.exclusive is not None: # self.exclusive is either True or False, thus a new client is active here. # if it is False and we get here, the caller did not use exclusive=True although # it is needed for a write operation. if it is True and we get here, something else # went very wrong, because we should have a exclusive lock, but we don't. raise AssertionError("bug in code, exclusive lock should exist here") # if we are here, this is an old client talking to a new server (expecting lock upgrade). # or we are replaying segments and might need a lock upgrade for that. try: self.lock.upgrade() except (LockError, LockErrorT): # if upgrading the lock to exclusive fails, we do not have an # active transaction. this is important for "serve" mode, where # the repository instance lives on - even if exceptions happened. self._active_txn = False raise if not self.index or transaction_id is None: self.index = self.open_index(transaction_id) if transaction_id is None: self.segments = {} # XXX bad name: usage_count_of_segment_x = self.segments[x] self.compact = set() # XXX bad name: segments_needing_compaction = self.compact else: if do_cleanup: self.io.cleanup(transaction_id) with open(os.path.join(self.path, 'hints.%d' % transaction_id), 'rb') as fd: hints = msgpack.unpack(fd) hints_version = hints[b'version'] if hints_version not in (1, 2): raise ValueError('Unknown hints file version: %d' % hints_version) self.segments = hints[b'segments'] if hints_version == 1: self.compact = set(hints[b'compact']) elif hints_version == 2: self.compact = set(hints[b'compact'].keys()) def write_index(self): hints = {b'version': 1, b'segments': self.segments, b'compact': list(self.compact)} transaction_id = self.io.get_segments_transaction_id() assert transaction_id is not None hints_file = os.path.join(self.path, 'hints.%d' % transaction_id) with open(hints_file + '.tmp', 'wb') as fd: msgpack.pack(hints, fd) fd.flush() os.fsync(fd.fileno()) os.rename(hints_file + '.tmp', hints_file) self.index.write(os.path.join(self.path, 'index.tmp')) os.rename(os.path.join(self.path, 'index.tmp'), os.path.join(self.path, 'index.%d' % transaction_id)) if self.append_only: with open(os.path.join(self.path, 'transactions'), 'a') as log: print('transaction %d, UTC time %s' % ( transaction_id, datetime.utcnow().strftime(ISO_FORMAT)), file=log) # Remove old indices current = '.%d' % transaction_id for name in os.listdir(self.path): if not name.startswith('index.') and not name.startswith('hints.'): continue if name.endswith(current): continue os.unlink(os.path.join(self.path, name)) self.index = None def compact_segments(self, save_space=False): """Compact sparse segments by copying data into new segments """ if not self.compact: return index_transaction_id = self.get_index_transaction_id() segments = self.segments unused = [] # list of segments, that are not used anymore def complete_xfer(): # complete the transfer (usually exactly when some target segment # is full, or at the very end when everything is processed) nonlocal unused # commit the new, compact, used segments self.io.write_commit() # get rid of the old, sparse, unused segments. free space. for segment in unused: assert self.segments.pop(segment) == 0 self.io.delete_segment(segment) unused = [] # The first segment compaction creates, if any first_new_segment = self.io.get_latest_segment() + 1 for segment in sorted(self.compact): if self.io.segment_exists(segment): for tag, key, offset, data in self.io.iter_objects(segment, include_data=True): if tag == TAG_PUT and self.index.get(key, (-1, -1)) == (segment, offset): try: new_segment, offset = self.io.write_put(key, data, raise_full=save_space) except LoggedIO.SegmentFull: complete_xfer() new_segment, offset = self.io.write_put(key, data) self.index[key] = new_segment, offset segments.setdefault(new_segment, 0) segments[new_segment] += 1 segments[segment] -= 1 elif tag == TAG_DELETE: if index_transaction_id is None or segment > index_transaction_id: # (introduced in 6425d16aa84be1eaaf88) # This is needed to avoid object un-deletion if we crash between the commit and the deletion # of old segments in complete_xfer(). # # However, this only happens if the crash also affects the FS to the effect that file deletions # did not materialize consistently after journal recovery. If they always materialize in-order # then this is not a problem, because the old segment containing a deleted object would be deleted # before the segment containing the delete. # # Consider the following series of operations if we would not do this, ie. this entire if: # would be removed. # Columns are segments, lines are different keys (line 1 = some key, line 2 = some other key) # Legend: P=TAG_PUT, D=TAG_DELETE, c=commit, i=index is written for latest commit # # Segment | 1 | 2 | 3 # --------+-------+-----+------ # Key 1 | P | D | # Key 2 | P | | P # commits | c i | c | c i # --------+-------+-----+------ # ^- compact_segments starts # ^- complete_xfer commits, after that complete_xfer deletes # segments 1 and 2 (and then the index would be written). # # Now we crash. But only segment 2 gets deleted, while segment 1 is still around. Now key 1 # is suddenly undeleted (because the delete in segment 2 is now missing). # Again, note the requirement here. We delete these in the correct order that this doesn't happen, # and only if the FS materialization of these deletes is reordered or parts dropped this can happen. # In this case it doesn't cause outright corruption, 'just' an index count mismatch, which will be # fixed by borg-check --repair. # # Note that in this check the index state is the proxy for a "most definitely settled" repository state, # ie. the assumption is that *all* operations on segments <= index state are completed and stable. try: new_segment = self.io.write_delete(key, raise_full=save_space) except LoggedIO.SegmentFull: complete_xfer() new_segment = self.io.write_delete(key) self.compact.add(new_segment) self.segments.setdefault(new_segment, 0) assert segments[segment] == 0 unused.append(segment) complete_xfer() # Moving of deletes creates new sparse segments, only store these. All other segments # are compact now. self.compact = {segment for segment in self.compact if segment >= first_new_segment} def replay_segments(self, index_transaction_id, segments_transaction_id): # fake an old client, so that in case we do not have an exclusive lock yet, prepare_txn will upgrade the lock: remember_exclusive = self.exclusive self.exclusive = None self.prepare_txn(index_transaction_id, do_cleanup=False) try: segment_count = sum(1 for _ in self.io.segment_iterator()) pi = ProgressIndicatorPercent(total=segment_count, msg="Replaying segments %3.0f%%", same_line=True) for i, (segment, filename) in enumerate(self.io.segment_iterator()): pi.show(i) if index_transaction_id is not None and segment <= index_transaction_id: continue if segment > segments_transaction_id: break objects = self.io.iter_objects(segment) self._update_index(segment, objects) pi.finish() self.write_index() finally: self.exclusive = remember_exclusive self.rollback() def _update_index(self, segment, objects, report=None): """some code shared between replay_segments and check""" self.segments[segment] = 0 for tag, key, offset in objects: if tag == TAG_PUT: try: s, _ = self.index[key] self.compact.add(s) self.segments[s] -= 1 except KeyError: pass self.index[key] = segment, offset self.segments[segment] += 1 elif tag == TAG_DELETE: try: s, _ = self.index.pop(key) self.segments[s] -= 1 self.compact.add(s) except KeyError: pass self.compact.add(segment) elif tag == TAG_COMMIT: continue else: msg = 'Unexpected tag {} in segment {}'.format(tag, segment) if report is None: raise self.CheckNeeded(msg) else: report(msg) if self.segments[segment] == 0: self.compact.add(segment) def check(self, repair=False, save_space=False): """Check repository consistency This method verifies all segment checksums and makes sure the index is consistent with the data stored in the segments. """ if self.append_only and repair: raise ValueError(self.path + " is in append-only mode") error_found = False def report_error(msg): nonlocal error_found error_found = True logger.error(msg) logger.info('Starting repository check') assert not self._active_txn try: transaction_id = self.get_transaction_id() current_index = self.open_index(transaction_id) except Exception: transaction_id = self.io.get_segments_transaction_id() current_index = None if transaction_id is None: transaction_id = self.get_index_transaction_id() if transaction_id is None: transaction_id = self.io.get_latest_segment() if transaction_id is None: report_error('This repository contains no valid data.') return False if repair: self.io.cleanup(transaction_id) segments_transaction_id = self.io.get_segments_transaction_id() self.prepare_txn(None) # self.index, self.compact, self.segments all empty now! segment_count = sum(1 for _ in self.io.segment_iterator()) pi = ProgressIndicatorPercent(total=segment_count, msg="Checking segments %3.1f%%", step=0.1, same_line=True) for i, (segment, filename) in enumerate(self.io.segment_iterator()): pi.show(i) if segment > transaction_id: continue try: objects = list(self.io.iter_objects(segment)) except IntegrityError as err: report_error(str(err)) objects = [] if repair: self.io.recover_segment(segment, filename) objects = list(self.io.iter_objects(segment)) self._update_index(segment, objects, report_error) pi.finish() # self.index, self.segments, self.compact now reflect the state of the segment files up to # We might need to add a commit tag if no committed segment is found if repair and segments_transaction_id is None: report_error('Adding commit tag to segment {}'.format(transaction_id)) self.io.segment = transaction_id + 1 self.io.write_commit() if current_index and not repair: # current_index = "as found on disk" # self.index = "as rebuilt in-memory from segments" if len(current_index) != len(self.index): report_error('Index object count mismatch. {} != {}'.format(len(current_index), len(self.index))) elif current_index: for key, value in self.index.iteritems(): if current_index.get(key, (-1, -1)) != value: report_error('Index mismatch for key {}. {} != {}'.format(key, value, current_index.get(key, (-1, -1)))) if repair: self.compact_segments(save_space=save_space) self.write_index() self.rollback() if error_found: if repair: logger.info('Completed repository check, errors found and repaired.') else: logger.error('Completed repository check, errors found.') else: logger.info('Completed repository check, no problems found.') return not error_found or repair def rollback(self): """ """ self.index = None self._active_txn = False def __len__(self): if not self.index: self.index = self.open_index(self.get_transaction_id()) return len(self.index) def __contains__(self, id): if not self.index: self.index = self.open_index(self.get_transaction_id()) return id in self.index def list(self, limit=None, marker=None): if not self.index: self.index = self.open_index(self.get_transaction_id()) return [id_ for id_, _ in islice(self.index.iteritems(marker=marker), limit)] def get(self, id_): if not self.index: self.index = self.open_index(self.get_transaction_id()) try: segment, offset = self.index[id_] return self.io.read(segment, offset, id_) except KeyError: raise self.ObjectNotFound(id_, self.path) from None def get_many(self, ids, is_preloaded=False): for id_ in ids: yield self.get(id_) def put(self, id, data, wait=True): if not self._active_txn: self.prepare_txn(self.get_transaction_id()) try: segment, _ = self.index[id] self.segments[segment] -= 1 self.compact.add(segment) segment = self.io.write_delete(id) self.segments.setdefault(segment, 0) self.compact.add(segment) except KeyError: pass segment, offset = self.io.write_put(id, data) self.segments.setdefault(segment, 0) self.segments[segment] += 1 self.index[id] = segment, offset def delete(self, id, wait=True): if not self._active_txn: self.prepare_txn(self.get_transaction_id()) try: segment, offset = self.index.pop(id) except KeyError: raise self.ObjectNotFound(id, self.path) from None self.segments[segment] -= 1 self.compact.add(segment) segment = self.io.write_delete(id) self.compact.add(segment) self.segments.setdefault(segment, 0) def preload(self, ids): """Preload objects (only applies to remote repositories) """ class LoggedIO: class SegmentFull(Exception): """raised when a segment is full, before opening next""" header_fmt = struct.Struct(' transaction_id: if segment in self.fds: del self.fds[segment] os.unlink(filename) else: break def is_committed_segment(self, segment): """Check if segment ends with a COMMIT_TAG tag """ try: iterator = self.iter_objects(segment) except IntegrityError: return False with open(self.segment_filename(segment), 'rb') as fd: try: fd.seek(-self.header_fmt.size, os.SEEK_END) except OSError as e: # return False if segment file is empty or too small if e.errno == errno.EINVAL: return False raise e if fd.read(self.header_fmt.size) != self.COMMIT: return False seen_commit = False while True: try: tag, key, offset = next(iterator) except IntegrityError: return False except StopIteration: break if tag == TAG_COMMIT: seen_commit = True continue if seen_commit: return False return seen_commit def segment_filename(self, segment): return os.path.join(self.path, 'data', str(segment // self.segments_per_dir), str(segment)) def get_write_fd(self, no_new=False, raise_full=False): if not no_new and self.offset and self.offset > self.limit: if raise_full: raise self.SegmentFull self.close_segment() if not self._write_fd: if self.segment % self.segments_per_dir == 0: dirname = os.path.join(self.path, 'data', str(self.segment // self.segments_per_dir)) if not os.path.exists(dirname): os.mkdir(dirname) sync_dir(os.path.join(self.path, 'data')) # play safe: fail if file exists (do not overwrite existing contents, do not append) self._write_fd = open(self.segment_filename(self.segment), 'xb') self._write_fd.write(MAGIC) self.offset = MAGIC_LEN if self.segment in self.fds: # we may have a cached fd for a segment file we already deleted and # we are writing now a new segment file to same file name. get rid of # of the cached fd that still refers to the old file, so it will later # get repopulated (on demand) with a fd that refers to the new file. del self.fds[self.segment] return self._write_fd def get_fd(self, segment): try: return self.fds[segment] except KeyError: fd = open(self.segment_filename(segment), 'rb') self.fds[segment] = fd return fd def delete_segment(self, segment): if segment in self.fds: del self.fds[segment] try: os.unlink(self.segment_filename(segment)) except FileNotFoundError: pass def segment_exists(self, segment): return os.path.exists(self.segment_filename(segment)) def iter_objects(self, segment, include_data=False): fd = self.get_fd(segment) fd.seek(0) if fd.read(MAGIC_LEN) != MAGIC: raise IntegrityError('Invalid segment magic [segment {}, offset {}]'.format(segment, 0)) offset = MAGIC_LEN header = fd.read(self.header_fmt.size) while header: size, tag, key, data = self._read(fd, self.header_fmt, header, segment, offset, (TAG_PUT, TAG_DELETE, TAG_COMMIT)) if include_data: yield tag, key, offset, data else: yield tag, key, offset offset += size # we must get the fd via get_fd() here again as we yielded to our caller and it might # have triggered closing of the fd we had before (e.g. by calling io.read() for # different segment(s)). # by calling get_fd() here again we also make our fd "recently used" so it likely # does not get kicked out of self.fds LRUcache. fd = self.get_fd(segment) fd.seek(offset) header = fd.read(self.header_fmt.size) def recover_segment(self, segment, filename): if segment in self.fds: del self.fds[segment] with open(filename, 'rb') as fd: data = memoryview(fd.read()) os.rename(filename, filename + '.beforerecover') logger.info('attempting to recover ' + filename) with open(filename, 'wb') as fd: fd.write(MAGIC) while len(data) >= self.header_fmt.size: crc, size, tag = self.header_fmt.unpack(data[:self.header_fmt.size]) if size < self.header_fmt.size or size > len(data): data = data[1:] continue if crc32(data[4:size]) & 0xffffffff != crc: data = data[1:] continue fd.write(data[:size]) data = data[size:] def read(self, segment, offset, id): if segment == self.segment and self._write_fd: self._write_fd.flush() fd = self.get_fd(segment) fd.seek(offset) header = fd.read(self.put_header_fmt.size) size, tag, key, data = self._read(fd, self.put_header_fmt, header, segment, offset, (TAG_PUT, )) if id != key: raise IntegrityError('Invalid segment entry header, is not for wanted id [segment {}, offset {}]'.format( segment, offset)) return data def _read(self, fd, fmt, header, segment, offset, acceptable_tags): # some code shared by read() and iter_objects() try: hdr_tuple = fmt.unpack(header) except struct.error as err: raise IntegrityError('Invalid segment entry header [segment {}, offset {}]: {}'.format( segment, offset, err)) from None if fmt is self.put_header_fmt: crc, size, tag, key = hdr_tuple elif fmt is self.header_fmt: crc, size, tag = hdr_tuple key = None else: raise TypeError("_read called with unsupported format") if size > MAX_OBJECT_SIZE: # if you get this on an archive made with borg < 1.0.7 and millions of files and # you need to restore it, you can disable this check by using "if False:" above. raise IntegrityError('Invalid segment entry size {} - too big [segment {}, offset {}]'.format( size, segment, offset)) if size < fmt.size: raise IntegrityError('Invalid segment entry size {} - too small [segment {}, offset {}]'.format( size, segment, offset)) length = size - fmt.size data = fd.read(length) if len(data) != length: raise IntegrityError('Segment entry data short read [segment {}, offset {}]: expected {}, got {} bytes'.format( segment, offset, length, len(data))) if crc32(data, crc32(memoryview(header)[4:])) & 0xffffffff != crc: raise IntegrityError('Segment entry checksum mismatch [segment {}, offset {}]'.format( segment, offset)) if tag not in acceptable_tags: raise IntegrityError('Invalid segment entry header, did not get acceptable tag [segment {}, offset {}]'.format( segment, offset)) if key is None and tag in (TAG_PUT, TAG_DELETE): key, data = data[:32], data[32:] return size, tag, key, data def write_put(self, id, data, raise_full=False): data_size = len(data) if data_size > MAX_DATA_SIZE: # this would push the segment entry size beyond MAX_OBJECT_SIZE. raise IntegrityError('More than allowed put data [{} > {}]'.format(data_size, MAX_DATA_SIZE)) fd = self.get_write_fd(raise_full=raise_full) size = data_size + self.put_header_fmt.size offset = self.offset header = self.header_no_crc_fmt.pack(size, TAG_PUT) crc = self.crc_fmt.pack(crc32(data, crc32(id, crc32(header))) & 0xffffffff) fd.write(b''.join((crc, header, id, data))) self.offset += size return self.segment, offset def write_delete(self, id, raise_full=False): fd = self.get_write_fd(raise_full=raise_full) header = self.header_no_crc_fmt.pack(self.put_header_fmt.size, TAG_DELETE) crc = self.crc_fmt.pack(crc32(id, crc32(header)) & 0xffffffff) fd.write(b''.join((crc, header, id))) self.offset += self.put_header_fmt.size return self.segment def write_commit(self): fd = self.get_write_fd(no_new=True) header = self.header_no_crc_fmt.pack(self.header_fmt.size, TAG_COMMIT) crc = self.crc_fmt.pack(crc32(header) & 0xffffffff) # first fsync(fd) here (to ensure data supposedly hits the disk before the commit tag) fd.flush() os.fsync(fd.fileno()) fd.write(b''.join((crc, header))) self.close_segment() # after-commit fsync() def close_segment(self): # set self._write_fd to None early to guard against reentry from error handling code pathes: fd, self._write_fd = self._write_fd, None if fd is not None: dirname = None try: self.segment += 1 self.offset = 0 dirname = os.path.dirname(fd.name) fd.flush() fileno = fd.fileno() os.fsync(fileno) if hasattr(os, 'posix_fadvise'): # only on UNIX try: # tell the OS that it does not need to cache what we just wrote, # avoids spoiling the cache for the OS and other processes. os.posix_fadvise(fileno, 0, 0, os.POSIX_FADV_DONTNEED) except OSError: # usually, posix_fadvise can't fail for us, but there seem to # be failures when running borg under docker on ARM, likely due # to a bug outside of borg. # also, there is a python wrapper bug, always giving errno = 0. # https://github.com/borgbackup/borg/issues/2095 # as this call is not critical for correct function (just to # optimize cache usage), we ignore these errors. pass finally: fd.close() if dirname: sync_dir(dirname) assert LoggedIO.put_header_fmt.size == 41 # see helpers.MAX_OBJECT_SIZE borgbackup-1.0.13/borg/shellpattern.py0000644000175000017500000000406413431513133017702 0ustar useruser00000000000000import re import os def translate(pat): """Translate a shell-style pattern to a regular expression. The pattern may include ``**`` ( stands for the platform-specific path separator; "/" on POSIX systems) for matching zero or more directory levels and "*" for matching zero or more arbitrary characters with the exception of any path separator. Wrap meta-characters in brackets for a literal match (i.e. "[?]" to match the literal character "?"). This function is derived from the "fnmatch" module distributed with the Python standard library. Copyright (C) 2001-2016 Python Software Foundation. All rights reserved. TODO: support {alt1,alt2} shell-style alternatives """ sep = os.path.sep n = len(pat) i = 0 res = "" while i < n: c = pat[i] i += 1 if c == "*": if i + 1 < n and pat[i] == "*" and pat[i + 1] == sep: # **/ == wildcard for 0+ full (relative) directory names with trailing slashes; the forward slash stands # for the platform-specific path separator res += r"(?:[^\%s]*\%s)*" % (sep, sep) i += 2 else: # * == wildcard for name parts (does not cross path separator) res += r"[^\%s]*" % sep elif c == "?": # ? == any single character excluding path separator res += r"[^\%s]" % sep elif c == "[": j = i if j < n and pat[j] == "!": j += 1 if j < n and pat[j] == "]": j += 1 while j < n and pat[j] != "]": j += 1 if j >= n: res += "\\[" else: stuff = pat[i:j].replace("\\", "\\\\") i = j + 1 if stuff[0] == "!": stuff = "^" + stuff[1:] elif stuff[0] == "^": stuff = "\\" + stuff res += "[%s]" % stuff else: res += re.escape(c) return res + r"\Z(?ms)" borgbackup-1.0.13/borg/testsuite/0000755000175000017500000000000013431514213016650 5ustar useruser00000000000000borgbackup-1.0.13/borg/testsuite/__init__.py0000644000175000017500000001346513431513133020772 0ustar useruser00000000000000from contextlib import contextmanager import filecmp import os import posix import stat import sys import sysconfig import time import unittest from ..xattr import get_all from ..logger import setup_logging from ..platform import umount try: import llfuse # Does this version of llfuse support ns precision? have_fuse_mtime_ns = hasattr(llfuse.EntryAttributes, 'st_mtime_ns') except ImportError: have_fuse_mtime_ns = False has_lchflags = hasattr(os, 'lchflags') # The mtime get/set precision varies on different OS and Python versions if 'HAVE_FUTIMENS' in getattr(posix, '_have_functions', []): st_mtime_ns_round = 0 elif 'HAVE_UTIMES' in sysconfig.get_config_vars(): st_mtime_ns_round = -6 else: st_mtime_ns_round = -9 if sys.platform.startswith('netbsd'): st_mtime_ns_round = -4 # only >1 microsecond resolution here? # Ensure that the loggers exist for all tests setup_logging() def no_selinux(x): # selinux fails our FUSE tests, thus ignore selinux xattrs SELINUX_KEY = 'security.selinux' if isinstance(x, dict): return {k: v for k, v in x.items() if k != SELINUX_KEY} if isinstance(x, list): return [k for k in x if k != SELINUX_KEY] class BaseTestCase(unittest.TestCase): """ """ assert_in = unittest.TestCase.assertIn assert_not_in = unittest.TestCase.assertNotIn assert_equal = unittest.TestCase.assertEqual assert_not_equal = unittest.TestCase.assertNotEqual assert_raises = unittest.TestCase.assertRaises assert_true = unittest.TestCase.assertTrue @contextmanager def assert_creates_file(self, path): self.assert_true(not os.path.exists(path), '{} should not exist'.format(path)) yield self.assert_true(os.path.exists(path), '{} should exist'.format(path)) def assert_dirs_equal(self, dir1, dir2, **kwargs): diff = filecmp.dircmp(dir1, dir2) self._assert_dirs_equal_cmp(diff, **kwargs) def _assert_dirs_equal_cmp(self, diff, ignore_bsdflags=False, ignore_xattrs=False): self.assert_equal(diff.left_only, []) self.assert_equal(diff.right_only, []) self.assert_equal(diff.diff_files, []) self.assert_equal(diff.funny_files, []) for filename in diff.common: path1 = os.path.join(diff.left, filename) path2 = os.path.join(diff.right, filename) s1 = os.stat(path1, follow_symlinks=False) s2 = os.stat(path2, follow_symlinks=False) # Assume path2 is on FUSE if st_dev is different fuse = s1.st_dev != s2.st_dev attrs = ['st_mode', 'st_uid', 'st_gid', 'st_rdev'] if has_lchflags and not ignore_bsdflags: attrs.append('st_flags') if not fuse or not os.path.isdir(path1): # dir nlink is always 1 on our fuse filesystem attrs.append('st_nlink') d1 = [filename] + [getattr(s1, a) for a in attrs] d2 = [filename] + [getattr(s2, a) for a in attrs] # ignore st_rdev if file is not a block/char device, fixes #203 if not stat.S_ISCHR(d1[1]) and not stat.S_ISBLK(d1[1]): d1[4] = None if not stat.S_ISCHR(d2[1]) and not stat.S_ISBLK(d2[1]): d2[4] = None # Older versions of llfuse do not support ns precision properly if fuse and not have_fuse_mtime_ns: d1.append(round(s1.st_mtime_ns, -4)) d2.append(round(s2.st_mtime_ns, -4)) else: d1.append(round(s1.st_mtime_ns, st_mtime_ns_round)) d2.append(round(s2.st_mtime_ns, st_mtime_ns_round)) if not ignore_xattrs: d1.append(no_selinux(get_all(path1, follow_symlinks=False))) d2.append(no_selinux(get_all(path2, follow_symlinks=False))) self.assert_equal(d1, d2) for sub_diff in diff.subdirs.values(): self._assert_dirs_equal_cmp(sub_diff, ignore_bsdflags=ignore_bsdflags, ignore_xattrs=ignore_xattrs) @contextmanager def fuse_mount(self, location, mountpoint, *options): os.mkdir(mountpoint) args = ['mount', location, mountpoint] + list(options) self.cmd(*args, fork=True) self.wait_for_mount(mountpoint) yield umount(mountpoint) os.rmdir(mountpoint) # Give the daemon some time to exit time.sleep(.2) def wait_for_mount(self, path, timeout=5): """Wait until a filesystem is mounted on `path` """ timeout += time.time() while timeout > time.time(): if os.path.ismount(path): return time.sleep(.1) raise Exception('wait_for_mount(%s) timeout' % path) class changedir: def __init__(self, dir): self.dir = dir def __enter__(self): self.old = os.getcwd() os.chdir(self.dir) def __exit__(self, *args, **kw): os.chdir(self.old) class environment_variable: def __init__(self, **values): self.values = values self.old_values = {} def __enter__(self): for k, v in self.values.items(): self.old_values[k] = os.environ.get(k) if v is None: os.environ.pop(k, None) else: os.environ[k] = v def __exit__(self, *args, **kw): for k, v in self.old_values.items(): if v is None: os.environ.pop(k, None) else: os.environ[k] = v class FakeInputs: """Simulate multiple user inputs, can be used as input() replacement""" def __init__(self, inputs): self.inputs = inputs def __call__(self, prompt=None): if prompt is not None: print(prompt, end='') try: return self.inputs.pop(0) except IndexError: raise EOFError from None borgbackup-1.0.13/borg/testsuite/archive.py0000644000175000017500000001344313431513133020650 0ustar useruser00000000000000from collections import OrderedDict from datetime import datetime, timezone from unittest.mock import Mock import msgpack import pytest from ..archive import Archive, CacheChunkBuffer, RobustUnpacker, valid_msgpacked_dict, ITEM_KEYS from ..archive import BackupOSError, backup_io, backup_io_iter from ..key import PlaintextKey from ..helpers import Manifest from . import BaseTestCase class MockCache: def __init__(self): self.objects = {} def add_chunk(self, id, data, stats=None): self.objects[id] = data return id, len(data), len(data) class ArchiveTimestampTestCase(BaseTestCase): def _test_timestamp_parsing(self, isoformat, expected): repository = Mock() key = PlaintextKey(repository) manifest = Manifest(repository, key) a = Archive(repository, key, manifest, 'test', create=True) a.metadata = {b'time': isoformat} self.assert_equal(a.ts, expected) def test_with_microseconds(self): self._test_timestamp_parsing( '1970-01-01T00:00:01.000001', datetime(1970, 1, 1, 0, 0, 1, 1, timezone.utc)) def test_without_microseconds(self): self._test_timestamp_parsing( '1970-01-01T00:00:01', datetime(1970, 1, 1, 0, 0, 1, 0, timezone.utc)) class ChunkBufferTestCase(BaseTestCase): def test(self): data = [{b'foo': 1}, {b'bar': 2}] cache = MockCache() key = PlaintextKey(None) chunks = CacheChunkBuffer(cache, key, None) for d in data: chunks.add(d) chunks.flush() chunks.flush(flush=True) self.assert_equal(len(chunks.chunks), 2) unpacker = msgpack.Unpacker() for id in chunks.chunks: unpacker.feed(cache.objects[id]) self.assert_equal(data, list(unpacker)) class RobustUnpackerTestCase(BaseTestCase): def make_chunks(self, items): return b''.join(msgpack.packb({'path': item}) for item in items) def _validator(self, value): return isinstance(value, dict) and value.get(b'path') in (b'foo', b'bar', b'boo', b'baz') def process(self, input): unpacker = RobustUnpacker(validator=self._validator, item_keys=ITEM_KEYS) result = [] for should_sync, chunks in input: if should_sync: unpacker.resync() for data in chunks: unpacker.feed(data) for item in unpacker: result.append(item) return result def test_extra_garbage_no_sync(self): chunks = [(False, [self.make_chunks([b'foo', b'bar'])]), (False, [b'garbage'] + [self.make_chunks([b'boo', b'baz'])])] result = self.process(chunks) self.assert_equal(result, [ {b'path': b'foo'}, {b'path': b'bar'}, 103, 97, 114, 98, 97, 103, 101, {b'path': b'boo'}, {b'path': b'baz'}]) def split(self, left, length): parts = [] while left: parts.append(left[:length]) left = left[length:] return parts def test_correct_stream(self): chunks = self.split(self.make_chunks([b'foo', b'bar', b'boo', b'baz']), 2) input = [(False, chunks)] result = self.process(input) self.assert_equal(result, [{b'path': b'foo'}, {b'path': b'bar'}, {b'path': b'boo'}, {b'path': b'baz'}]) def test_missing_chunk(self): chunks = self.split(self.make_chunks([b'foo', b'bar', b'boo', b'baz']), 4) input = [(False, chunks[:3]), (True, chunks[4:])] result = self.process(input) self.assert_equal(result, [{b'path': b'foo'}, {b'path': b'boo'}, {b'path': b'baz'}]) def test_corrupt_chunk(self): chunks = self.split(self.make_chunks([b'foo', b'bar', b'boo', b'baz']), 4) input = [(False, chunks[:3]), (True, [b'gar', b'bage'] + chunks[3:])] result = self.process(input) self.assert_equal(result, [{b'path': b'foo'}, {b'path': b'boo'}, {b'path': b'baz'}]) @pytest.fixture def item_keys_serialized(): return [msgpack.packb(name) for name in ITEM_KEYS] @pytest.mark.parametrize('packed', [b'', b'x', b'foobar', ] + [msgpack.packb(o) for o in ( [None, 0, 0.0, False, '', {}, [], ()] + [42, 23.42, True, b'foobar', {b'foo': b'bar'}, [b'foo', b'bar'], (b'foo', b'bar')] )]) def test_invalid_msgpacked_item(packed, item_keys_serialized): assert not valid_msgpacked_dict(packed, item_keys_serialized) # pytest-xdist requires always same order for the keys and dicts: IK = sorted(list(ITEM_KEYS)) @pytest.mark.parametrize('packed', [msgpack.packb(o) for o in [ {b'path': b'/a/b/c'}, # small (different msgpack mapping type!) OrderedDict((k, b'') for k in IK), # as big (key count) as it gets OrderedDict((k, b'x' * 1000) for k in IK), # as big (key count and volume) as it gets ]]) def test_valid_msgpacked_items(packed, item_keys_serialized): assert valid_msgpacked_dict(packed, item_keys_serialized) def test_key_length_msgpacked_items(): key = b'x' * 32 # 31 bytes is the limit for fixstr msgpack type data = {key: b''} item_keys_serialized = [msgpack.packb(key), ] assert valid_msgpacked_dict(msgpack.packb(data), item_keys_serialized) def test_backup_io(): with pytest.raises(BackupOSError): with backup_io(): raise OSError(123) def test_backup_io_iter(): class Iterator: def __init__(self, exc): self.exc = exc def __next__(self): raise self.exc() oserror_iterator = Iterator(OSError) with pytest.raises(BackupOSError): for _ in backup_io_iter(oserror_iterator): pass normal_iterator = Iterator(StopIteration) for _ in backup_io_iter(normal_iterator): assert False, 'StopIteration handled incorrectly' borgbackup-1.0.13/borg/testsuite/archiver.py0000644000175000017500000027164313431513133021042 0ustar useruser00000000000000from binascii import unhexlify, b2a_base64 from configparser import ConfigParser import errno import os from datetime import datetime from datetime import timedelta from io import StringIO import random import stat import subprocess import sys import shutil import tempfile import time import unittest from unittest.mock import patch from hashlib import sha256 import msgpack import pytest from .. import xattr, helpers from ..archive import Archive, ChunkBuffer, CHUNK_MAX_EXP, flags_noatime, flags_normal from ..archiver import Archiver from ..cache import Cache from ..crypto import bytes_to_long, num_aes_blocks from ..helpers import Manifest, PatternMatcher, parse_pattern, EXIT_SUCCESS, EXIT_WARNING, EXIT_ERROR, bin_to_hex, \ get_security_dir, MAX_S, MandatoryFeatureUnsupported, Location, ISO_FORMAT from ..key import RepoKey, KeyfileKey, Passphrase, TAMRequiredError from ..keymanager import RepoIdMismatch, NotABorgKeyFile from ..remote import RemoteRepository, PathNotAllowed from ..repository import Repository from . import BaseTestCase, changedir, environment_variable, no_selinux from .platform import fakeroot_detected from . import key try: import llfuse has_llfuse = True or llfuse # avoids "unused import" except ImportError: has_llfuse = False has_lchflags = hasattr(os, 'lchflags') src_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..')) def exec_cmd(*args, archiver=None, fork=False, exe=None, input=b'', **kw): if fork: try: if exe is None: borg = (sys.executable, '-m', 'borg.archiver') elif isinstance(exe, str): borg = (exe, ) elif not isinstance(exe, tuple): raise ValueError('exe must be None, a tuple or a str') output = subprocess.check_output(borg + args, stderr=subprocess.STDOUT, input=input) ret = 0 except subprocess.CalledProcessError as e: output = e.output ret = e.returncode return ret, os.fsdecode(output) else: stdin, stdout, stderr = sys.stdin, sys.stdout, sys.stderr try: sys.stdin = StringIO(input.decode()) sys.stdout = sys.stderr = output = StringIO() if archiver is None: archiver = Archiver() archiver.exit_code = EXIT_SUCCESS helpers.exit_code = EXIT_SUCCESS args = archiver.parse_args(list(args)) ret = archiver.run(args) return ret, output.getvalue() finally: sys.stdin, sys.stdout, sys.stderr = stdin, stdout, stderr # check if the binary "borg.exe" is available (for local testing a symlink to virtualenv/bin/borg should do) try: exec_cmd('help', exe='borg.exe', fork=True) BORG_EXES = ['python', 'binary', ] except FileNotFoundError: BORG_EXES = ['python', ] @pytest.fixture(params=BORG_EXES) def cmd(request): if request.param == 'python': exe = None elif request.param == 'binary': exe = 'borg.exe' else: raise ValueError("param must be 'python' or 'binary'") def exec_fn(*args, **kw): return exec_cmd(*args, exe=exe, fork=True, **kw) return exec_fn def test_return_codes(cmd, tmpdir): repo = tmpdir.mkdir('repo') input = tmpdir.mkdir('input') output = tmpdir.mkdir('output') input.join('test_file').write('content') rc, out = cmd('init', '--encryption=none', '%s' % str(repo)) assert rc == EXIT_SUCCESS rc, out = cmd('create', '%s::archive' % repo, str(input)) assert rc == EXIT_SUCCESS with changedir(str(output)): rc, out = cmd('extract', '%s::archive' % repo) assert rc == EXIT_SUCCESS rc, out = cmd('extract', '%s::archive' % repo, 'does/not/match') assert rc == EXIT_WARNING # pattern did not match rc, out = cmd('create', '%s::archive' % repo, str(input)) assert rc == EXIT_ERROR # duplicate archive name """ test_disk_full is very slow and not recommended to be included in daily testing. for this test, an empty, writable 16MB filesystem mounted on DF_MOUNT is required. for speed and other reasons, it is recommended that the underlying block device is in RAM, not a magnetic or flash disk. assuming /tmp is a tmpfs (in memory filesystem), one can use this: dd if=/dev/zero of=/tmp/borg-disk bs=16M count=1 mkfs.ext4 /tmp/borg-disk mkdir /tmp/borg-mount sudo mount /tmp/borg-disk /tmp/borg-mount if the directory does not exist, the test will be skipped. """ DF_MOUNT = '/tmp/borg-mount' @pytest.mark.skipif(not os.path.exists(DF_MOUNT), reason="needs a 16MB fs mounted on %s" % DF_MOUNT) def test_disk_full(cmd): def make_files(dir, count, size, rnd=True): shutil.rmtree(dir, ignore_errors=True) os.mkdir(dir) if rnd: count = random.randint(1, count) if size > 1: size = random.randint(1, size) for i in range(count): fn = os.path.join(dir, "file%03d" % i) with open(fn, 'wb') as f: data = os.urandom(size) f.write(data) with environment_variable(BORG_CHECK_I_KNOW_WHAT_I_AM_DOING='YES'): mount = DF_MOUNT assert os.path.exists(mount) repo = os.path.join(mount, 'repo') input = os.path.join(mount, 'input') reserve = os.path.join(mount, 'reserve') for j in range(100): shutil.rmtree(repo, ignore_errors=True) shutil.rmtree(input, ignore_errors=True) # keep some space and some inodes in reserve that we can free up later: make_files(reserve, 80, 100000, rnd=False) rc, out = cmd('init', repo) if rc != EXIT_SUCCESS: print('init', rc, out) assert rc == EXIT_SUCCESS try: success, i = True, 0 while success: i += 1 try: make_files(input, 20, 200000) except OSError as err: if err.errno == errno.ENOSPC: # already out of space break raise try: rc, out = cmd('create', '%s::test%03d' % (repo, i), input) success = rc == EXIT_SUCCESS if not success: print('create', rc, out) finally: # make sure repo is not locked shutil.rmtree(os.path.join(repo, 'lock.exclusive'), ignore_errors=True) os.remove(os.path.join(repo, 'lock.roster')) finally: # now some error happened, likely we are out of disk space. # free some space so we can expect borg to be able to work normally: shutil.rmtree(reserve, ignore_errors=True) rc, out = cmd('list', repo) if rc != EXIT_SUCCESS: print('list', rc, out) rc, out = cmd('check', '--repair', repo) if rc != EXIT_SUCCESS: print('check', rc, out) assert rc == EXIT_SUCCESS class ArchiverTestCaseBase(BaseTestCase): EXE = None # python source based FORK_DEFAULT = False prefix = '' def setUp(self): os.environ['BORG_CHECK_I_KNOW_WHAT_I_AM_DOING'] = 'YES' os.environ['BORG_DELETE_I_KNOW_WHAT_I_AM_DOING'] = 'YES' os.environ['BORG_PASSPHRASE'] = 'waytooeasyonlyfortests' self.archiver = not self.FORK_DEFAULT and Archiver() or None self.tmpdir = tempfile.mkdtemp() self.repository_path = os.path.join(self.tmpdir, 'repository') self.repository_location = self.prefix + self.repository_path self.input_path = os.path.join(self.tmpdir, 'input') self.output_path = os.path.join(self.tmpdir, 'output') self.keys_path = os.path.join(self.tmpdir, 'keys') self.cache_path = os.path.join(self.tmpdir, 'cache') self.exclude_file_path = os.path.join(self.tmpdir, 'excludes') os.environ['BORG_KEYS_DIR'] = self.keys_path os.environ['BORG_CACHE_DIR'] = self.cache_path os.mkdir(self.input_path) os.chmod(self.input_path, 0o777) # avoid troubles with fakeroot / FUSE os.mkdir(self.output_path) os.mkdir(self.keys_path) os.mkdir(self.cache_path) with open(self.exclude_file_path, 'wb') as fd: fd.write(b'input/file2\n# A comment line, then a blank line\n\n') self._old_wd = os.getcwd() os.chdir(self.tmpdir) def tearDown(self): os.chdir(self._old_wd) # note: ignore_errors=True as workaround for issue #862 shutil.rmtree(self.tmpdir, ignore_errors=True) def cmd(self, *args, **kw): exit_code = kw.pop('exit_code', 0) fork = kw.pop('fork', None) if fork is None: fork = self.FORK_DEFAULT ret, output = exec_cmd(*args, fork=fork, exe=self.EXE, archiver=self.archiver, **kw) if ret != exit_code: print(output) self.assert_equal(ret, exit_code) return output def create_src_archive(self, name): self.cmd('create', '--compression=lz4', self.repository_location + '::' + name, src_dir) def open_archive(self, name): repository = Repository(self.repository_path, exclusive=True) with repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) archive = Archive(repository, key, manifest, name) return archive, repository class ArchiverTestCase(ArchiverTestCaseBase): def create_regular_file(self, name, size=0, contents=None): filename = os.path.join(self.input_path, name) if not os.path.exists(os.path.dirname(filename)): os.makedirs(os.path.dirname(filename)) with open(filename, 'wb') as fd: if contents is None: contents = b'X' * size fd.write(contents) def create_test_files(self): """Create a minimal test case including all supported file types """ # File self.create_regular_file('empty', size=0) os.utime('input/empty', (MAX_S, MAX_S)) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('flagfile', size=1024) # Directory self.create_regular_file('dir2/file2', size=1024 * 80) # File mode os.chmod('input/file1', 0o4755) # Hard link os.link(os.path.join(self.input_path, 'file1'), os.path.join(self.input_path, 'hardlink')) # Symlink os.symlink('somewhere', os.path.join(self.input_path, 'link1')) self.create_regular_file('fusexattr', size=1) if not xattr.XATTR_FAKEROOT and xattr.is_enabled(self.input_path): # ironically, due to the way how fakeroot works, comparing fuse file xattrs to orig file xattrs # will FAIL if fakeroot supports xattrs, thus we only set the xattr if XATTR_FAKEROOT is False. # This is because fakeroot with xattr-support does not propagate xattrs of the underlying file # into "fakeroot space". Because the xattrs exposed by borgfs are these of an underlying file # (from fakeroots point of view) they are invisible to the test process inside the fakeroot. xattr.setxattr(os.path.join(self.input_path, 'fusexattr'), 'user.foo', b'bar') xattr.setxattr(os.path.join(self.input_path, 'fusexattr'), 'user.empty', b'') # XXX this always fails for me # ubuntu 14.04, on a TMP dir filesystem with user_xattr, using fakeroot # same for newer ubuntu and centos. # if this is supported just on specific platform, platform should be checked first, # so that the test setup for all tests using it does not fail here always for others. # xattr.setxattr(os.path.join(self.input_path, 'link1'), 'user.foo_symlink', b'bar_symlink', follow_symlinks=False) # FIFO node os.mkfifo(os.path.join(self.input_path, 'fifo1')) if has_lchflags: os.lchflags(os.path.join(self.input_path, 'flagfile'), stat.UF_NODUMP) try: # Block device os.mknod('input/bdev', 0o600 | stat.S_IFBLK, os.makedev(10, 20)) # Char device os.mknod('input/cdev', 0o600 | stat.S_IFCHR, os.makedev(30, 40)) # File mode os.chmod('input/dir2', 0o555) # if we take away write perms, we need root to remove contents # File owner os.chown('input/file1', 100, 200) # raises OSError invalid argument on cygwin have_root = True # we have (fake)root except PermissionError: have_root = False except OSError as e: # Note: ENOSYS "Function not implemented" happens as non-root on Win 10 Linux Subsystem. if e.errno not in (errno.EINVAL, errno.ENOSYS): raise have_root = False return have_root def test_basic_functionality(self): have_root = self.create_test_files() self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') self.cmd('create', '--stats', self.repository_location + '::test.2', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') list_output = self.cmd('list', '--short', self.repository_location) self.assert_in('test', list_output) self.assert_in('test.2', list_output) expected = [ 'input', 'input/bdev', 'input/cdev', 'input/dir2', 'input/dir2/file2', 'input/empty', 'input/fifo1', 'input/file1', 'input/flagfile', 'input/hardlink', 'input/link1', ] if not have_root: # we could not create these device files without (fake)root expected.remove('input/bdev') expected.remove('input/cdev') if has_lchflags: # remove the file we did not backup, so input and output become equal expected.remove('input/flagfile') # this file is UF_NODUMP os.remove(os.path.join('input', 'flagfile')) list_output = self.cmd('list', '--short', self.repository_location + '::test') for name in expected: self.assert_in(name, list_output) self.assert_dirs_equal('input', 'output/input') info_output = self.cmd('info', self.repository_location + '::test') item_count = 4 if has_lchflags else 5 # one file is UF_NODUMP self.assert_in('Number of files: %d' % item_count, info_output) shutil.rmtree(self.cache_path) with environment_variable(BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK='yes'): info_output2 = self.cmd('info', self.repository_location + '::test') def filter(output): # filter for interesting "info" output, ignore cache rebuilding related stuff prefixes = ['Name:', 'Fingerprint:', 'Number of files:', 'This archive:', 'All archives:', 'Chunk index:', ] result = [] for line in output.splitlines(): for prefix in prefixes: if line.startswith(prefix): result.append(line) return '\n'.join(result) # the interesting parts of info_output2 and info_output should be same self.assert_equal(filter(info_output), filter(info_output2)) def test_atime(self): def has_noatime(some_file): atime_before = os.stat(some_file).st_atime_ns try: os.close(os.open(some_file, flags_noatime)) except PermissionError: return False else: atime_after = os.stat(some_file).st_atime_ns noatime_used = flags_noatime != flags_normal return noatime_used and atime_before == atime_after self.create_test_files() atime, mtime = 123456780, 234567890 have_noatime = has_noatime('input/file1') os.utime('input/file1', (atime, mtime)) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') sti = os.stat('input/file1') sto = os.stat('output/input/file1') assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9 if have_noatime: assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9 else: # it touched the input file's atime while backing it up assert sto.st_atime_ns == atime * 1e9 def _extract_repository_id(self, path): with Repository(self.repository_path) as repository: return repository.id def _set_repository_id(self, path, id): config = ConfigParser(interpolation=None) config.read(os.path.join(path, 'config')) config.set('repository', 'id', bin_to_hex(id)) with open(os.path.join(path, 'config'), 'w') as fd: config.write(fd) with Repository(self.repository_path) as repository: return repository.id def test_sparse_file(self): def is_sparse(fn, total_size, hole_size): st = os.stat(fn) assert st.st_size == total_size sparse = True if sparse and hasattr(st, 'st_blocks') and st.st_blocks * 512 >= st.st_size: sparse = False if sparse and hasattr(os, 'SEEK_HOLE') and hasattr(os, 'SEEK_DATA'): with open(fn, 'rb') as fd: # only check if the first hole is as expected, because the 2nd hole check # is problematic on xfs due to its "dynamic speculative EOF preallocation try: if fd.seek(0, os.SEEK_HOLE) != 0: sparse = False if fd.seek(0, os.SEEK_DATA) != hole_size: sparse = False except OSError: # OS/FS does not really support SEEK_HOLE/SEEK_DATA sparse = False return sparse filename = os.path.join(self.input_path, 'sparse') content = b'foobar' hole_size = 5 * (1 << CHUNK_MAX_EXP) # 5 full chunker buffers total_size = hole_size + len(content) + hole_size with open(filename, 'wb') as fd: # create a file that has a hole at the beginning and end (if the # OS and filesystem supports sparse files) fd.seek(hole_size, 1) fd.write(content) fd.seek(hole_size, 1) pos = fd.tell() fd.truncate(pos) # we first check if we could create a sparse input file: sparse_support = is_sparse(filename, total_size, hole_size) if sparse_support: # we could create a sparse input file, so creating a backup of it and # extracting it again (as sparse) should also work: self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') with changedir(self.output_path): self.cmd('extract', '--sparse', self.repository_location + '::test') self.assert_dirs_equal('input', 'output/input') filename = os.path.join(self.output_path, 'input', 'sparse') with open(filename, 'rb') as fd: # check if file contents are as expected self.assert_equal(fd.read(hole_size), b'\0' * hole_size) self.assert_equal(fd.read(len(content)), content) self.assert_equal(fd.read(hole_size), b'\0' * hole_size) self.assert_true(is_sparse(filename, total_size, hole_size)) def test_unusual_filenames(self): filenames = ['normal', 'with some blanks', '(with_parens)', ] for filename in filenames: filename = os.path.join(self.input_path, filename) with open(filename, 'wb'): pass self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') for filename in filenames: with changedir('output'): self.cmd('extract', self.repository_location + '::test', os.path.join('input', filename)) assert os.path.exists(os.path.join('output', 'input', filename)) def test_repository_swap_detection(self): self.create_test_files() os.environ['BORG_PASSPHRASE'] = 'passphrase' self.cmd('init', '--encryption=repokey', self.repository_location) repository_id = self._extract_repository_id(self.repository_path) self.cmd('create', self.repository_location + '::test', 'input') shutil.rmtree(self.repository_path) self.cmd('init', '--encryption=none', self.repository_location) self._set_repository_id(self.repository_path, repository_id) self.assert_equal(repository_id, self._extract_repository_id(self.repository_path)) if self.FORK_DEFAULT: self.cmd('create', self.repository_location + '::test.2', 'input', exit_code=EXIT_ERROR) else: self.assert_raises(Cache.EncryptionMethodMismatch, lambda: self.cmd('create', self.repository_location + '::test.2', 'input')) def test_repository_swap_detection2(self): self.create_test_files() self.cmd('init', '--encryption=none', self.repository_location + '_unencrypted') os.environ['BORG_PASSPHRASE'] = 'passphrase' self.cmd('init', '--encryption=repokey', self.repository_location + '_encrypted') self.cmd('create', self.repository_location + '_encrypted::test', 'input') shutil.rmtree(self.repository_path + '_encrypted') os.rename(self.repository_path + '_unencrypted', self.repository_path + '_encrypted') if self.FORK_DEFAULT: self.cmd('create', self.repository_location + '_encrypted::test.2', 'input', exit_code=EXIT_ERROR) else: self.assert_raises(Cache.RepositoryAccessAborted, lambda: self.cmd('create', self.repository_location + '_encrypted::test.2', 'input')) def test_strip_components(self): self.cmd('init', self.repository_location) self.create_regular_file('dir/file') self.cmd('create', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test', '--strip-components', '3') self.assert_true(not os.path.exists('file')) with self.assert_creates_file('file'): self.cmd('extract', self.repository_location + '::test', '--strip-components', '2') with self.assert_creates_file('dir/file'): self.cmd('extract', self.repository_location + '::test', '--strip-components', '1') with self.assert_creates_file('input/dir/file'): self.cmd('extract', self.repository_location + '::test', '--strip-components', '0') def test_extract_include_exclude(self): self.cmd('init', self.repository_location) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('file2', size=1024 * 80) self.create_regular_file('file3', size=1024 * 80) self.create_regular_file('file4', size=1024 * 80) self.cmd('create', '--exclude=input/file4', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test', 'input/file1', ) self.assert_equal(sorted(os.listdir('output/input')), ['file1']) with changedir('output'): self.cmd('extract', '--exclude=input/file2', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'file3']) with changedir('output'): self.cmd('extract', '--exclude-from=' + self.exclude_file_path, self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'file3']) def test_extract_include_exclude_regex(self): self.cmd('init', self.repository_location) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('file2', size=1024 * 80) self.create_regular_file('file3', size=1024 * 80) self.create_regular_file('file4', size=1024 * 80) self.create_regular_file('file333', size=1024 * 80) # Create with regular expression exclusion for file4 self.cmd('create', '--exclude=re:input/file4$', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'file2', 'file3', 'file333']) shutil.rmtree('output/input') # Extract with regular expression exclusion with changedir('output'): self.cmd('extract', '--exclude=re:file3+', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'file2']) shutil.rmtree('output/input') # Combine --exclude with fnmatch and regular expression with changedir('output'): self.cmd('extract', '--exclude=input/file2', '--exclude=re:file[01]', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file3', 'file333']) shutil.rmtree('output/input') # Combine --exclude-from and regular expression exclusion with changedir('output'): self.cmd('extract', '--exclude-from=' + self.exclude_file_path, '--exclude=re:file1', '--exclude=re:file(\\d)\\1\\1$', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file3']) def test_extract_include_exclude_regex_from_file(self): self.cmd('init', self.repository_location) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('file2', size=1024 * 80) self.create_regular_file('file3', size=1024 * 80) self.create_regular_file('file4', size=1024 * 80) self.create_regular_file('file333', size=1024 * 80) self.create_regular_file('aa:something', size=1024 * 80) # Create while excluding using mixed pattern styles with open(self.exclude_file_path, 'wb') as fd: fd.write(b're:input/file4$\n') fd.write(b'fm:*aa:*thing\n') self.cmd('create', '--exclude-from=' + self.exclude_file_path, self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'file2', 'file3', 'file333']) shutil.rmtree('output/input') # Exclude using regular expression with open(self.exclude_file_path, 'wb') as fd: fd.write(b're:file3+\n') with changedir('output'): self.cmd('extract', '--exclude-from=' + self.exclude_file_path, self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'file2']) shutil.rmtree('output/input') # Mixed exclude pattern styles with open(self.exclude_file_path, 'wb') as fd: fd.write(b're:file(\\d)\\1\\1$\n') fd.write(b'fm:nothingwillmatchthis\n') fd.write(b'*/file1\n') fd.write(b're:file2$\n') with changedir('output'): self.cmd('extract', '--exclude-from=' + self.exclude_file_path, self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file3']) def test_extract_with_pattern(self): self.cmd("init", self.repository_location) self.create_regular_file("file1", size=1024 * 80) self.create_regular_file("file2", size=1024 * 80) self.create_regular_file("file3", size=1024 * 80) self.create_regular_file("file4", size=1024 * 80) self.create_regular_file("file333", size=1024 * 80) self.cmd("create", self.repository_location + "::test", "input") # Extract everything with regular expression with changedir("output"): self.cmd("extract", self.repository_location + "::test", "re:.*") self.assert_equal(sorted(os.listdir("output/input")), ["file1", "file2", "file3", "file333", "file4"]) shutil.rmtree("output/input") # Extract with pattern while also excluding files with changedir("output"): self.cmd("extract", "--exclude=re:file[34]$", self.repository_location + "::test", r"re:file\d$") self.assert_equal(sorted(os.listdir("output/input")), ["file1", "file2"]) shutil.rmtree("output/input") # Combine --exclude with pattern for extraction with changedir("output"): self.cmd("extract", "--exclude=input/file1", self.repository_location + "::test", "re:file[12]$") self.assert_equal(sorted(os.listdir("output/input")), ["file2"]) shutil.rmtree("output/input") # Multiple pattern with changedir("output"): self.cmd("extract", self.repository_location + "::test", "fm:input/file1", "fm:*file33*", "input/file2") self.assert_equal(sorted(os.listdir("output/input")), ["file1", "file2", "file333"]) def test_exclude_caches(self): self.cmd('init', self.repository_location) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('cache1/CACHEDIR.TAG', contents=b'Signature: 8a477f597d28d172789f06886806bc55 extra stuff') self.create_regular_file('cache2/CACHEDIR.TAG', contents=b'invalid signature') self.cmd('create', '--exclude-caches', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['cache2', 'file1']) self.assert_equal(sorted(os.listdir('output/input/cache2')), ['CACHEDIR.TAG']) def test_exclude_tagged(self): self.cmd('init', self.repository_location) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('tagged1/.NOBACKUP') self.create_regular_file('tagged2/00-NOBACKUP') self.create_regular_file('tagged3/.NOBACKUP/file2') self.cmd('create', '--exclude-if-present', '.NOBACKUP', '--exclude-if-present', '00-NOBACKUP', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file1', 'tagged3']) def test_exclude_keep_tagged(self): self.cmd('init', self.repository_location) self.create_regular_file('file0', size=1024) self.create_regular_file('tagged1/.NOBACKUP1') self.create_regular_file('tagged1/file1', size=1024) self.create_regular_file('tagged2/.NOBACKUP2') self.create_regular_file('tagged2/file2', size=1024) self.create_regular_file('tagged3/CACHEDIR.TAG', contents=b'Signature: 8a477f597d28d172789f06886806bc55 extra stuff') self.create_regular_file('tagged3/file3', size=1024) self.create_regular_file('taggedall/.NOBACKUP1') self.create_regular_file('taggedall/.NOBACKUP2') self.create_regular_file('taggedall/CACHEDIR.TAG', contents=b'Signature: 8a477f597d28d172789f06886806bc55 extra stuff') self.create_regular_file('taggedall/file4', size=1024) self.cmd('create', '--exclude-if-present', '.NOBACKUP1', '--exclude-if-present', '.NOBACKUP2', '--exclude-caches', '--keep-tag-files', self.repository_location + '::test', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test') self.assert_equal(sorted(os.listdir('output/input')), ['file0', 'tagged1', 'tagged2', 'tagged3', 'taggedall']) self.assert_equal(os.listdir('output/input/tagged1'), ['.NOBACKUP1']) self.assert_equal(os.listdir('output/input/tagged2'), ['.NOBACKUP2']) self.assert_equal(os.listdir('output/input/tagged3'), ['CACHEDIR.TAG']) self.assert_equal(sorted(os.listdir('output/input/taggedall')), ['.NOBACKUP1', '.NOBACKUP2', 'CACHEDIR.TAG', ]) @pytest.mark.skipif(not xattr.XATTR_FAKEROOT, reason='Linux capabilities test, requires fakeroot >= 1.20.2') def test_extract_capabilities(self): fchown = os.fchown # We need to manually patch chown to get the behaviour Linux has, since fakeroot does not # accurately model the interaction of chown(2) and Linux capabilities, i.e. it does not remove them. def patched_fchown(fd, uid, gid): xattr.setxattr(fd, 'security.capability', None, follow_symlinks=False) fchown(fd, uid, gid) # The capability descriptor used here is valid and taken from a /usr/bin/ping capabilities = b'\x01\x00\x00\x02\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' self.create_regular_file('file') xattr.setxattr('input/file', 'security.capability', capabilities) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') with changedir('output'): with patch.object(os, 'fchown', patched_fchown): self.cmd('extract', self.repository_location + '::test') assert xattr.getxattr('input/file', 'security.capability') == capabilities @pytest.mark.skipif(not xattr.XATTR_FAKEROOT, reason='xattr not supported on this system or on this version of' 'fakeroot') def test_extract_xattrs_errors(self): def patched_setxattr_E2BIG(*args, **kwargs): raise OSError(errno.E2BIG, 'E2BIG') def patched_setxattr_ENOTSUP(*args, **kwargs): raise OSError(errno.ENOTSUP, 'ENOTSUP') def patched_setxattr_EACCES(*args, **kwargs): raise OSError(errno.EACCES, 'EACCES') self.create_regular_file('file') xattr.setxattr('input/file', 'attribute', 'value') self.cmd('init', self.repository_location, '-e' 'none') self.cmd('create', self.repository_location + '::test', 'input') with changedir('output'): input_abspath = os.path.abspath('input/file') with patch.object(xattr, 'setxattr', patched_setxattr_E2BIG): out = self.cmd('extract', self.repository_location + '::test', exit_code=EXIT_WARNING) assert out == (input_abspath + ': Value or key of extended attribute attribute is too big for this ' 'filesystem\n') os.remove(input_abspath) with patch.object(xattr, 'setxattr', patched_setxattr_ENOTSUP): out = self.cmd('extract', self.repository_location + '::test', exit_code=EXIT_WARNING) assert out == (input_abspath + ': Extended attributes are not supported on this filesystem\n') os.remove(input_abspath) with patch.object(xattr, 'setxattr', patched_setxattr_EACCES): out = self.cmd('extract', self.repository_location + '::test', exit_code=EXIT_WARNING) assert out == (input_abspath + ': Permission denied when setting extended attribute attribute\n') assert os.path.isfile(input_abspath) def test_path_normalization(self): self.cmd('init', self.repository_location) self.create_regular_file('dir1/dir2/file', size=1024 * 80) with changedir('input/dir1/dir2'): self.cmd('create', self.repository_location + '::test', '../../../input/dir1/../dir1/dir2/..') output = self.cmd('list', self.repository_location + '::test') self.assert_not_in('..', output) self.assert_in(' input/dir1/dir2/file', output) def test_exclude_normalization(self): self.cmd('init', self.repository_location) self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('file2', size=1024 * 80) with changedir('input'): self.cmd('create', '--exclude=file1', self.repository_location + '::test1', '.') with changedir('output'): self.cmd('extract', self.repository_location + '::test1') self.assert_equal(sorted(os.listdir('output')), ['file2']) with changedir('input'): self.cmd('create', '--exclude=./file1', self.repository_location + '::test2', '.') with changedir('output'): self.cmd('extract', self.repository_location + '::test2') self.assert_equal(sorted(os.listdir('output')), ['file2']) self.cmd('create', '--exclude=input/./file1', self.repository_location + '::test3', 'input') with changedir('output'): self.cmd('extract', self.repository_location + '::test3') self.assert_equal(sorted(os.listdir('output/input')), ['file2']) def test_repeated_files(self): self.create_regular_file('file1', size=1024 * 80) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input', 'input') def test_overwrite(self): self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('dir2/file2', size=1024 * 80) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') # Overwriting regular files and directories should be supported os.mkdir('output/input') os.mkdir('output/input/file1') os.mkdir('output/input/dir2') with changedir('output'): self.cmd('extract', self.repository_location + '::test') self.assert_dirs_equal('input', 'output/input') # But non-empty dirs should fail os.unlink('output/input/file1') os.mkdir('output/input/file1') os.mkdir('output/input/file1/dir') with changedir('output'): self.cmd('extract', self.repository_location + '::test', exit_code=1) def test_rename(self): self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('dir2/file2', size=1024 * 80) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') self.cmd('create', self.repository_location + '::test.2', 'input') self.cmd('extract', '--dry-run', self.repository_location + '::test') self.cmd('extract', '--dry-run', self.repository_location + '::test.2') self.cmd('rename', self.repository_location + '::test', 'test.3') self.cmd('extract', '--dry-run', self.repository_location + '::test.2') self.cmd('rename', self.repository_location + '::test.2', 'test.4') self.cmd('extract', '--dry-run', self.repository_location + '::test.3') self.cmd('extract', '--dry-run', self.repository_location + '::test.4') # Make sure both archives have been renamed with Repository(self.repository_path) as repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) self.assert_equal(len(manifest.archives), 2) self.assert_in('test.3', manifest.archives) self.assert_in('test.4', manifest.archives) def test_delete(self): self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('dir2/file2', size=1024 * 80) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') self.cmd('create', self.repository_location + '::test.2', 'input') self.cmd('extract', '--dry-run', self.repository_location + '::test') self.cmd('extract', '--dry-run', self.repository_location + '::test.2') self.cmd('delete', self.repository_location + '::test') self.cmd('extract', '--dry-run', self.repository_location + '::test.2') self.cmd('delete', '--stats', self.repository_location + '::test.2') # Make sure all data except the manifest has been deleted with Repository(self.repository_path) as repository: self.assert_equal(len(repository), 1) def test_delete_repo(self): self.create_regular_file('file1', size=1024 * 80) self.create_regular_file('dir2/file2', size=1024 * 80) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') self.cmd('create', self.repository_location + '::test.2', 'input') os.environ['BORG_DELETE_I_KNOW_WHAT_I_AM_DOING'] = 'no' self.cmd('delete', self.repository_location, exit_code=2) assert os.path.exists(self.repository_path) os.environ['BORG_DELETE_I_KNOW_WHAT_I_AM_DOING'] = 'YES' self.cmd('delete', self.repository_location) # Make sure the repo is gone self.assertFalse(os.path.exists(self.repository_path)) def test_delete_force(self): self.cmd('init', '--encryption=none', self.repository_location) self.create_src_archive('test') with Repository(self.repository_path, exclusive=True) as repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) archive = Archive(repository, key, manifest, 'test') for item in archive.iter_items(): if 'chunks' in item: first_chunk_id = item.chunks[0].id repository.delete(first_chunk_id) repository.commit() break self.cmd('delete', '--force', self.repository_location + '::test') self.cmd('check', '--repair', self.repository_location) output = self.cmd('list', self.repository_location) self.assert_not_in('test', output) def test_delete_double_force(self): self.cmd('init', '--encryption=none', self.repository_location) self.create_src_archive('test') with Repository(self.repository_path, exclusive=True) as repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) archive = Archive(repository, key, manifest, 'test') id = archive.metadata[b'items'][0] repository.put(id, b'corrupted items metadata stream chunk') repository.commit() self.cmd('delete', '--force', '--force', self.repository_location + '::test') self.cmd('check', '--repair', self.repository_location) output = self.cmd('list', self.repository_location) self.assert_not_in('test', output) def test_corrupted_repository(self): self.cmd('init', self.repository_location) self.create_src_archive('test') self.cmd('extract', '--dry-run', self.repository_location + '::test') self.cmd('check', self.repository_location) name = sorted(os.listdir(os.path.join(self.tmpdir, 'repository', 'data', '0')), reverse=True)[0] with open(os.path.join(self.tmpdir, 'repository', 'data', '0', name), 'r+b') as fd: fd.seek(100) fd.write(b'XXXX') self.cmd('check', self.repository_location, exit_code=1) # we currently need to be able to create a lock directory inside the repo: @pytest.mark.xfail(reason="we need to be able to create the lock directory inside the repo") def test_readonly_repository(self): self.cmd('init', self.repository_location) self.create_src_archive('test') os.system('chmod -R ugo-w ' + self.repository_path) try: self.cmd('extract', '--dry-run', self.repository_location + '::test') finally: # Restore permissions so shutil.rmtree is able to delete it os.system('chmod -R u+w ' + self.repository_path) def test_umask(self): self.create_regular_file('file1', size=1024 * 80) self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') mode = os.stat(self.repository_path).st_mode self.assertEqual(stat.S_IMODE(mode), 0o700) def test_create_dry_run(self): self.cmd('init', self.repository_location) self.cmd('create', '--dry-run', self.repository_location + '::test', 'input') # Make sure no archive has been created with Repository(self.repository_path) as repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) self.assert_equal(len(manifest.archives), 0) def add_unknown_feature(self, operation): with Repository(self.repository_path, exclusive=True) as repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) manifest.config[b'feature_flags'] = {operation.value.encode(): {b'mandatory': [b'unknown-feature']}} manifest.write() repository.commit() def cmd_raises_unknown_feature(self, args): if self.FORK_DEFAULT: self.cmd(*args, exit_code=EXIT_ERROR) else: with pytest.raises(MandatoryFeatureUnsupported) as excinfo: self.cmd(*args) assert excinfo.value.args == (['unknown-feature'],) def test_unknown_feature_on_create(self): print(self.cmd('init', self.repository_location)) self.add_unknown_feature(Manifest.Operation.WRITE) self.cmd_raises_unknown_feature(['create', self.repository_location + '::test', 'input']) def test_unknown_feature_on_cache_sync(self): self.cmd('init', '--encryption=repokey', self.repository_location) self.cmd('delete', '--cache-only', self.repository_location) self.add_unknown_feature(Manifest.Operation.READ) self.cmd_raises_unknown_feature(['create', self.repository_location + '::test', 'input']) def test_unknown_feature_on_change_passphrase(self): print(self.cmd('init', self.repository_location)) self.add_unknown_feature(Manifest.Operation.CHECK) self.cmd_raises_unknown_feature(['change-passphrase', self.repository_location]) def test_unknown_feature_on_read(self): print(self.cmd('init', self.repository_location)) self.cmd('create', self.repository_location + '::test', 'input') self.add_unknown_feature(Manifest.Operation.READ) with changedir('output'): self.cmd_raises_unknown_feature(['extract', self.repository_location + '::test']) self.cmd_raises_unknown_feature(['list', self.repository_location]) self.cmd_raises_unknown_feature(['info', self.repository_location + '::test']) def test_unknown_feature_on_rename(self): print(self.cmd('init', self.repository_location)) self.cmd('create', self.repository_location + '::test', 'input') self.add_unknown_feature(Manifest.Operation.CHECK) self.cmd_raises_unknown_feature(['rename', self.repository_location + '::test', 'other']) def test_unknown_feature_on_delete(self): print(self.cmd('init', self.repository_location)) self.cmd('create', self.repository_location + '::test', 'input') self.add_unknown_feature(Manifest.Operation.DELETE) # delete of an archive raises self.cmd_raises_unknown_feature(['delete', self.repository_location + '::test']) self.cmd_raises_unknown_feature(['prune', '--keep-daily=3', self.repository_location]) # delete of the whole repository ignores features self.cmd('delete', self.repository_location) @unittest.skipUnless(has_llfuse, 'llfuse not installed') def test_unknown_feature_on_mount(self): self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') self.add_unknown_feature(Manifest.Operation.READ) mountpoint = os.path.join(self.tmpdir, 'mountpoint') os.mkdir(mountpoint) # XXX this might hang if it doesn't raise an error self.cmd_raises_unknown_feature(['mount', self.repository_location + '::test', mountpoint]) @pytest.mark.allow_cache_wipe def test_unknown_mandatory_feature_in_cache(self): if self.prefix: path_prefix = 'ssh://__testsuite__' else: path_prefix = '' print(self.cmd('init', self.repository_location)) with Repository(self.repository_path, exclusive=True) as repository: if path_prefix: repository._location = Location(self.repository_location) manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) with Cache(repository, key, manifest) as cache: cache.begin_txn() cache.mandatory_features = set(['unknown-feature']) cache.commit() if self.FORK_DEFAULT: self.cmd('create', self.repository_location + '::test', 'input') else: called = False wipe_cache_safe = Cache.wipe_cache def wipe_wrapper(*args): nonlocal called called = True wipe_cache_safe(*args) with patch.object(Cache, 'wipe_cache', wipe_wrapper): self.cmd('create', self.repository_location + '::test', 'input') assert called with Repository(self.repository_path, exclusive=True) as repository: if path_prefix: repository._location = Location(self.repository_location) manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) with Cache(repository, key, manifest) as cache: assert cache.mandatory_features == set([]) def test_progress(self): self.create_regular_file('file1', size=1024 * 80) self.cmd('init', self.repository_location) # progress forced on output = self.cmd('create', '--progress', self.repository_location + '::test4', 'input') self.assert_in("\r", output) # progress forced off output = self.cmd('create', self.repository_location + '::test5', 'input') self.assert_not_in("\r", output) def test_file_status(self): """test that various file status show expected results clearly incomplete: only tests for the weird "unchanged" status for now""" now = time.time() self.create_regular_file('file1', size=1024 * 80) os.utime('input/file1', (now - 5, now - 5)) # 5 seconds ago self.create_regular_file('file2', size=1024 * 80) self.cmd('init', self.repository_location) output = self.cmd('create', '-v', '--list', self.repository_location + '::test', 'input') self.assert_in("A input/file1", output) self.assert_in("A input/file2", output) # should find first file as unmodified output = self.cmd('create', '-v', '--list', self.repository_location + '::test1', 'input') self.assert_in("U input/file1", output) # this is expected, although surprising, for why, see: # https://borgbackup.readthedocs.org/en/latest/faq.html#i-am-seeing-a-added-status-for-a-unchanged-file self.assert_in("A input/file2", output) def test_create_topical(self): now = time.time() self.create_regular_file('file1', size=1024 * 80) os.utime('input/file1', (now-5, now-5)) self.create_regular_file('file2', size=1024 * 80) self.cmd('init', self.repository_location) # no listing by default output = self.cmd('create', self.repository_location + '::test', 'input') self.assert_not_in('file1', output) # shouldn't be listed even if unchanged output = self.cmd('create', self.repository_location + '::test0', 'input') self.assert_not_in('file1', output) # should list the file as unchanged output = self.cmd('create', '-v', '--list', '--filter=U', self.repository_location + '::test1', 'input') self.assert_in('file1', output) # should *not* list the file as changed output = self.cmd('create', '-v', '--filter=AM', self.repository_location + '::test2', 'input') self.assert_not_in('file1', output) # change the file self.create_regular_file('file1', size=1024 * 100) # should list the file as changed output = self.cmd('create', '-v', '--list', '--filter=AM', self.repository_location + '::test3', 'input') self.assert_in('file1', output) def test_create_read_special_broken_symlink(self): os.symlink('somewhere doesnt exist', os.path.join(self.input_path, 'link')) self.cmd('init', self.repository_location) archive = self.repository_location + '::test' self.cmd('create', '--read-special', archive, 'input') output = self.cmd('list', archive) assert 'input/link -> somewhere doesnt exist' in output # def test_cmdline_compatibility(self): # self.create_regular_file('file1', size=1024 * 80) # self.cmd('init', self.repository_location) # self.cmd('create', self.repository_location + '::test', 'input') # output = self.cmd('foo', self.repository_location, '--old') # self.assert_in('"--old" has been deprecated. Use "--new" instead', output) def test_prune_repository(self): self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test1', src_dir) self.cmd('create', self.repository_location + '::test2', src_dir) # these are not really a checkpoints, but they look like some: self.cmd('create', self.repository_location + '::test3.checkpoint', src_dir) self.cmd('create', self.repository_location + '::test3.checkpoint.1', src_dir) output = self.cmd('prune', '-v', '--list', '--dry-run', self.repository_location, '--keep-daily=2') self.assert_in('Would prune: test1', output) # must keep the latest non-checkpoint archive: self.assert_in('Keeping archive: test2', output) output = self.cmd('list', self.repository_location) self.assert_in('test1', output) self.assert_in('test2', output) self.assert_in('test3.checkpoint', output) self.assert_in('test3.checkpoint.1', output) self.cmd('prune', self.repository_location, '--keep-daily=2') output = self.cmd('list', self.repository_location) self.assert_not_in('test1', output) # the latest non-checkpoint archive must be still there: self.assert_in('test2', output) def test_prune_repository_save_space(self): self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test1', src_dir) self.cmd('create', self.repository_location + '::test2', src_dir) output = self.cmd('prune', '-v', '--list', '--dry-run', self.repository_location, '--keep-daily=2') self.assert_in('Keeping archive: test2', output) self.assert_in('Would prune: test1', output) output = self.cmd('list', self.repository_location) self.assert_in('test1', output) self.assert_in('test2', output) self.cmd('prune', '--save-space', self.repository_location, '--keep-daily=2') output = self.cmd('list', self.repository_location) self.assert_not_in('test1', output) self.assert_in('test2', output) def test_prune_repository_prefix(self): self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::foo-2015-08-12-10:00', src_dir) self.cmd('create', self.repository_location + '::foo-2015-08-12-20:00', src_dir) self.cmd('create', self.repository_location + '::bar-2015-08-12-10:00', src_dir) self.cmd('create', self.repository_location + '::bar-2015-08-12-20:00', src_dir) output = self.cmd('prune', '-v', '--list', '--dry-run', self.repository_location, '--keep-daily=2', '--prefix=foo-') self.assert_in('Keeping archive: foo-2015-08-12-20:00', output) self.assert_in('Would prune: foo-2015-08-12-10:00', output) output = self.cmd('list', self.repository_location) self.assert_in('foo-2015-08-12-10:00', output) self.assert_in('foo-2015-08-12-20:00', output) self.assert_in('bar-2015-08-12-10:00', output) self.assert_in('bar-2015-08-12-20:00', output) self.cmd('prune', self.repository_location, '--keep-daily=2', '--prefix=foo-') output = self.cmd('list', self.repository_location) self.assert_not_in('foo-2015-08-12-10:00', output) self.assert_in('foo-2015-08-12-20:00', output) self.assert_in('bar-2015-08-12-10:00', output) self.assert_in('bar-2015-08-12-20:00', output) def test_list_prefix(self): self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test-1', src_dir) self.cmd('create', self.repository_location + '::something-else-than-test-1', src_dir) self.cmd('create', self.repository_location + '::test-2', src_dir) output = self.cmd('list', '--prefix=test-', self.repository_location) self.assert_in('test-1', output) self.assert_in('test-2', output) self.assert_not_in('something-else', output) def test_list_list_format(self): self.cmd('init', self.repository_location) test_archive = self.repository_location + '::test' self.cmd('create', test_archive, src_dir) output_1 = self.cmd('list', test_archive) output_2 = self.cmd('list', '--list-format', '{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NEWLINE}', test_archive) output_3 = self.cmd('list', '--list-format', '{mtime:%s} {path}{NEWLINE}', test_archive) self.assertEqual(output_1, output_2) self.assertNotEqual(output_1, output_3) def test_break_lock(self): self.cmd('init', self.repository_location) self.cmd('break-lock', self.repository_location) def test_usage(self): if self.FORK_DEFAULT: self.cmd(exit_code=0) self.cmd('-h', exit_code=0) else: self.assert_raises(SystemExit, lambda: self.cmd()) self.assert_raises(SystemExit, lambda: self.cmd('-h')) def test_help(self): assert 'Borg' in self.cmd('help') assert 'patterns' in self.cmd('help', 'patterns') assert 'Initialize' in self.cmd('help', 'init') assert 'positional arguments' not in self.cmd('help', 'init', '--epilog-only') assert 'This command initializes' not in self.cmd('help', 'init', '--usage-only') @unittest.skipUnless(has_llfuse, 'llfuse not installed') def test_fuse(self): def has_noatime(some_file): atime_before = os.stat(some_file).st_atime_ns try: os.close(os.open(some_file, flags_noatime)) except PermissionError: return False else: atime_after = os.stat(some_file).st_atime_ns noatime_used = flags_noatime != flags_normal return noatime_used and atime_before == atime_after self.cmd('init', self.repository_location) self.create_test_files() have_noatime = has_noatime('input/file1') self.cmd('create', self.repository_location + '::archive', 'input') self.cmd('create', self.repository_location + '::archive2', 'input') if has_lchflags: # remove the file we did not backup, so input and mount become equal os.remove(os.path.join('input', 'flagfile')) mountpoint = os.path.join(self.tmpdir, 'mountpoint') # mount the whole repository, archive contents shall show up in archivename subdirs of mountpoint: with self.fuse_mount(self.repository_location, mountpoint): # bsdflags are not supported by the FUSE mount # we also ignore xattrs here, they are tested separately self.assert_dirs_equal(self.input_path, os.path.join(mountpoint, 'archive', 'input'), ignore_bsdflags=True, ignore_xattrs=True) self.assert_dirs_equal(self.input_path, os.path.join(mountpoint, 'archive2', 'input'), ignore_bsdflags=True, ignore_xattrs=True) # mount only 1 archive, its contents shall show up directly in mountpoint: with self.fuse_mount(self.repository_location + '::archive', mountpoint): self.assert_dirs_equal(self.input_path, os.path.join(mountpoint, 'input'), ignore_bsdflags=True, ignore_xattrs=True) # regular file in_fn = 'input/file1' out_fn = os.path.join(mountpoint, 'input', 'file1') # stat sti1 = os.stat(in_fn) sto1 = os.stat(out_fn) assert sti1.st_mode == sto1.st_mode assert sti1.st_uid == sto1.st_uid assert sti1.st_gid == sto1.st_gid assert sti1.st_size == sto1.st_size if have_noatime: assert sti1.st_atime == sto1.st_atime assert sti1.st_ctime == sto1.st_ctime assert sti1.st_mtime == sto1.st_mtime # note: there is another hardlink to this, see below assert sti1.st_nlink == sto1.st_nlink == 2 # read with open(in_fn, 'rb') as in_f, open(out_fn, 'rb') as out_f: assert in_f.read() == out_f.read() # hardlink (to 'input/file1') in_fn = 'input/hardlink' out_fn = os.path.join(mountpoint, 'input', 'hardlink') sti2 = os.stat(in_fn) sto2 = os.stat(out_fn) assert sti2.st_nlink == sto2.st_nlink == 2 assert sto1.st_ino == sto2.st_ino # symlink in_fn = 'input/link1' out_fn = os.path.join(mountpoint, 'input', 'link1') sti = os.stat(in_fn, follow_symlinks=False) sto = os.stat(out_fn, follow_symlinks=False) assert stat.S_ISLNK(sti.st_mode) assert stat.S_ISLNK(sto.st_mode) assert os.readlink(in_fn) == os.readlink(out_fn) # FIFO out_fn = os.path.join(mountpoint, 'input', 'fifo1') sto = os.stat(out_fn) assert stat.S_ISFIFO(sto.st_mode) # list/read xattrs try: in_fn = 'input/fusexattr' out_fn = os.path.join(mountpoint, 'input', 'fusexattr') if not xattr.XATTR_FAKEROOT and xattr.is_enabled(self.input_path): assert sorted(no_selinux(xattr.listxattr(out_fn))) == ['user.empty', 'user.foo', ] assert xattr.getxattr(out_fn, 'user.foo') == b'bar' # Special case: getxattr returns None (not b'') when reading an empty xattr. assert xattr.getxattr(out_fn, 'user.empty') is None else: assert xattr.listxattr(out_fn) == [] try: xattr.getxattr(out_fn, 'user.foo') except OSError as e: assert e.errno == llfuse.ENOATTR else: assert False, "expected OSError(ENOATTR), but no error was raised" except OSError as err: if sys.platform.startswith(('freebsd', )) and err.errno == errno.ENOTSUP: # some systems have no xattr support on FUSE pass else: raise @unittest.skipUnless(has_llfuse, 'llfuse not installed') def test_fuse_allow_damaged_files(self): self.cmd('init', self.repository_location) self.create_src_archive('archive') # Get rid of a chunk and repair it archive, repository = self.open_archive('archive') with repository: for item in archive.iter_items(): if item[b'path'].endswith('testsuite/archiver.py'): repository.delete(item[b'chunks'][-1][0]) path = item[b'path'] # store full path for later break else: assert False # missed the file repository.commit() self.cmd('check', '--repair', self.repository_location, exit_code=0) mountpoint = os.path.join(self.tmpdir, 'mountpoint') with self.fuse_mount(self.repository_location + '::archive', mountpoint): with pytest.raises(OSError) as excinfo: open(os.path.join(mountpoint, path)) assert excinfo.value.errno == errno.EIO with self.fuse_mount(self.repository_location + '::archive', mountpoint, '-o', 'allow_damaged_files'): open(os.path.join(mountpoint, path)).close() def verify_aes_counter_uniqueness(self, method): seen = set() # Chunks already seen used = set() # counter values already used def verify_uniqueness(): with Repository(self.repository_path) as repository: for id, _ in repository.open_index(repository.get_transaction_id()).iteritems(): data = repository.get(id) hash = sha256(data).digest() if hash not in seen: seen.add(hash) num_blocks = num_aes_blocks(len(data) - 41) nonce = bytes_to_long(data[33:41]) for counter in range(nonce, nonce + num_blocks): self.assert_not_in(counter, used) used.add(counter) self.create_test_files() os.environ['BORG_PASSPHRASE'] = 'passphrase' self.cmd('init', '--encryption=' + method, self.repository_location) verify_uniqueness() self.cmd('create', self.repository_location + '::test', 'input') verify_uniqueness() self.cmd('create', self.repository_location + '::test.2', 'input') verify_uniqueness() self.cmd('delete', self.repository_location + '::test.2') verify_uniqueness() self.assert_equal(used, set(range(len(used)))) def test_aes_counter_uniqueness_keyfile(self): self.verify_aes_counter_uniqueness('keyfile') def test_aes_counter_uniqueness_passphrase(self): self.verify_aes_counter_uniqueness('repokey') def test_debug_dump_archive_items(self): self.create_test_files() self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') with changedir('output'): output = self.cmd('debug-dump-archive-items', self.repository_location + '::test') output_dir = sorted(os.listdir('output')) assert len(output_dir) > 0 and output_dir[0].startswith('000000_') assert 'Done.' in output def test_debug_dump_repo_objs(self): self.create_test_files() self.cmd('init', self.repository_location) self.cmd('create', self.repository_location + '::test', 'input') with changedir('output'): output = self.cmd('debug-dump-repo-objs', self.repository_location) output_dir = sorted(os.listdir('output')) assert len(output_dir) > 0 and output_dir[0].startswith('000000_') assert 'Done.' in output def test_debug_put_get_delete_obj(self): self.cmd('init', self.repository_location) data = b'some data' hexkey = sha256(data).hexdigest() self.create_regular_file('file', contents=data) output = self.cmd('debug-put-obj', self.repository_location, 'input/file') assert hexkey in output output = self.cmd('debug-get-obj', self.repository_location, hexkey, 'output/file') assert hexkey in output with open('output/file', 'rb') as f: data_read = f.read() assert data == data_read output = self.cmd('debug-delete-obj', self.repository_location, hexkey) assert "deleted" in output output = self.cmd('debug-delete-obj', self.repository_location, hexkey) assert "not found" in output output = self.cmd('debug-delete-obj', self.repository_location, 'invalid') assert "is invalid" in output def test_key_export_keyfile(self): export_file = self.output_path + '/exported' self.cmd('init', self.repository_location, '--encryption', 'keyfile') repo_id = self._extract_repository_id(self.repository_path) self.cmd('key', 'export', self.repository_location, export_file) with open(export_file, 'r') as fd: export_contents = fd.read() assert export_contents.startswith('BORG_KEY ' + bin_to_hex(repo_id) + '\n') key_file = self.keys_path + '/' + os.listdir(self.keys_path)[0] with open(key_file, 'r') as fd: key_contents = fd.read() assert key_contents == export_contents os.unlink(key_file) self.cmd('key', 'import', self.repository_location, export_file) with open(key_file, 'r') as fd: key_contents2 = fd.read() assert key_contents2 == key_contents def test_key_export_repokey(self): export_file = self.output_path + '/exported' self.cmd('init', self.repository_location, '--encryption', 'repokey') repo_id = self._extract_repository_id(self.repository_path) self.cmd('key', 'export', self.repository_location, export_file) with open(export_file, 'r') as fd: export_contents = fd.read() assert export_contents.startswith('BORG_KEY ' + bin_to_hex(repo_id) + '\n') with Repository(self.repository_path) as repository: repo_key = RepoKey(repository) repo_key.load(None, Passphrase.env_passphrase()) backup_key = KeyfileKey(key.KeyTestCase.MockRepository()) backup_key.load(export_file, Passphrase.env_passphrase()) assert repo_key.enc_key == backup_key.enc_key with Repository(self.repository_path) as repository: repository.save_key(b'') self.cmd('key', 'import', self.repository_location, export_file) with Repository(self.repository_path) as repository: repo_key2 = RepoKey(repository) repo_key2.load(None, Passphrase.env_passphrase()) assert repo_key2.enc_key == repo_key2.enc_key def test_key_export_qr(self): export_file = self.output_path + '/exported.html' self.cmd('init', self.repository_location, '--encryption', 'repokey') repo_id = self._extract_repository_id(self.repository_path) self.cmd('key', 'export', '--qr-html', self.repository_location, export_file) with open(export_file, 'r', encoding='utf-8') as fd: export_contents = fd.read() assert bin_to_hex(repo_id) in export_contents assert export_contents.startswith('') assert export_contents.endswith('') def test_key_import_errors(self): export_file = self.output_path + '/exported' self.cmd('init', self.repository_location, '--encryption', 'keyfile') self.cmd('key', 'import', self.repository_location, export_file, exit_code=EXIT_ERROR) with open(export_file, 'w') as fd: fd.write('something not a key\n') if self.FORK_DEFAULT: self.cmd('key', 'import', self.repository_location, export_file, exit_code=2) else: self.assert_raises(NotABorgKeyFile, lambda: self.cmd('key', 'import', self.repository_location, export_file)) with open(export_file, 'w') as fd: fd.write('BORG_KEY a0a0a0\n') if self.FORK_DEFAULT: self.cmd('key', 'import', self.repository_location, export_file, exit_code=2) else: self.assert_raises(RepoIdMismatch, lambda: self.cmd('key', 'import', self.repository_location, export_file)) def test_key_export_paperkey(self): repo_id = 'e294423506da4e1ea76e8dcdf1a3919624ae3ae496fddf905610c351d3f09239' export_file = self.output_path + '/exported' self.cmd('init', self.repository_location, '--encryption', 'keyfile') self._set_repository_id(self.repository_path, unhexlify(repo_id)) key_file = self.keys_path + '/' + os.listdir(self.keys_path)[0] with open(key_file, 'w') as fd: fd.write(KeyfileKey.FILE_ID + ' ' + repo_id + '\n') fd.write(b2a_base64(b'abcdefghijklmnopqrstu').decode()) self.cmd('key', 'export', '--paper', self.repository_location, export_file) with open(export_file, 'r') as fd: export_contents = fd.read() assert export_contents == """To restore key use borg key import --paper /path/to/repo BORG PAPER KEY v1 id: 2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02 1: 616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d 2: 737475 - 88 """ def test_key_import_paperkey(self): repo_id = 'e294423506da4e1ea76e8dcdf1a3919624ae3ae496fddf905610c351d3f09239' self.cmd('init', self.repository_location, '--encryption', 'keyfile') self._set_repository_id(self.repository_path, unhexlify(repo_id)) key_file = self.keys_path + '/' + os.listdir(self.keys_path)[0] with open(key_file, 'w') as fd: fd.write(KeyfileKey.FILE_ID + ' ' + repo_id + '\n') fd.write(b2a_base64(b'abcdefghijklmnopqrstu').decode()) typed_input = ( b'2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 02\n' # Forgot to type "-" b'2 / e29442 3506da 4e1ea7 25f62a 5a3d41 - 02\n' # Forgot to type second "/" b'2 / e29442 3506da 4e1ea7 / 25f62a 5a3d42 - 02\n' # Typo (..42 not ..41) b'2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02\n' # Correct! Congratulations b'616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d\n' b'\n\n' # Abort [yN] => N b'737475 88\n' # missing "-" b'73747i - 88\n' # typo b'73747 - 88\n' # missing nibble b'73 74 75 - 89\n' # line checksum mismatch b'00a1 - 88\n' # line hash collision - overall hash mismatch, have to start over b'2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02\n' b'616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d\n' b'73 74 75 - 88\n' ) # In case that this has to change, here is a quick way to find a colliding line hash: # # from hashlib import sha256 # hash_fn = lambda x: sha256(b'\x00\x02' + x).hexdigest()[:2] # for i in range(1000): # if hash_fn(i.to_bytes(2, byteorder='big')) == '88': # 88 = line hash # print(i.to_bytes(2, 'big')) # break self.cmd('key', 'import', '--paper', self.repository_location, input=typed_input) # Test abort paths typed_input = b'\ny\n' self.cmd('key', 'import', '--paper', self.repository_location, input=typed_input) typed_input = b'2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02\n\ny\n' self.cmd('key', 'import', '--paper', self.repository_location, input=typed_input) @unittest.skipUnless('binary' in BORG_EXES, 'no borg.exe available') class ArchiverTestCaseBinary(ArchiverTestCase): EXE = 'borg.exe' FORK_DEFAULT = True @unittest.skip('test_basic_functionality seems incompatible with fakeroot and/or the binary.') def test_basic_functionality(self): pass @unittest.skip('test_overwrite seems incompatible with fakeroot and/or the binary.') def test_overwrite(self): pass def test_fuse(self): if fakeroot_detected(): unittest.skip('test_fuse with the binary is not compatible with fakeroot') else: super().test_fuse() class ArchiverCheckTestCase(ArchiverTestCaseBase): def setUp(self): super().setUp() with patch.object(ChunkBuffer, 'BUFFER_SIZE', 10): self.cmd('init', self.repository_location) self.create_src_archive('archive1') self.create_src_archive('archive2') def test_check_usage(self): output = self.cmd('check', '-v', self.repository_location, exit_code=0) self.assert_in('Starting repository check', output) self.assert_in('Starting archive consistency check', output) output = self.cmd('check', '-v', '--repository-only', self.repository_location, exit_code=0) self.assert_in('Starting repository check', output) self.assert_not_in('Starting archive consistency check', output) output = self.cmd('check', '-v', '--archives-only', self.repository_location, exit_code=0) self.assert_not_in('Starting repository check', output) self.assert_in('Starting archive consistency check', output) output = self.cmd('check', '-v', '--archives-only', '--prefix=archive2', self.repository_location, exit_code=0) self.assert_not_in('archive1', output) def test_missing_file_chunk(self): archive, repository = self.open_archive('archive1') with repository: for item in archive.iter_items(): if item[b'path'].endswith('testsuite/archiver.py'): valid_chunks = item[b'chunks'] killed_chunk = valid_chunks[-1] repository.delete(killed_chunk[0]) break else: self.assert_true(False) # should not happen repository.commit() self.cmd('check', self.repository_location, exit_code=1) output = self.cmd('check', '--repair', self.repository_location, exit_code=0) self.assert_in('New missing file chunk detected', output) self.cmd('check', self.repository_location, exit_code=0) # check that the file in the old archives has now a different chunk list without the killed chunk for archive_name in ('archive1', 'archive2'): archive, repository = self.open_archive(archive_name) with repository: for item in archive.iter_items(): if item[b'path'].endswith('testsuite/archiver.py'): self.assert_not_equal(valid_chunks, item[b'chunks']) self.assert_not_in(killed_chunk, item[b'chunks']) break else: self.assert_true(False) # should not happen # do a fresh backup (that will include the killed chunk) with patch.object(ChunkBuffer, 'BUFFER_SIZE', 10): self.create_src_archive('archive3') # check should be able to heal the file now: output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0) self.assert_in('Healed previously missing file chunk', output) self.assert_in('testsuite/archiver.py: Completely healed previously damaged file!', output) # check that the file in the old archives has the correct chunks again for archive_name in ('archive1', 'archive2'): archive, repository = self.open_archive(archive_name) with repository: for item in archive.iter_items(): if item[b'path'].endswith('testsuite/archiver.py'): self.assert_equal(valid_chunks, item[b'chunks']) break else: self.assert_true(False) # should not happen def test_missing_archive_item_chunk(self): archive, repository = self.open_archive('archive1') with repository: repository.delete(archive.metadata[b'items'][-5]) repository.commit() self.cmd('check', self.repository_location, exit_code=1) self.cmd('check', '--repair', self.repository_location, exit_code=0) self.cmd('check', self.repository_location, exit_code=0) def test_missing_archive_metadata(self): archive, repository = self.open_archive('archive1') with repository: repository.delete(archive.id) repository.commit() self.cmd('check', self.repository_location, exit_code=1) self.cmd('check', '--repair', self.repository_location, exit_code=0) self.cmd('check', self.repository_location, exit_code=0) def test_missing_manifest(self): archive, repository = self.open_archive('archive1') with repository: repository.delete(Manifest.MANIFEST_ID) repository.commit() self.cmd('check', self.repository_location, exit_code=1) output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0) self.assert_in('archive1', output) self.assert_in('archive2', output) self.cmd('check', self.repository_location, exit_code=0) def test_corrupted_manifest(self): archive, repository = self.open_archive('archive1') with repository: manifest = repository.get(Manifest.MANIFEST_ID) corrupted_manifest = manifest + b'corrupted!' repository.put(Manifest.MANIFEST_ID, corrupted_manifest) repository.commit() self.cmd('check', self.repository_location, exit_code=1) output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0) self.assert_in('archive1', output) self.assert_in('archive2', output) self.cmd('check', self.repository_location, exit_code=0) def test_manifest_rebuild_corrupted_chunk(self): archive, repository = self.open_archive('archive1') with repository: manifest = repository.get(Manifest.MANIFEST_ID) corrupted_manifest = manifest + b'corrupted!' repository.put(Manifest.MANIFEST_ID, corrupted_manifest) chunk = repository.get(archive.id) corrupted_chunk = chunk + b'corrupted!' repository.put(archive.id, corrupted_chunk) repository.commit() self.cmd('check', self.repository_location, exit_code=1) output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0) self.assert_in('archive2', output) self.cmd('check', self.repository_location, exit_code=0) def test_manifest_rebuild_duplicate_archive(self): archive, repository = self.open_archive('archive1') key = archive.key with repository: manifest = repository.get(Manifest.MANIFEST_ID) corrupted_manifest = manifest + b'corrupted!' repository.put(Manifest.MANIFEST_ID, corrupted_manifest) archive = msgpack.packb({ 'cmdline': [], 'items': [], 'hostname': 'foo', 'username': 'bar', 'name': 'archive1', 'time': '2016-12-15T18:49:51.849711', 'version': 1, }) archive_id = key.id_hash(archive) repository.put(archive_id, key.encrypt(archive)) repository.commit() self.cmd('check', self.repository_location, exit_code=1) self.cmd('check', '--repair', self.repository_location, exit_code=0) output = self.cmd('list', self.repository_location) self.assert_in('archive1', output) self.assert_in('archive1.1', output) self.assert_in('archive2', output) def test_extra_chunks(self): self.cmd('check', self.repository_location, exit_code=0) with Repository(self.repository_location, exclusive=True) as repository: repository.put(b'01234567890123456789012345678901', b'xxxx') repository.commit() self.cmd('check', self.repository_location, exit_code=1) self.cmd('check', self.repository_location, exit_code=1) self.cmd('check', '--repair', self.repository_location, exit_code=0) self.cmd('check', self.repository_location, exit_code=0) self.cmd('extract', '--dry-run', self.repository_location + '::archive1', exit_code=0) def test_empty_repository(self): with Repository(self.repository_location, exclusive=True) as repository: for id_ in repository.list(): repository.delete(id_) repository.commit() self.cmd('check', self.repository_location, exit_code=1) def test_attic013_acl_bug(self): # Attic up to release 0.13 contained a bug where every item unintentionally received # a b'acl'=None key-value pair. # This bug can still live on in Borg repositories (through borg upgrade). archive, repository = self.open_archive('archive1') with repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) with Cache(repository, key, manifest) as cache: archive = Archive(repository, key, manifest, '0.13', cache=cache, create=True) archive.items_buffer.add({ # path and mtime are required. b'path': '1234', b'mtime': 0, # acl is the offending key. b'acl': None }) archive.save() self.cmd('check', self.repository_location, exit_code=0) class ManifestAuthenticationTest(ArchiverTestCaseBase): def spoof_manifest(self, repository): with repository: _, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) repository.put(Manifest.MANIFEST_ID, key.encrypt(msgpack.packb({ 'version': 1, 'archives': {}, 'config': {}, 'timestamp': (datetime.utcnow() + timedelta(days=1)).strftime(ISO_FORMAT), }))) repository.commit() def test_fresh_init_tam_required(self): self.cmd('init', self.repository_location) repository = Repository(self.repository_path, exclusive=True) with repository: manifest, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) repository.put(Manifest.MANIFEST_ID, key.encrypt(msgpack.packb({ 'version': 1, 'archives': {}, 'timestamp': (datetime.utcnow() + timedelta(days=1)).strftime(ISO_FORMAT), }))) repository.commit() with pytest.raises(TAMRequiredError): self.cmd('list', self.repository_location) def test_not_required(self): self.cmd('init', self.repository_location) self.create_src_archive('archive1234') repository = Repository(self.repository_path, exclusive=True) with repository: shutil.rmtree(get_security_dir(bin_to_hex(repository.id))) _, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK) key.tam_required = False key.change_passphrase(key._passphrase) manifest = msgpack.unpackb(key.decrypt(None, repository.get(Manifest.MANIFEST_ID))) del manifest[b'tam'] repository.put(Manifest.MANIFEST_ID, key.encrypt(msgpack.packb(manifest))) repository.commit() output = self.cmd('list', '--debug', self.repository_location) assert 'archive1234' in output assert 'TAM not found and not required' in output # Run upgrade self.cmd('upgrade', '--tam', self.repository_location) # Manifest must be authenticated now output = self.cmd('list', '--debug', self.repository_location) assert 'archive1234' in output assert 'TAM-verified manifest' in output # Try to spoof / modify pre-1.0.9 self.spoof_manifest(repository) # Fails with pytest.raises(TAMRequiredError): self.cmd('list', self.repository_location) # Force upgrade self.cmd('upgrade', '--tam', '--force', self.repository_location) self.cmd('list', self.repository_location) def test_disable(self): self.cmd('init', self.repository_location) self.create_src_archive('archive1234') self.cmd('upgrade', '--disable-tam', self.repository_location) repository = Repository(self.repository_path, exclusive=True) self.spoof_manifest(repository) assert not self.cmd('list', self.repository_location) def test_disable2(self): self.cmd('init', self.repository_location) self.create_src_archive('archive1234') repository = Repository(self.repository_path, exclusive=True) self.spoof_manifest(repository) self.cmd('upgrade', '--disable-tam', self.repository_location) assert not self.cmd('list', self.repository_location) class RemoteArchiverTestCase(ArchiverTestCase): prefix = '__testsuite__:' def test_remote_repo_restrict_to_path(self): # restricted to repo directory itself: with patch.object(RemoteRepository, 'extra_test_args', ['--restrict-to-path', self.repository_path]): self.cmd('init', self.repository_location) # restricted to repo directory itself, fail for other directories with same prefix: with patch.object(RemoteRepository, 'extra_test_args', ['--restrict-to-path', self.repository_path]): self.assert_raises(PathNotAllowed, lambda: self.cmd('init', self.repository_location + '_0')) # restricted to a completely different path: with patch.object(RemoteRepository, 'extra_test_args', ['--restrict-to-path', '/foo']): self.assert_raises(PathNotAllowed, lambda: self.cmd('init', self.repository_location + '_1')) path_prefix = os.path.dirname(self.repository_path) # restrict to repo directory's parent directory: with patch.object(RemoteRepository, 'extra_test_args', ['--restrict-to-path', path_prefix]): self.cmd('init', self.repository_location + '_2') # restrict to repo directory's parent directory and another directory: with patch.object(RemoteRepository, 'extra_test_args', ['--restrict-to-path', '/foo', '--restrict-to-path', path_prefix]): self.cmd('init', self.repository_location + '_3') @unittest.skip('only works locally') def test_debug_put_get_delete_obj(self): pass def test_strip_components_doesnt_leak(self): self.cmd('init', self.repository_location) self.create_regular_file('dir/file', contents=b"test file contents 1") self.create_regular_file('dir/file2', contents=b"test file contents 2") self.create_regular_file('skipped-file1', contents=b"test file contents 3") self.create_regular_file('skipped-file2', contents=b"test file contents 4") self.create_regular_file('skipped-file3', contents=b"test file contents 5") self.cmd('create', self.repository_location + '::test', 'input') marker = 'cached responses left in RemoteRepository' with changedir('output'): res = self.cmd('extract', "--debug", self.repository_location + '::test', '--strip-components', '3') self.assert_true(marker not in res) with self.assert_creates_file('file'): res = self.cmd('extract', "--debug", self.repository_location + '::test', '--strip-components', '2') self.assert_true(marker not in res) with self.assert_creates_file('dir/file'): res = self.cmd('extract', "--debug", self.repository_location + '::test', '--strip-components', '1') self.assert_true(marker not in res) with self.assert_creates_file('input/dir/file'): res = self.cmd('extract', "--debug", self.repository_location + '::test', '--strip-components', '0') self.assert_true(marker not in res) def test_get_args(): archiver = Archiver() # everything normal: # first param is argv as produced by ssh forced command, # second param is like from SSH_ORIGINAL_COMMAND env variable args = archiver.get_args(['borg', 'serve', '--restrict-to-path=/p1', '--restrict-to-path=/p2', ], 'borg serve --info --umask=0027') assert args.func == archiver.do_serve assert args.restrict_to_paths == ['/p1', '/p2'] assert args.umask == 0o027 assert args.log_level == 'info' # trying to cheat - break out of path restriction args = archiver.get_args(['borg', 'serve', '--restrict-to-path=/p1', '--restrict-to-path=/p2', ], 'borg serve --restrict-to-path=/') assert args.restrict_to_paths == ['/p1', '/p2'] # trying to cheat - try to execute different subcommand args = archiver.get_args(['borg', 'serve', '--restrict-to-path=/p1', '--restrict-to-path=/p2', ], 'borg init /') assert args.func == archiver.do_serve # Check that environment variables in the forced command don't cause issues. If the command # were not forced, environment variables would be interpreted by the shell, but this does not # happen for forced commands - we get the verbatim command line and need to deal with env vars. args = archiver.get_args(['borg', 'serve', ], 'BORG_HOSTNAME_IS_UNIQUE=yes borg serve --info') assert args.func == archiver.do_serve class TestBuildFilter: def test_basic(self): matcher = PatternMatcher() matcher.add([parse_pattern('included')], True) filter = Archiver.build_filter(matcher) assert filter({b'path': 'included'}) assert filter({b'path': 'included/file'}) assert not filter({b'path': 'something else'}) def test_empty(self): matcher = PatternMatcher(fallback=True) filter = Archiver.build_filter(matcher) assert filter({b'path': 'anything'}) def test_strip_components(self): matcher = PatternMatcher(fallback=True) filter = Archiver.build_filter(matcher, strip_components=1) assert not filter({b'path': 'shallow'}) assert not filter({b'path': 'shallow/'}) # can this even happen? paths are normalized... assert filter({b'path': 'deep enough/file'}) assert filter({b'path': 'something/dir/file'}) borgbackup-1.0.13/borg/testsuite/attic.tar.gz0000644000175000017500000000540613431513133021110 0ustar useruser00000000000000 6Y S"hNŒ}D]0ɾd_dFeddsReDgġҊն"EVS0c:VhSA;JN~+q3dW9&9Kfν7nGyBQ$+w^yITdQ=T*EajaαF_2 I K`s,8\\i$p}H0ZVRTͣ^I5]e^L #ɊGUT]xکԙj^/9S}L[,*kZu6uzsSkTIu]ᆖgtE1FQj(=NJjQ]܉6_G* \յѺlk1^ipSOkYz:4 FTSBN&@,)!Eq7:ޖ^_IƋ͍q}:mJ;9kP* 5EТhN%5av;׹xuQ E=5Qw?_HiBfdMuT,v@F.j 9_mE@n,B:#\4TWq;¹-^j"Z&!TI>fuTBmmtǚ@` T6/BC7/{k<~xJ ?&'Ne\3Ui@,:2+  F:uUxwYhd@^$f9{l7EzCG8@ݻHEuFu_%/ P7MuC8@;H_n 7}pä/ Pn$n. fP fuҗOKC9C9C9C9C9C9C9C9CO3.Gc\`psH_ ү7-$}x|-7hq q!"}s|ۦ: >ȷ\ o's[Eȷ/ 9C9C9C9C9C9C9C9C9;8q^B7%\H HH \r}=@;}\7-7\o?@s4{m'|'|?C9C9C9C9C9C9C10spl58+d#ϼ"3 EDERu<ŕmN~ .cH*m?-f:"H°s QIA2;}+K*&}gl#_E,T˵ގGN_$ Tդ*y7zߴ$`xDo/ ׍nͥ%tDU$;|$`byD0tﺒqMߞ}$`42$P}/ x|ǰ8@ mAfKөa-xf0n4a-? Vۂo,]w VۂIRj[3|Vۂ/\Td#tTt/[QP=ۨ;ͦ?E@yd-ӏJ/=F@ON@ۯEtz;ͺ-AzXY؂ ]t@wN8Jt7>%mާÛu-XW@_XuTtzsӏ5y/z~g<?a G|*>~*:=h7c3/Ѕtu7ӇNK@kU:Hi֥m}ΝgeHtzۍw-ni*|˴ۂoÛu^-h='~ Ngy7EUު˩5;X.*:}ͷQyػ"*:S[?dӡ/Wh?;zU*@:4w+: O:stx+@8|[%oֹ`S*^]O@W_x:p9 N??Tt8sN^ܗޠ;TtYy *:}_ ՏҝέaNna *?V9`^Y tu;[Ãtx/vE@_㛏Qk_sdſ5borgbackup-1.0.13/borg/testsuite/benchmark.py0000644000175000017500000000535413431513133021163 0ustar useruser00000000000000""" Do benchmarks using pytest-benchmark. Usage: py.test --benchmark-only """ import os import pytest from .archiver import changedir, cmd @pytest.yield_fixture def repo_url(request, tmpdir, monkeypatch): monkeypatch.setenv('BORG_PASSPHRASE', '123456') monkeypatch.setenv('BORG_CHECK_I_KNOW_WHAT_I_AM_DOING', 'YES') monkeypatch.setenv('BORG_DELETE_I_KNOW_WHAT_I_AM_DOING', 'YES') monkeypatch.setenv('BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK', 'yes') monkeypatch.setenv('BORG_KEYS_DIR', str(tmpdir.join('keys'))) monkeypatch.setenv('BORG_CACHE_DIR', str(tmpdir.join('cache'))) yield str(tmpdir.join('repository')) tmpdir.remove(rec=1) @pytest.fixture(params=["none", "repokey"]) def repo(request, cmd, repo_url): cmd('init', '--encryption', request.param, repo_url) return repo_url @pytest.yield_fixture(scope='session', params=["zeros", "random"]) def testdata(request, tmpdir_factory): count, size = 10, 1000*1000 p = tmpdir_factory.mktemp('data') data_type = request.param if data_type == 'zeros': # do not use a binary zero (\0) to avoid sparse detection def data(size): return b'0' * size if data_type == 'random': def data(size): return os.urandom(size) for i in range(count): with open(str(p.join(str(i))), "wb") as f: f.write(data(size)) yield str(p) p.remove(rec=1) @pytest.fixture(params=['none', 'lz4']) def archive(request, cmd, repo, testdata): archive_url = repo + '::test' cmd('create', '--compression', request.param, archive_url, testdata) return archive_url def test_create_none(benchmark, cmd, repo, testdata): result, out = benchmark.pedantic(cmd, ('create', '--compression', 'none', repo + '::test', testdata)) assert result == 0 def test_create_lz4(benchmark, cmd, repo, testdata): result, out = benchmark.pedantic(cmd, ('create', '--compression', 'lz4', repo + '::test', testdata)) assert result == 0 def test_extract(benchmark, cmd, archive, tmpdir): with changedir(str(tmpdir)): result, out = benchmark.pedantic(cmd, ('extract', archive)) assert result == 0 def test_delete(benchmark, cmd, archive): result, out = benchmark.pedantic(cmd, ('delete', archive)) assert result == 0 def test_list(benchmark, cmd, archive): result, out = benchmark(cmd, 'list', archive) assert result == 0 def test_info(benchmark, cmd, archive): result, out = benchmark(cmd, 'info', archive) assert result == 0 def test_check(benchmark, cmd, archive): repo = archive.split('::')[0] result, out = benchmark(cmd, 'check', repo) assert result == 0 def test_help(benchmark, cmd): result, out = benchmark(cmd, 'help') assert result == 0 borgbackup-1.0.13/borg/testsuite/chunker.py0000644000175000017500000000574013431513133020667 0ustar useruser00000000000000from io import BytesIO from ..chunker import Chunker, buzhash, buzhash_update from ..archive import CHUNK_MAX_EXP, CHUNKER_PARAMS from . import BaseTestCase class ChunkerTestCase(BaseTestCase): def test_chunkify(self): data = b'0' * int(1.5 * (1 << CHUNK_MAX_EXP)) + b'Y' parts = [bytes(c) for c in Chunker(0, 1, CHUNK_MAX_EXP, 2, 2).chunkify(BytesIO(data))] self.assert_equal(len(parts), 2) self.assert_equal(b''.join(parts), data) self.assert_equal([bytes(c) for c in Chunker(0, 1, CHUNK_MAX_EXP, 2, 2).chunkify(BytesIO(b''))], []) self.assert_equal([bytes(c) for c in Chunker(0, 1, CHUNK_MAX_EXP, 2, 2).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'fooba', b'rboobaz', b'fooba', b'rboobaz', b'fooba', b'rboobaz']) self.assert_equal([bytes(c) for c in Chunker(1, 1, CHUNK_MAX_EXP, 2, 2).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'fo', b'obarb', b'oob', b'azf', b'oobarb', b'oob', b'azf', b'oobarb', b'oobaz']) self.assert_equal([bytes(c) for c in Chunker(2, 1, CHUNK_MAX_EXP, 2, 2).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foob', b'ar', b'boobazfoob', b'ar', b'boobazfoob', b'ar', b'boobaz']) self.assert_equal([bytes(c) for c in Chunker(0, 2, CHUNK_MAX_EXP, 2, 3).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foobarboobaz' * 3]) self.assert_equal([bytes(c) for c in Chunker(1, 2, CHUNK_MAX_EXP, 2, 3).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foobar', b'boobazfo', b'obar', b'boobazfo', b'obar', b'boobaz']) self.assert_equal([bytes(c) for c in Chunker(2, 2, CHUNK_MAX_EXP, 2, 3).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foob', b'arboobaz', b'foob', b'arboobaz', b'foob', b'arboobaz']) self.assert_equal([bytes(c) for c in Chunker(0, 3, CHUNK_MAX_EXP, 2, 3).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foobarboobaz' * 3]) self.assert_equal([bytes(c) for c in Chunker(1, 3, CHUNK_MAX_EXP, 2, 3).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foobarbo', b'obazfoobar', b'boobazfo', b'obarboobaz']) self.assert_equal([bytes(c) for c in Chunker(2, 3, CHUNK_MAX_EXP, 2, 3).chunkify(BytesIO(b'foobarboobaz' * 3))], [b'foobarboobaz', b'foobarboobaz', b'foobarboobaz']) def test_buzhash(self): self.assert_equal(buzhash(b'abcdefghijklmnop', 0), 3795437769) self.assert_equal(buzhash(b'abcdefghijklmnop', 1), 3795400502) self.assert_equal(buzhash(b'abcdefghijklmnop', 1), buzhash_update(buzhash(b'Xabcdefghijklmno', 1), ord('X'), ord('p'), 16, 1)) # Test with more than 31 bytes to make sure our barrel_shift macro works correctly self.assert_equal(buzhash(b'abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz', 0), 566521248) def test_small_reads(self): class SmallReadFile: input = b'a' * (20 + 1) def read(self, nbytes): self.input = self.input[:-1] return self.input[:1] reconstructed = b''.join(Chunker(0, *CHUNKER_PARAMS).chunkify(SmallReadFile())) assert reconstructed == b'a' * 20 borgbackup-1.0.13/borg/testsuite/compress.py0000644000175000017500000000614513431513133021063 0ustar useruser00000000000000import os import zlib try: import lzma except ImportError: lzma = None import pytest from ..compress import get_compressor, Compressor, CNONE, ZLIB, LZ4 buffer = bytes(2**16) data = b'fooooooooobaaaaaaaar' * 10 params = dict(name='zlib', level=6) def test_get_compressor(): c = get_compressor(name='none') assert isinstance(c, CNONE) c = get_compressor(name='lz4') assert isinstance(c, LZ4) c = get_compressor(name='zlib') assert isinstance(c, ZLIB) with pytest.raises(KeyError): get_compressor(name='foobar') def test_cnull(): c = get_compressor(name='none') cdata = c.compress(data) assert len(cdata) > len(data) assert data in cdata # it's not compressed and just in there 1:1 assert data == c.decompress(cdata) assert data == Compressor(**params).decompress(cdata) # autodetect def test_lz4(): c = get_compressor(name='lz4') cdata = c.compress(data) assert len(cdata) < len(data) assert data == c.decompress(cdata) assert data == Compressor(**params).decompress(cdata) # autodetect def test_lz4_buffer_allocation(): # test with a rather huge data object to see if buffer allocation / resizing works data = os.urandom(50 * 2**20) # 50MiB incompressible data c = get_compressor(name='lz4') cdata = c.compress(data) assert data == c.decompress(cdata) def test_zlib(): c = get_compressor(name='zlib') cdata = c.compress(data) assert len(cdata) < len(data) assert data == c.decompress(cdata) assert data == Compressor(**params).decompress(cdata) # autodetect def test_lzma(): if lzma is None: pytest.skip("No lzma support found.") c = get_compressor(name='lzma') cdata = c.compress(data) assert len(cdata) < len(data) assert data == c.decompress(cdata) assert data == Compressor(**params).decompress(cdata) # autodetect def test_autodetect_invalid(): with pytest.raises(ValueError): Compressor(**params).decompress(b'\xff\xfftotalcrap') with pytest.raises(ValueError): Compressor(**params).decompress(b'\x08\x00notreallyzlib') def test_zlib_compat(): # for compatibility reasons, we do not add an extra header for zlib, # nor do we expect one when decompressing / autodetecting for level in range(10): c = get_compressor(name='zlib', level=level) cdata1 = c.compress(data) cdata2 = zlib.compress(data, level) assert cdata1 == cdata2 data2 = c.decompress(cdata2) assert data == data2 data2 = Compressor(**params).decompress(cdata2) assert data == data2 def test_compressor(): params_list = [ dict(name='none'), dict(name='lz4'), dict(name='zlib', level=0), dict(name='zlib', level=6), dict(name='zlib', level=9), ] if lzma: params_list += [ dict(name='lzma', level=0), dict(name='lzma', level=6), # we do not test lzma on level 9 because of the huge memory needs ] for params in params_list: c = Compressor(**params) assert data == c.decompress(c.compress(data)) borgbackup-1.0.13/borg/testsuite/crypto.py0000644000175000017500000001153213431513133020544 0ustar useruser00000000000000from binascii import hexlify from ..crypto import AES, bytes_to_long, bytes_to_int, long_to_bytes from ..crypto import increment_iv, bytes16_to_int, int_to_bytes16 from ..crypto import hkdf_hmac_sha512 from . import BaseTestCase class CryptoTestCase(BaseTestCase): def test_bytes_to_int(self): self.assert_equal(bytes_to_int(b'\0\0\0\1'), 1) def test_bytes_to_long(self): self.assert_equal(bytes_to_long(b'\0\0\0\0\0\0\0\1'), 1) self.assert_equal(long_to_bytes(1), b'\0\0\0\0\0\0\0\1') def test_bytes16_to_int(self): self.assert_equal(bytes16_to_int(b'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1'), 1) self.assert_equal(int_to_bytes16(1), b'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1') self.assert_equal(bytes16_to_int(b'\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0'), 2 ** 64) self.assert_equal(int_to_bytes16(2 ** 64), b'\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0') def test_increment_iv(self): iv0 = b'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0' iv1 = b'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1' iv2 = b'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\2' self.assert_equal(increment_iv(iv0, 0), iv0) self.assert_equal(increment_iv(iv0, 1), iv1) self.assert_equal(increment_iv(iv0, 2), iv2) iva = b'\0\0\0\0\0\0\0\0\xff\xff\xff\xff\xff\xff\xff\xff' ivb = b'\0\0\0\0\0\0\0\1\x00\x00\x00\x00\x00\x00\x00\x00' ivc = b'\0\0\0\0\0\0\0\1\x00\x00\x00\x00\x00\x00\x00\x01' self.assert_equal(increment_iv(iva, 0), iva) self.assert_equal(increment_iv(iva, 1), ivb) self.assert_equal(increment_iv(iva, 2), ivc) self.assert_equal(increment_iv(iv0, 2**64), ivb) def test_aes(self): key = b'X' * 32 data = b'foo' * 10 # encrypt aes = AES(is_encrypt=True, key=key) self.assert_equal(bytes_to_long(aes.iv, 8), 0) cdata = aes.encrypt(data) self.assert_equal(hexlify(cdata), b'c6efb702de12498f34a2c2bbc8149e759996d08bf6dc5c610aefc0c3a466') self.assert_equal(bytes_to_long(aes.iv, 8), 2) # decrypt aes = AES(is_encrypt=False, key=key) self.assert_equal(bytes_to_long(aes.iv, 8), 0) pdata = aes.decrypt(cdata) self.assert_equal(data, pdata) self.assert_equal(bytes_to_long(aes.iv, 8), 2) # These test vectors come from https://www.kullo.net/blog/hkdf-sha-512-test-vectors/ # who claims to have verified these against independent Python and C++ implementations. def test_hkdf_hmac_sha512(self): ikm = b'\x0b' * 22 salt = bytes.fromhex('000102030405060708090a0b0c') info = bytes.fromhex('f0f1f2f3f4f5f6f7f8f9') l = 42 okm = hkdf_hmac_sha512(ikm, salt, info, l) assert okm == bytes.fromhex('832390086cda71fb47625bb5ceb168e4c8e26a1a16ed34d9fc7fe92c1481579338da362cb8d9f925d7cb') def test_hkdf_hmac_sha512_2(self): ikm = bytes.fromhex('000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f2021222324252627' '28292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f') salt = bytes.fromhex('606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868' '788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeaf') info = bytes.fromhex('b0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7' 'd8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff') l = 82 okm = hkdf_hmac_sha512(ikm, salt, info, l) assert okm == bytes.fromhex('ce6c97192805b346e6161e821ed165673b84f400a2b514b2fe23d84cd189ddf1b695b48cbd1c838844' '1137b3ce28f16aa64ba33ba466b24df6cfcb021ecff235f6a2056ce3af1de44d572097a8505d9e7a93') def test_hkdf_hmac_sha512_3(self): ikm = bytes.fromhex('0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b') salt = None info = b'' l = 42 okm = hkdf_hmac_sha512(ikm, salt, info, l) assert okm == bytes.fromhex('f5fa02b18298a72a8c23898a8703472c6eb179dc204c03425c970e3b164bf90fff22d04836d0e2343bac') def test_hkdf_hmac_sha512_4(self): ikm = bytes.fromhex('0b0b0b0b0b0b0b0b0b0b0b') salt = bytes.fromhex('000102030405060708090a0b0c') info = bytes.fromhex('f0f1f2f3f4f5f6f7f8f9') l = 42 okm = hkdf_hmac_sha512(ikm, salt, info, l) assert okm == bytes.fromhex('7413e8997e020610fbf6823f2ce14bff01875db1ca55f68cfcf3954dc8aff53559bd5e3028b080f7c068') def test_hkdf_hmac_sha512_5(self): ikm = bytes.fromhex('0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c') salt = None info = b'' l = 42 okm = hkdf_hmac_sha512(ikm, salt, info, l) assert okm == bytes.fromhex('1407d46013d98bc6decefcfee55f0f90b0c7f63d68eb1a80eaf07e953cfc0a3a5240a155d6e4daa965bb') borgbackup-1.0.13/borg/testsuite/hashindex.py0000644000175000017500000002326613431513133021206 0ustar useruser00000000000000import base64 import hashlib import os import struct import tempfile import zlib import pytest from ..hashindex import NSIndex, ChunkIndex from .. import hashindex from . import BaseTestCase def H(x): # make some 32byte long thing that depends on x return bytes('%-0.32d' % x, 'ascii') class HashIndexTestCase(BaseTestCase): def _generic_test(self, cls, make_value, sha): idx = cls() self.assert_equal(len(idx), 0) # Test set for x in range(100): idx[H(x)] = make_value(x) self.assert_equal(len(idx), 100) for x in range(100): self.assert_equal(idx[H(x)], make_value(x)) # Test update for x in range(100): idx[H(x)] = make_value(x * 2) self.assert_equal(len(idx), 100) for x in range(100): self.assert_equal(idx[H(x)], make_value(x * 2)) # Test delete for x in range(50): del idx[H(x)] self.assert_equal(len(idx), 50) idx_name = tempfile.NamedTemporaryFile() idx.write(idx_name.name) del idx # Verify file contents with open(idx_name.name, 'rb') as fd: self.assert_equal(hashlib.sha256(fd.read()).hexdigest(), sha) # Make sure we can open the file idx = cls.read(idx_name.name) self.assert_equal(len(idx), 50) for x in range(50, 100): self.assert_equal(idx[H(x)], make_value(x * 2)) idx.clear() self.assert_equal(len(idx), 0) idx.write(idx_name.name) del idx self.assert_equal(len(cls.read(idx_name.name)), 0) def test_nsindex(self): self._generic_test(NSIndex, lambda x: (x, x), 'b96ec1ddabb4278cc92261ee171f7efc979dc19397cc5e89b778f05fa25bf93f') def test_chunkindex(self): self._generic_test(ChunkIndex, lambda x: (x, x, x), '9d437a1e145beccc790c69e66ba94fc17bd982d83a401c9c6e524609405529d8') def test_resize(self): n = 2000 # Must be >= MIN_BUCKETS idx_name = tempfile.NamedTemporaryFile() idx = NSIndex() idx.write(idx_name.name) initial_size = os.path.getsize(idx_name.name) self.assert_equal(len(idx), 0) for x in range(n): idx[H(x)] = x, x idx.write(idx_name.name) self.assert_true(initial_size < os.path.getsize(idx_name.name)) for x in range(n): del idx[H(x)] self.assert_equal(len(idx), 0) idx.write(idx_name.name) self.assert_equal(initial_size, os.path.getsize(idx_name.name)) def test_iteritems(self): idx = NSIndex() for x in range(100): idx[H(x)] = x, x iterator = idx.iteritems() all = list(iterator) self.assert_equal(len(all), 100) # iterator is already exhausted by list(): self.assert_raises(StopIteration, next, iterator) second_half = list(idx.iteritems(marker=all[49][0])) self.assert_equal(len(second_half), 50) self.assert_equal(second_half, all[50:]) def test_chunkindex_merge(self): idx1 = ChunkIndex() idx1[H(1)] = 1, 100, 100 idx1[H(2)] = 2, 200, 200 idx1[H(3)] = 3, 300, 300 # no H(4) entry idx2 = ChunkIndex() idx2[H(1)] = 4, 100, 100 idx2[H(2)] = 5, 200, 200 # no H(3) entry idx2[H(4)] = 6, 400, 400 idx1.merge(idx2) assert idx1[H(1)] == (5, 100, 100) assert idx1[H(2)] == (7, 200, 200) assert idx1[H(3)] == (3, 300, 300) assert idx1[H(4)] == (6, 400, 400) def test_chunkindex_summarize(self): idx = ChunkIndex() idx[H(1)] = 1, 1000, 100 idx[H(2)] = 2, 2000, 200 idx[H(3)] = 3, 3000, 300 size, csize, unique_size, unique_csize, unique_chunks, chunks = idx.summarize() assert size == 1000 + 2 * 2000 + 3 * 3000 assert csize == 100 + 2 * 200 + 3 * 300 assert unique_size == 1000 + 2000 + 3000 assert unique_csize == 100 + 200 + 300 assert chunks == 1 + 2 + 3 assert unique_chunks == 3 class HashIndexRefcountingTestCase(BaseTestCase): def test_chunkindex_limit(self): idx = ChunkIndex() idx[H(1)] = ChunkIndex.MAX_VALUE - 1, 1, 2 # 5 is arbitray, any number of incref/decrefs shouldn't move it once it's limited for i in range(5): # first incref to move it to the limit refcount, *_ = idx.incref(H(1)) assert refcount == ChunkIndex.MAX_VALUE for i in range(5): refcount, *_ = idx.decref(H(1)) assert refcount == ChunkIndex.MAX_VALUE def _merge(self, refcounta, refcountb): def merge(refcount1, refcount2): idx1 = ChunkIndex() idx1[H(1)] = refcount1, 1, 2 idx2 = ChunkIndex() idx2[H(1)] = refcount2, 1, 2 idx1.merge(idx2) refcount, *_ = idx1[H(1)] return refcount result = merge(refcounta, refcountb) # check for commutativity assert result == merge(refcountb, refcounta) return result def test_chunkindex_merge_limit1(self): # Check that it does *not* limit at MAX_VALUE - 1 # (MAX_VALUE is odd) half = ChunkIndex.MAX_VALUE // 2 assert self._merge(half, half) == ChunkIndex.MAX_VALUE - 1 def test_chunkindex_merge_limit2(self): # 3000000000 + 2000000000 > MAX_VALUE assert self._merge(3000000000, 2000000000) == ChunkIndex.MAX_VALUE def test_chunkindex_merge_limit3(self): # Crossover point: both addition and limit semantics will yield the same result half = ChunkIndex.MAX_VALUE // 2 assert self._merge(half + 1, half) == ChunkIndex.MAX_VALUE def test_chunkindex_merge_limit4(self): # Beyond crossover, result of addition would be 2**31 half = ChunkIndex.MAX_VALUE // 2 assert self._merge(half + 2, half) == ChunkIndex.MAX_VALUE assert self._merge(half + 1, half + 1) == ChunkIndex.MAX_VALUE def test_chunkindex_add(self): idx1 = ChunkIndex() idx1.add(H(1), 5, 6, 7) assert idx1[H(1)] == (5, 6, 7) idx1.add(H(1), 1, 0, 0) assert idx1[H(1)] == (6, 6, 7) def test_incref_limit(self): idx1 = ChunkIndex() idx1[H(1)] = (ChunkIndex.MAX_VALUE, 6, 7) idx1.incref(H(1)) refcount, *_ = idx1[H(1)] assert refcount == ChunkIndex.MAX_VALUE def test_decref_limit(self): idx1 = ChunkIndex() idx1[H(1)] = ChunkIndex.MAX_VALUE, 6, 7 idx1.decref(H(1)) refcount, *_ = idx1[H(1)] assert refcount == ChunkIndex.MAX_VALUE def test_decref_zero(self): idx1 = ChunkIndex() idx1[H(1)] = 0, 0, 0 with pytest.raises(AssertionError): idx1.decref(H(1)) def test_incref_decref(self): idx1 = ChunkIndex() idx1.add(H(1), 5, 6, 7) assert idx1[H(1)] == (5, 6, 7) idx1.incref(H(1)) assert idx1[H(1)] == (6, 6, 7) idx1.decref(H(1)) assert idx1[H(1)] == (5, 6, 7) def test_setitem_raises(self): idx1 = ChunkIndex() with pytest.raises(AssertionError): idx1[H(1)] = ChunkIndex.MAX_VALUE + 1, 0, 0 def test_keyerror(self): idx = ChunkIndex() with pytest.raises(KeyError): idx.incref(H(1)) with pytest.raises(KeyError): idx.decref(H(1)) with pytest.raises(KeyError): idx[H(1)] with pytest.raises(OverflowError): idx.add(H(1), -1, 0, 0) class HashIndexDataTestCase(BaseTestCase): # This bytestring was created with 1.0-maint at c2f9533 HASHINDEX = b'eJzt0L0NgmAUhtHLT0LDEI6AuAEhMVYmVnSuYefC7AB3Aj9KNedJbnfyFne6P67P27w0EdG1Eac+Cm1ZybAsy7Isy7Isy7Isy7I' \ b'sy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7Isy7LsL9nhc+cqTZ' \ b'3XlO2Ys++Du5fX+l1/YFmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVmWZVn2/+0O2rYccw==' def _serialize_hashindex(self, idx): with tempfile.TemporaryDirectory() as tempdir: file = os.path.join(tempdir, 'idx') idx.write(file) with open(file, 'rb') as f: return self._pack(f.read()) def _deserialize_hashindex(self, bytestring): with tempfile.TemporaryDirectory() as tempdir: file = os.path.join(tempdir, 'idx') with open(file, 'wb') as f: f.write(self._unpack(bytestring)) return ChunkIndex.read(file) def _pack(self, bytestring): return base64.b64encode(zlib.compress(bytestring)) def _unpack(self, bytestring): return zlib.decompress(base64.b64decode(bytestring)) def test_identical_creation(self): idx1 = ChunkIndex() idx1[H(1)] = 1, 2, 3 idx1[H(2)] = 2**31 - 1, 0, 0 idx1[H(3)] = 4294962296, 0, 0 # 4294962296 is -5000 interpreted as an uint32_t assert self._serialize_hashindex(idx1) == self.HASHINDEX def test_read_known_good(self): idx1 = self._deserialize_hashindex(self.HASHINDEX) assert idx1[H(1)] == (1, 2, 3) assert idx1[H(2)] == (2**31 - 1, 0, 0) assert idx1[H(3)] == (4294962296, 0, 0) idx2 = ChunkIndex() idx2[H(3)] = 2**32 - 123456, 6, 7 idx1.merge(idx2) assert idx1[H(3)] == (ChunkIndex.MAX_VALUE, 0, 0) def test_nsindex_segment_limit(): idx = NSIndex() with pytest.raises(AssertionError): idx[H(1)] = NSIndex.MAX_VALUE + 1, 0 assert H(1) not in idx idx[H(2)] = NSIndex.MAX_VALUE, 0 assert H(2) in idx def test_max_load_factor(): assert NSIndex.MAX_LOAD_FACTOR < 1.0 assert ChunkIndex.MAX_LOAD_FACTOR < 1.0 borgbackup-1.0.13/borg/testsuite/helpers.py0000644000175000017500000013763713431513133020705 0ustar useruser00000000000000import hashlib from argparse import ArgumentTypeError from time import mktime, strptime from datetime import datetime, timezone, timedelta from io import StringIO import os import pytest import sys import msgpack import msgpack.fallback import time from ..helpers import Location, format_file_size, format_timedelta, format_line, PlaceholderError, make_path_safe, \ interval, prune_within, prune_split, get_cache_dir, get_keys_dir, get_security_dir, Statistics, is_slow_msgpack, \ yes, TRUISH, FALSISH, DEFAULTISH, \ StableDict, int_to_bigint, bigint_to_int, parse_timestamp, CompressionSpec, ChunkerParams, \ ProgressIndicatorPercent, ProgressIndicatorEndless, load_excludes, parse_pattern, \ PatternMatcher, RegexPattern, PathPrefixPattern, FnmatchPattern, ShellPattern, \ Buffer, safe_ns, safe_s, SUPPORT_32BIT_PLATFORMS from . import BaseTestCase, FakeInputs class BigIntTestCase(BaseTestCase): def test_bigint(self): self.assert_equal(int_to_bigint(0), 0) self.assert_equal(int_to_bigint(2**63-1), 2**63-1) self.assert_equal(int_to_bigint(-2**63+1), -2**63+1) self.assert_equal(int_to_bigint(2**63), b'\x00\x00\x00\x00\x00\x00\x00\x80\x00') self.assert_equal(int_to_bigint(-2**63), b'\x00\x00\x00\x00\x00\x00\x00\x80\xff') self.assert_equal(bigint_to_int(int_to_bigint(-2**70)), -2**70) self.assert_equal(bigint_to_int(int_to_bigint(2**70)), 2**70) class TestLocationWithoutEnv: @pytest.fixture def keys_dir(self, tmpdir, monkeypatch): tmpdir = str(tmpdir) monkeypatch.setenv('BORG_KEYS_DIR', tmpdir) if not tmpdir.endswith(os.path.sep): tmpdir += os.path.sep return tmpdir def test_ssh(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('ssh://user@host:1234/some/path::archive')) == \ "Location(proto='ssh', user='user', host='host', port=1234, path='/some/path', archive='archive')" assert Location('ssh://user@host:1234/some/path::archive').to_key_filename() == keys_dir + 'host__some_path' assert repr(Location('ssh://user@host:1234/some/path')) == \ "Location(proto='ssh', user='user', host='host', port=1234, path='/some/path', archive=None)" assert repr(Location('ssh://user@host/some/path')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive=None)" assert repr(Location('ssh://user@[::]:1234/some/path::archive')) == \ "Location(proto='ssh', user='user', host='::', port=1234, path='/some/path', archive='archive')" assert repr(Location('ssh://user@[::]:1234/some/path')) == \ "Location(proto='ssh', user='user', host='::', port=1234, path='/some/path', archive=None)" assert Location('ssh://user@[::]:1234/some/path').to_key_filename() == keys_dir + '____some_path' assert repr(Location('ssh://user@[::]/some/path')) == \ "Location(proto='ssh', user='user', host='::', port=None, path='/some/path', archive=None)" assert repr(Location('ssh://user@[2001:db8::]:1234/some/path::archive')) == \ "Location(proto='ssh', user='user', host='2001:db8::', port=1234, path='/some/path', archive='archive')" assert repr(Location('ssh://user@[2001:db8::]:1234/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::', port=1234, path='/some/path', archive=None)" assert Location('ssh://user@[2001:db8::]:1234/some/path').to_key_filename() == keys_dir + '2001_db8____some_path' assert repr(Location('ssh://user@[2001:db8::]/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::', port=None, path='/some/path', archive=None)" assert repr(Location('ssh://user@[2001:db8::c0:ffee]:1234/some/path::archive')) == \ "Location(proto='ssh', user='user', host='2001:db8::c0:ffee', port=1234, path='/some/path', archive='archive')" assert repr(Location('ssh://user@[2001:db8::c0:ffee]:1234/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::c0:ffee', port=1234, path='/some/path', archive=None)" assert repr(Location('ssh://user@[2001:db8::c0:ffee]/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::c0:ffee', port=None, path='/some/path', archive=None)" assert repr(Location('ssh://user@[2001:db8::192.0.2.1]:1234/some/path::archive')) == \ "Location(proto='ssh', user='user', host='2001:db8::192.0.2.1', port=1234, path='/some/path', archive='archive')" assert repr(Location('ssh://user@[2001:db8::192.0.2.1]:1234/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::192.0.2.1', port=1234, path='/some/path', archive=None)" assert repr(Location('ssh://user@[2001:db8::192.0.2.1]/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::192.0.2.1', port=None, path='/some/path', archive=None)" assert Location('ssh://user@[2001:db8::192.0.2.1]/some/path').to_key_filename() == keys_dir + '2001_db8__192_0_2_1__some_path' def test_file(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('file:///some/path::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/path', archive='archive')" assert repr(Location('file:///some/path')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/path', archive=None)" assert Location('file:///some/path').to_key_filename() == keys_dir + 'some_path' def test_scp(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('user@host:/some/path::archive')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive='archive')" assert repr(Location('user@host:/some/path')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive=None)" assert repr(Location('user@[::]:/some/path::archive')) == \ "Location(proto='ssh', user='user', host='::', port=None, path='/some/path', archive='archive')" assert repr(Location('user@[::]:/some/path')) == \ "Location(proto='ssh', user='user', host='::', port=None, path='/some/path', archive=None)" assert repr(Location('user@[2001:db8::]:/some/path::archive')) == \ "Location(proto='ssh', user='user', host='2001:db8::', port=None, path='/some/path', archive='archive')" assert repr(Location('user@[2001:db8::]:/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::', port=None, path='/some/path', archive=None)" assert repr(Location('user@[2001:db8::c0:ffee]:/some/path::archive')) == \ "Location(proto='ssh', user='user', host='2001:db8::c0:ffee', port=None, path='/some/path', archive='archive')" assert repr(Location('user@[2001:db8::c0:ffee]:/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::c0:ffee', port=None, path='/some/path', archive=None)" assert repr(Location('user@[2001:db8::192.0.2.1]:/some/path::archive')) == \ "Location(proto='ssh', user='user', host='2001:db8::192.0.2.1', port=None, path='/some/path', archive='archive')" assert repr(Location('user@[2001:db8::192.0.2.1]:/some/path')) == \ "Location(proto='ssh', user='user', host='2001:db8::192.0.2.1', port=None, path='/some/path', archive=None)" assert Location('user@[2001:db8::192.0.2.1]:/some/path').to_key_filename() == keys_dir + '2001_db8__192_0_2_1__some_path' def test_smb(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('file:////server/share/path::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='//server/share/path', archive='archive')" assert Location('file:////server/share/path::archive').to_key_filename() == keys_dir + 'server_share_path' def test_folder(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('path::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='path', archive='archive')" assert repr(Location('path')) == \ "Location(proto='file', user=None, host=None, port=None, path='path', archive=None)" assert Location('path').to_key_filename() == keys_dir + 'path' def test_long_path(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert Location(os.path.join(*(40 * ['path']))).to_key_filename() == keys_dir + '_'.join(20 * ['path']) + '_' def test_abspath(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('/some/absolute/path::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/absolute/path', archive='archive')" assert repr(Location('/some/absolute/path')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/absolute/path', archive=None)" assert Location('/some/absolute/path').to_key_filename() == keys_dir + 'some_absolute_path' assert repr(Location('ssh://user@host/some/path')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive=None)" assert Location('ssh://user@host/some/path').to_key_filename() == keys_dir + 'host__some_path' def test_relpath(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('some/relative/path::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='some/relative/path', archive='archive')" assert repr(Location('some/relative/path')) == \ "Location(proto='file', user=None, host=None, port=None, path='some/relative/path', archive=None)" assert Location('some/relative/path').to_key_filename() == keys_dir + 'some_relative_path' assert repr(Location('ssh://user@host/./some/path')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/./some/path', archive=None)" assert Location('ssh://user@host/./some/path').to_key_filename() == keys_dir + 'host__some_path' assert repr(Location('ssh://user@host/~/some/path')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/~/some/path', archive=None)" assert Location('ssh://user@host/~/some/path').to_key_filename() == keys_dir + 'host__some_path' assert repr(Location('ssh://user@host/~user/some/path')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/~user/some/path', archive=None)" assert Location('ssh://user@host/~user/some/path').to_key_filename() == keys_dir + 'host__user_some_path' def test_with_colons(self, monkeypatch, keys_dir): monkeypatch.delenv('BORG_REPO', raising=False) assert repr(Location('/abs/path:w:cols::arch:col')) == \ "Location(proto='file', user=None, host=None, port=None, path='/abs/path:w:cols', archive='arch:col')" assert repr(Location('/abs/path:with:colons::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='/abs/path:with:colons', archive='archive')" assert repr(Location('/abs/path:with:colons')) == \ "Location(proto='file', user=None, host=None, port=None, path='/abs/path:with:colons', archive=None)" assert Location('/abs/path:with:colons').to_key_filename() == keys_dir + 'abs_path_with_colons' def test_user_parsing(self): # see issue #1930 assert repr(Location('host:path::2016-12-31@23:59:59')) == \ "Location(proto='ssh', user=None, host='host', port=None, path='path', archive='2016-12-31@23:59:59')" assert repr(Location('ssh://host/path::2016-12-31@23:59:59')) == \ "Location(proto='ssh', user=None, host='host', port=None, path='/path', archive='2016-12-31@23:59:59')" def test_underspecified(self, monkeypatch): monkeypatch.delenv('BORG_REPO', raising=False) with pytest.raises(ValueError): Location('::archive') with pytest.raises(ValueError): Location('::') with pytest.raises(ValueError): Location() def test_no_slashes(self, monkeypatch): monkeypatch.delenv('BORG_REPO', raising=False) with pytest.raises(ValueError): Location('/some/path/to/repo::archive_name_with/slashes/is_invalid') def test_canonical_path(self, monkeypatch): monkeypatch.delenv('BORG_REPO', raising=False) locations = ['some/path::archive', 'file://some/path::archive', 'host:some/path::archive', 'host:~user/some/path::archive', 'ssh://host/some/path::archive', 'ssh://user@host:1234/some/path::archive'] for location in locations: assert Location(location).canonical_path() == \ Location(Location(location).canonical_path()).canonical_path(), "failed: %s" % location def test_format_path(self, monkeypatch): monkeypatch.delenv('BORG_REPO', raising=False) test_pid = os.getpid() assert repr(Location('/some/path::archive{pid}')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/path', archive='archive{}')".format(test_pid) location_time1 = Location('/some/path::archive{now:%s}') time.sleep(1.1) location_time2 = Location('/some/path::archive{now:%s}') assert location_time1.archive != location_time2.archive def test_bad_syntax(self): with pytest.raises(ValueError): # this is invalid due to the 2nd colon, correct: 'ssh://user@host/path' Location('ssh://user@host:/path') class TestLocationWithEnv: def test_ssh(self, monkeypatch): monkeypatch.setenv('BORG_REPO', 'ssh://user@host:1234/some/path') assert repr(Location('::archive')) == \ "Location(proto='ssh', user='user', host='host', port=1234, path='/some/path', archive='archive')" assert repr(Location('::')) == \ "Location(proto='ssh', user='user', host='host', port=1234, path='/some/path', archive=None)" assert repr(Location()) == \ "Location(proto='ssh', user='user', host='host', port=1234, path='/some/path', archive=None)" def test_file(self, monkeypatch): monkeypatch.setenv('BORG_REPO', 'file:///some/path') assert repr(Location('::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/path', archive='archive')" assert repr(Location('::')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/path', archive=None)" assert repr(Location()) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/path', archive=None)" def test_scp(self, monkeypatch): monkeypatch.setenv('BORG_REPO', 'user@host:/some/path') assert repr(Location('::archive')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive='archive')" assert repr(Location('::')) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive=None)" assert repr(Location()) == \ "Location(proto='ssh', user='user', host='host', port=None, path='/some/path', archive=None)" def test_folder(self, monkeypatch): monkeypatch.setenv('BORG_REPO', 'path') assert repr(Location('::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='path', archive='archive')" assert repr(Location('::')) == \ "Location(proto='file', user=None, host=None, port=None, path='path', archive=None)" assert repr(Location()) == \ "Location(proto='file', user=None, host=None, port=None, path='path', archive=None)" def test_abspath(self, monkeypatch): monkeypatch.setenv('BORG_REPO', '/some/absolute/path') assert repr(Location('::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/absolute/path', archive='archive')" assert repr(Location('::')) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/absolute/path', archive=None)" assert repr(Location()) == \ "Location(proto='file', user=None, host=None, port=None, path='/some/absolute/path', archive=None)" def test_relpath(self, monkeypatch): monkeypatch.setenv('BORG_REPO', 'some/relative/path') assert repr(Location('::archive')) == \ "Location(proto='file', user=None, host=None, port=None, path='some/relative/path', archive='archive')" assert repr(Location('::')) == \ "Location(proto='file', user=None, host=None, port=None, path='some/relative/path', archive=None)" assert repr(Location()) == \ "Location(proto='file', user=None, host=None, port=None, path='some/relative/path', archive=None)" def test_with_colons(self, monkeypatch): monkeypatch.setenv('BORG_REPO', '/abs/path:w:cols') assert repr(Location('::arch:col')) == \ "Location(proto='file', user=None, host=None, port=None, path='/abs/path:w:cols', archive='arch:col')" assert repr(Location('::')) == \ "Location(proto='file', user=None, host=None, port=None, path='/abs/path:w:cols', archive=None)" assert repr(Location()) == \ "Location(proto='file', user=None, host=None, port=None, path='/abs/path:w:cols', archive=None)" def test_no_slashes(self, monkeypatch): monkeypatch.setenv('BORG_REPO', '/some/absolute/path') with pytest.raises(ValueError): Location('::archive_name_with/slashes/is_invalid') class FormatTimedeltaTestCase(BaseTestCase): def test(self): t0 = datetime(2001, 1, 1, 10, 20, 3, 0) t1 = datetime(2001, 1, 1, 12, 20, 4, 100000) self.assert_equal( format_timedelta(t1 - t0), '2 hours 1.10 seconds' ) def check_patterns(files, pattern, expected): """Utility for testing patterns. """ assert all([f == os.path.normpath(f) for f in files]), "Pattern matchers expect normalized input paths" matched = [f for f in files if pattern.match(f)] assert matched == (files if expected is None else expected) @pytest.mark.parametrize("pattern, expected", [ # "None" means all files, i.e. all match the given pattern ("/", None), ("/./", None), ("", []), ("/home/u", []), ("/home/user", ["/home/user/.profile", "/home/user/.bashrc"]), ("/etc", ["/etc/server/config", "/etc/server/hosts"]), ("///etc//////", ["/etc/server/config", "/etc/server/hosts"]), ("/./home//..//home/user2", ["/home/user2/.profile", "/home/user2/public_html/index.html"]), ("/srv", ["/srv/messages", "/srv/dmesg"]), ]) def test_patterns_prefix(pattern, expected): files = [ "/etc/server/config", "/etc/server/hosts", "/home", "/home/user/.profile", "/home/user/.bashrc", "/home/user2/.profile", "/home/user2/public_html/index.html", "/srv/messages", "/srv/dmesg", ] check_patterns(files, PathPrefixPattern(pattern), expected) @pytest.mark.parametrize("pattern, expected", [ # "None" means all files, i.e. all match the given pattern ("", []), ("foo", []), ("relative", ["relative/path1", "relative/two"]), ("more", ["more/relative"]), ]) def test_patterns_prefix_relative(pattern, expected): files = ["relative/path1", "relative/two", "more/relative"] check_patterns(files, PathPrefixPattern(pattern), expected) @pytest.mark.parametrize("pattern, expected", [ # "None" means all files, i.e. all match the given pattern ("/*", None), ("/./*", None), ("*", None), ("*/*", None), ("*///*", None), ("/home/u", []), ("/home/*", ["/home/user/.profile", "/home/user/.bashrc", "/home/user2/.profile", "/home/user2/public_html/index.html", "/home/foo/.thumbnails", "/home/foo/bar/.thumbnails"]), ("/home/user/*", ["/home/user/.profile", "/home/user/.bashrc"]), ("/etc/*", ["/etc/server/config", "/etc/server/hosts"]), ("*/.pr????e", ["/home/user/.profile", "/home/user2/.profile"]), ("///etc//////*", ["/etc/server/config", "/etc/server/hosts"]), ("/./home//..//home/user2/*", ["/home/user2/.profile", "/home/user2/public_html/index.html"]), ("/srv*", ["/srv/messages", "/srv/dmesg"]), ("/home/*/.thumbnails", ["/home/foo/.thumbnails", "/home/foo/bar/.thumbnails"]), ]) def test_patterns_fnmatch(pattern, expected): files = [ "/etc/server/config", "/etc/server/hosts", "/home", "/home/user/.profile", "/home/user/.bashrc", "/home/user2/.profile", "/home/user2/public_html/index.html", "/srv/messages", "/srv/dmesg", "/home/foo/.thumbnails", "/home/foo/bar/.thumbnails", ] check_patterns(files, FnmatchPattern(pattern), expected) @pytest.mark.parametrize("pattern, expected", [ # "None" means all files, i.e. all match the given pattern ("*", None), ("**/*", None), ("/**/*", None), ("/./*", None), ("*/*", None), ("*///*", None), ("/home/u", []), ("/home/*", ["/home/user/.profile", "/home/user/.bashrc", "/home/user2/.profile", "/home/user2/public_html/index.html", "/home/foo/.thumbnails", "/home/foo/bar/.thumbnails"]), ("/home/user/*", ["/home/user/.profile", "/home/user/.bashrc"]), ("/etc/*/*", ["/etc/server/config", "/etc/server/hosts"]), ("/etc/**/*", ["/etc/server/config", "/etc/server/hosts"]), ("/etc/**/*/*", ["/etc/server/config", "/etc/server/hosts"]), ("*/.pr????e", []), ("**/.pr????e", ["/home/user/.profile", "/home/user2/.profile"]), ("///etc//////*", ["/etc/server/config", "/etc/server/hosts"]), ("/./home//..//home/user2/", ["/home/user2/.profile", "/home/user2/public_html/index.html"]), ("/./home//..//home/user2/**/*", ["/home/user2/.profile", "/home/user2/public_html/index.html"]), ("/srv*/", ["/srv/messages", "/srv/dmesg", "/srv2/blafasel"]), ("/srv*", ["/srv", "/srv/messages", "/srv/dmesg", "/srv2", "/srv2/blafasel"]), ("/srv/*", ["/srv/messages", "/srv/dmesg"]), ("/srv2/**", ["/srv2", "/srv2/blafasel"]), ("/srv2/**/", ["/srv2/blafasel"]), ("/home/*/.thumbnails", ["/home/foo/.thumbnails"]), ("/home/*/*/.thumbnails", ["/home/foo/bar/.thumbnails"]), ]) def test_patterns_shell(pattern, expected): files = [ "/etc/server/config", "/etc/server/hosts", "/home", "/home/user/.profile", "/home/user/.bashrc", "/home/user2/.profile", "/home/user2/public_html/index.html", "/srv", "/srv/messages", "/srv/dmesg", "/srv2", "/srv2/blafasel", "/home/foo/.thumbnails", "/home/foo/bar/.thumbnails", ] check_patterns(files, ShellPattern(pattern), expected) @pytest.mark.parametrize("pattern, expected", [ # "None" means all files, i.e. all match the given pattern ("", None), (".*", None), ("^/", None), ("^abc$", []), ("^[^/]", []), ("^(?!/srv|/foo|/opt)", ["/home", "/home/user/.profile", "/home/user/.bashrc", "/home/user2/.profile", "/home/user2/public_html/index.html", "/home/foo/.thumbnails", "/home/foo/bar/.thumbnails", ]), ]) def test_patterns_regex(pattern, expected): files = [ '/srv/data', '/foo/bar', '/home', '/home/user/.profile', '/home/user/.bashrc', '/home/user2/.profile', '/home/user2/public_html/index.html', '/opt/log/messages.txt', '/opt/log/dmesg.txt', "/home/foo/.thumbnails", "/home/foo/bar/.thumbnails", ] obj = RegexPattern(pattern) assert str(obj) == pattern assert obj.pattern == pattern check_patterns(files, obj, expected) def test_regex_pattern(): # The forward slash must match the platform-specific path separator assert RegexPattern("^/$").match("/") assert RegexPattern("^/$").match(os.path.sep) assert not RegexPattern(r"^\\$").match("/") def use_normalized_unicode(): return sys.platform in ("darwin",) def _make_test_patterns(pattern): return [PathPrefixPattern(pattern), FnmatchPattern(pattern), RegexPattern("^{}/foo$".format(pattern)), ShellPattern(pattern), ] @pytest.mark.parametrize("pattern", _make_test_patterns("b\N{LATIN SMALL LETTER A WITH ACUTE}")) def test_composed_unicode_pattern(pattern): assert pattern.match("b\N{LATIN SMALL LETTER A WITH ACUTE}/foo") assert pattern.match("ba\N{COMBINING ACUTE ACCENT}/foo") == use_normalized_unicode() @pytest.mark.parametrize("pattern", _make_test_patterns("ba\N{COMBINING ACUTE ACCENT}")) def test_decomposed_unicode_pattern(pattern): assert pattern.match("b\N{LATIN SMALL LETTER A WITH ACUTE}/foo") == use_normalized_unicode() assert pattern.match("ba\N{COMBINING ACUTE ACCENT}/foo") @pytest.mark.parametrize("pattern", _make_test_patterns(str(b"ba\x80", "latin1"))) def test_invalid_unicode_pattern(pattern): assert not pattern.match("ba/foo") assert pattern.match(str(b"ba\x80/foo", "latin1")) @pytest.mark.parametrize("lines, expected", [ # "None" means all files, i.e. none excluded ([], None), (["# Comment only"], None), (["*"], []), (["# Comment", "*/something00.txt", " *whitespace* ", # Whitespace before comment " #/ws*", # Empty line "", "# EOF"], ["/more/data", "/home", " #/wsfoobar"]), (["re:.*"], []), (["re:\s"], ["/data/something00.txt", "/more/data", "/home"]), ([r"re:(.)(\1)"], ["/more/data", "/home", "\tstart/whitespace", "/whitespace/end\t"]), (["", "", "", "# This is a test with mixed pattern styles", # Case-insensitive pattern "re:(?i)BAR|ME$", "", "*whitespace*", "fm:*/something00*"], ["/more/data"]), ([r" re:^\s "], ["/data/something00.txt", "/more/data", "/home", "/whitespace/end\t"]), ([r" re:\s$ "], ["/data/something00.txt", "/more/data", "/home", " #/wsfoobar", "\tstart/whitespace"]), (["pp:./"], None), (["pp:/"], [" #/wsfoobar", "\tstart/whitespace"]), (["pp:aaabbb"], None), (["pp:/data", "pp: #/", "pp:\tstart", "pp:/whitespace"], ["/more/data", "/home"]), ]) def test_patterns_from_file(tmpdir, lines, expected): files = [ '/data/something00.txt', '/more/data', '/home', ' #/wsfoobar', '\tstart/whitespace', '/whitespace/end\t', ] def evaluate(filename): matcher = PatternMatcher(fallback=True) matcher.add(load_excludes(open(filename, "rt")), False) return [path for path in files if matcher.match(path)] exclfile = tmpdir.join("exclude.txt") with exclfile.open("wt") as fh: fh.write("\n".join(lines)) assert evaluate(str(exclfile)) == (files if expected is None else expected) @pytest.mark.parametrize("pattern, cls", [ ("", FnmatchPattern), # Default style ("*", FnmatchPattern), ("/data/*", FnmatchPattern), # fnmatch style ("fm:", FnmatchPattern), ("fm:*", FnmatchPattern), ("fm:/data/*", FnmatchPattern), ("fm:fm:/data/*", FnmatchPattern), # Regular expression ("re:", RegexPattern), ("re:.*", RegexPattern), ("re:^/something/", RegexPattern), ("re:re:^/something/", RegexPattern), # Path prefix ("pp:", PathPrefixPattern), ("pp:/", PathPrefixPattern), ("pp:/data/", PathPrefixPattern), ("pp:pp:/data/", PathPrefixPattern), # Shell-pattern style ("sh:", ShellPattern), ("sh:*", ShellPattern), ("sh:/data/*", ShellPattern), ("sh:sh:/data/*", ShellPattern), ]) def test_parse_pattern(pattern, cls): assert isinstance(parse_pattern(pattern), cls) @pytest.mark.parametrize("pattern", ["aa:", "fo:*", "00:", "x1:abc"]) def test_parse_pattern_error(pattern): with pytest.raises(ValueError): parse_pattern(pattern) def test_pattern_matcher(): pm = PatternMatcher() assert pm.fallback is None for i in ["", "foo", "bar"]: assert pm.match(i) is None pm.add([RegexPattern("^a")], "A") pm.add([RegexPattern("^b"), RegexPattern("^z")], "B") pm.add([RegexPattern("^$")], "Empty") pm.fallback = "FileNotFound" assert pm.match("") == "Empty" assert pm.match("aaa") == "A" assert pm.match("bbb") == "B" assert pm.match("ccc") == "FileNotFound" assert pm.match("xyz") == "FileNotFound" assert pm.match("z") == "B" assert PatternMatcher(fallback="hey!").fallback == "hey!" def test_compression_specs(): with pytest.raises(ValueError): CompressionSpec('') assert CompressionSpec('none') == dict(name='none') assert CompressionSpec('lz4') == dict(name='lz4') assert CompressionSpec('zlib') == dict(name='zlib', level=6) assert CompressionSpec('zlib,0') == dict(name='zlib', level=0) assert CompressionSpec('zlib,9') == dict(name='zlib', level=9) with pytest.raises(ValueError): CompressionSpec('zlib,9,invalid') assert CompressionSpec('lzma') == dict(name='lzma', level=6) assert CompressionSpec('lzma,0') == dict(name='lzma', level=0) assert CompressionSpec('lzma,9') == dict(name='lzma', level=9) with pytest.raises(ValueError): CompressionSpec('lzma,9,invalid') with pytest.raises(ValueError): CompressionSpec('invalid') def test_chunkerparams(): assert ChunkerParams('19,23,21,4095') == (19, 23, 21, 4095) assert ChunkerParams('10,23,16,4095') == (10, 23, 16, 4095) with pytest.raises(ValueError): ChunkerParams('19,24,21,4095') class MakePathSafeTestCase(BaseTestCase): def test(self): self.assert_equal(make_path_safe('/foo/bar'), 'foo/bar') self.assert_equal(make_path_safe('/foo/bar'), 'foo/bar') self.assert_equal(make_path_safe('/f/bar'), 'f/bar') self.assert_equal(make_path_safe('fo/bar'), 'fo/bar') self.assert_equal(make_path_safe('../foo/bar'), 'foo/bar') self.assert_equal(make_path_safe('../../foo/bar'), 'foo/bar') self.assert_equal(make_path_safe('/'), '.') self.assert_equal(make_path_safe('/'), '.') class MockArchive: def __init__(self, ts): self.ts = ts def __repr__(self): return repr(self.ts) class PruneSplitTestCase(BaseTestCase): def test(self): def local_to_UTC(month, day): """Convert noon on the month and day in 2013 to UTC.""" seconds = mktime(strptime('2013-%02d-%02d 12:00' % (month, day), '%Y-%m-%d %H:%M')) return datetime.fromtimestamp(seconds, tz=timezone.utc) def subset(lst, indices): return {lst[i] for i in indices} def dotest(test_archives, n, skip, indices): for ta in test_archives, reversed(test_archives): self.assert_equal(set(prune_split(ta, '%Y-%m', n, skip)), subset(test_archives, indices)) test_pairs = [(1, 1), (2, 1), (2, 28), (3, 1), (3, 2), (3, 31), (5, 1)] test_dates = [local_to_UTC(month, day) for month, day in test_pairs] test_archives = [MockArchive(date) for date in test_dates] dotest(test_archives, 3, [], [6, 5, 2]) dotest(test_archives, -1, [], [6, 5, 2, 0]) dotest(test_archives, 3, [test_archives[6]], [5, 2, 0]) dotest(test_archives, 3, [test_archives[5]], [6, 2, 0]) dotest(test_archives, 3, [test_archives[4]], [6, 5, 2]) dotest(test_archives, 0, [], []) class IntervalTestCase(BaseTestCase): def test_interval(self): self.assert_equal(interval('1H'), 1) self.assert_equal(interval('1d'), 24) self.assert_equal(interval('1w'), 168) self.assert_equal(interval('1m'), 744) self.assert_equal(interval('1y'), 8760) def test_interval_time_unit(self): with pytest.raises(ArgumentTypeError) as exc: interval('H') self.assert_equal( exc.value.args, ('Unexpected interval number "": expected an integer greater than 0',)) with pytest.raises(ArgumentTypeError) as exc: interval('-1d') self.assert_equal( exc.value.args, ('Unexpected interval number "-1": expected an integer greater than 0',)) with pytest.raises(ArgumentTypeError) as exc: interval('food') self.assert_equal( exc.value.args, ('Unexpected interval number "foo": expected an integer greater than 0',)) def test_interval_number(self): with pytest.raises(ArgumentTypeError) as exc: interval('5') self.assert_equal( exc.value.args, ("Unexpected interval time unit \"5\": expected one of ['H', 'd', 'w', 'm', 'y']",)) class PruneWithinTestCase(BaseTestCase): def test_prune_within(self): def subset(lst, indices): return {lst[i] for i in indices} def dotest(test_archives, within, indices): for ta in test_archives, reversed(test_archives): self.assert_equal(set(prune_within(ta, interval(within))), subset(test_archives, indices)) # 1 minute, 1.5 hours, 2.5 hours, 3.5 hours, 25 hours, 49 hours test_offsets = [60, 90*60, 150*60, 210*60, 25*60*60, 49*60*60] now = datetime.now(timezone.utc) test_dates = [now - timedelta(seconds=s) for s in test_offsets] test_archives = [MockArchive(date) for date in test_dates] dotest(test_archives, '1H', [0]) dotest(test_archives, '2H', [0, 1]) dotest(test_archives, '3H', [0, 1, 2]) dotest(test_archives, '24H', [0, 1, 2, 3]) dotest(test_archives, '26H', [0, 1, 2, 3, 4]) dotest(test_archives, '2d', [0, 1, 2, 3, 4]) dotest(test_archives, '50H', [0, 1, 2, 3, 4, 5]) dotest(test_archives, '3d', [0, 1, 2, 3, 4, 5]) dotest(test_archives, '1w', [0, 1, 2, 3, 4, 5]) dotest(test_archives, '1m', [0, 1, 2, 3, 4, 5]) dotest(test_archives, '1y', [0, 1, 2, 3, 4, 5]) class StableDictTestCase(BaseTestCase): def test(self): d = StableDict(foo=1, bar=2, boo=3, baz=4) self.assert_equal(list(d.items()), [('bar', 2), ('baz', 4), ('boo', 3), ('foo', 1)]) self.assert_equal(hashlib.md5(msgpack.packb(d)).hexdigest(), 'fc78df42cd60691b3ac3dd2a2b39903f') class TestParseTimestamp(BaseTestCase): def test(self): self.assert_equal(parse_timestamp('2015-04-19T20:25:00.226410'), datetime(2015, 4, 19, 20, 25, 0, 226410, timezone.utc)) self.assert_equal(parse_timestamp('2015-04-19T20:25:00'), datetime(2015, 4, 19, 20, 25, 0, 0, timezone.utc)) def test_get_cache_dir(monkeypatch): """test that get_cache_dir respects environment""" monkeypatch.delenv('BORG_CACHE_DIR', raising=False) monkeypatch.delenv('XDG_CACHE_HOME', raising=False) assert get_cache_dir() == os.path.join(os.path.expanduser('~'), '.cache', 'borg') monkeypatch.setenv('XDG_CACHE_HOME', '/var/tmp/.cache') assert get_cache_dir() == os.path.join('/var/tmp/.cache', 'borg') monkeypatch.setenv('BORG_CACHE_DIR', '/var/tmp') assert get_cache_dir() == '/var/tmp' def test_get_keys_dir(monkeypatch): """test that get_keys_dir respects environment""" monkeypatch.delenv('BORG_KEYS_DIR', raising=False) monkeypatch.delenv('XDG_CONFIG_HOME', raising=False) assert get_keys_dir() == os.path.join(os.path.expanduser('~'), '.config', 'borg', 'keys') monkeypatch.setenv('XDG_CONFIG_HOME', '/var/tmp/.config') assert get_keys_dir() == os.path.join('/var/tmp/.config', 'borg', 'keys') monkeypatch.setenv('BORG_KEYS_DIR', '/var/tmp') assert get_keys_dir() == '/var/tmp' def test_get_security_dir(monkeypatch): """test that get_security_dir respects environment""" monkeypatch.delenv('BORG_SECURITY_DIR', raising=False) monkeypatch.delenv('XDG_CONFIG_HOME', raising=False) assert get_security_dir() == os.path.join(os.path.expanduser('~'), '.config', 'borg', 'security') assert get_security_dir(repository_id='1234') == os.path.join(os.path.expanduser('~'), '.config', 'borg', 'security', '1234') monkeypatch.setenv('XDG_CONFIG_HOME', '/var/tmp/.config') assert get_security_dir() == os.path.join('/var/tmp/.config', 'borg', 'security') monkeypatch.setenv('BORG_SECURITY_DIR', '/var/tmp') assert get_security_dir() == '/var/tmp' @pytest.fixture() def stats(): stats = Statistics() stats.update(20, 10, unique=True) return stats def test_stats_basic(stats): assert stats.osize == 20 assert stats.csize == stats.usize == 10 stats.update(20, 10, unique=False) assert stats.osize == 40 assert stats.csize == 20 assert stats.usize == 10 def tests_stats_progress(stats, monkeypatch, columns=80): monkeypatch.setenv('COLUMNS', str(columns)) out = StringIO() stats.show_progress(stream=out) s = '20 B O 10 B C 10 B D 0 N ' buf = ' ' * (columns - len(s)) assert out.getvalue() == s + buf + "\r" out = StringIO() stats.update(10**3, 0, unique=False) stats.show_progress(item={b'path': 'foo'}, final=False, stream=out) s = '1.02 kB O 10 B C 10 B D 0 N foo' buf = ' ' * (columns - len(s)) assert out.getvalue() == s + buf + "\r" out = StringIO() stats.show_progress(item={b'path': 'foo'*40}, final=False, stream=out) s = '1.02 kB O 10 B C 10 B D 0 N foofoofoofoofoofoofoofo...oofoofoofoofoofoofoofoofoo' buf = ' ' * (columns - len(s)) assert out.getvalue() == s + buf + "\r" def test_stats_format(stats): assert str(stats) == """\ Original size Compressed size Deduplicated size This archive: 20 B 10 B 10 B""" s = "{0.osize_fmt}".format(stats) assert s == "20 B" # kind of redundant, but id is variable so we can't match reliably assert repr(stats) == ''.format(id(stats)) def test_file_size(): """test the size formatting routines""" si_size_map = { 0: '0 B', # no rounding necessary for those 1: '1 B', 142: '142 B', 999: '999 B', 1000: '1.00 kB', # rounding starts here 1001: '1.00 kB', # should be rounded away 1234: '1.23 kB', # should be rounded down 1235: '1.24 kB', # should be rounded up 1010: '1.01 kB', # rounded down as well 999990000: '999.99 MB', # rounded down 999990001: '999.99 MB', # rounded down 999995000: '1.00 GB', # rounded up to next unit 10**6: '1.00 MB', # and all the remaining units, megabytes 10**9: '1.00 GB', # gigabytes 10**12: '1.00 TB', # terabytes 10**15: '1.00 PB', # petabytes 10**18: '1.00 EB', # exabytes 10**21: '1.00 ZB', # zottabytes 10**24: '1.00 YB', # yottabytes } for size, fmt in si_size_map.items(): assert format_file_size(size) == fmt def test_file_size_precision(): assert format_file_size(1234, precision=1) == '1.2 kB' # rounded down assert format_file_size(1254, precision=1) == '1.3 kB' # rounded up assert format_file_size(999990000, precision=1) == '1.0 GB' # and not 999.9 MB or 1000.0 MB def test_is_slow_msgpack(): saved_packer = msgpack.Packer try: msgpack.Packer = msgpack.fallback.Packer assert is_slow_msgpack() finally: msgpack.Packer = saved_packer # this assumes that we have fast msgpack on test platform: assert not is_slow_msgpack() class TestBuffer: def test_type(self): buffer = Buffer(bytearray) assert isinstance(buffer.get(), bytearray) buffer = Buffer(bytes) # don't do that in practice assert isinstance(buffer.get(), bytes) def test_len(self): buffer = Buffer(bytearray, size=0) b = buffer.get() assert len(buffer) == len(b) == 0 buffer = Buffer(bytearray, size=1234) b = buffer.get() assert len(buffer) == len(b) == 1234 def test_resize(self): buffer = Buffer(bytearray, size=100) assert len(buffer) == 100 b1 = buffer.get() buffer.resize(200) assert len(buffer) == 200 b2 = buffer.get() assert b2 is not b1 # new, bigger buffer buffer.resize(100) assert len(buffer) >= 100 b3 = buffer.get() assert b3 is b2 # still same buffer (200) buffer.resize(100, init=True) assert len(buffer) == 100 # except on init b4 = buffer.get() assert b4 is not b3 # new, smaller buffer def test_limit(self): buffer = Buffer(bytearray, size=100, limit=200) buffer.resize(200) assert len(buffer) == 200 with pytest.raises(Buffer.MemoryLimitExceeded): buffer.resize(201) assert len(buffer) == 200 def test_get(self): buffer = Buffer(bytearray, size=100, limit=200) b1 = buffer.get(50) assert len(b1) >= 50 # == 100 b2 = buffer.get(100) assert len(b2) >= 100 # == 100 assert b2 is b1 # did not need resizing yet b3 = buffer.get(200) assert len(b3) == 200 assert b3 is not b2 # new, resized buffer with pytest.raises(Buffer.MemoryLimitExceeded): buffer.get(201) # beyond limit assert len(buffer) == 200 def test_yes_input(): inputs = list(TRUISH) input = FakeInputs(inputs) for i in inputs: assert yes(input=input) inputs = list(FALSISH) input = FakeInputs(inputs) for i in inputs: assert not yes(input=input) def test_yes_input_defaults(): inputs = list(DEFAULTISH) input = FakeInputs(inputs) for i in inputs: assert yes(default=True, input=input) input = FakeInputs(inputs) for i in inputs: assert not yes(default=False, input=input) def test_yes_input_custom(): input = FakeInputs(['YES', 'SURE', 'NOPE', ]) assert yes(truish=('YES', ), input=input) assert yes(truish=('SURE', ), input=input) assert not yes(falsish=('NOPE', ), input=input) def test_yes_env(monkeypatch): for value in TRUISH: monkeypatch.setenv('OVERRIDE_THIS', value) assert yes(env_var_override='OVERRIDE_THIS') for value in FALSISH: monkeypatch.setenv('OVERRIDE_THIS', value) assert not yes(env_var_override='OVERRIDE_THIS') def test_yes_env_default(monkeypatch): for value in DEFAULTISH: monkeypatch.setenv('OVERRIDE_THIS', value) assert yes(env_var_override='OVERRIDE_THIS', default=True) assert not yes(env_var_override='OVERRIDE_THIS', default=False) def test_yes_defaults(): input = FakeInputs(['invalid', '', ' ']) assert not yes(input=input) # default=False assert not yes(input=input) assert not yes(input=input) input = FakeInputs(['invalid', '', ' ']) assert yes(default=True, input=input) assert yes(default=True, input=input) assert yes(default=True, input=input) input = FakeInputs([]) assert yes(default=True, input=input) assert not yes(default=False, input=input) with pytest.raises(ValueError): yes(default=None) def test_yes_retry(): input = FakeInputs(['foo', 'bar', TRUISH[0], ]) assert yes(retry_msg='Retry: ', input=input) input = FakeInputs(['foo', 'bar', FALSISH[0], ]) assert not yes(retry_msg='Retry: ', input=input) def test_yes_no_retry(): input = FakeInputs(['foo', 'bar', TRUISH[0], ]) assert not yes(retry=False, default=False, input=input) input = FakeInputs(['foo', 'bar', FALSISH[0], ]) assert yes(retry=False, default=True, input=input) def test_yes_output(capfd): input = FakeInputs(['invalid', 'y', 'n']) assert yes(msg='intro-msg', false_msg='false-msg', true_msg='true-msg', retry_msg='retry-msg', input=input) out, err = capfd.readouterr() assert out == '' assert 'intro-msg' in err assert 'retry-msg' in err assert 'true-msg' in err assert not yes(msg='intro-msg', false_msg='false-msg', true_msg='true-msg', retry_msg='retry-msg', input=input) out, err = capfd.readouterr() assert out == '' assert 'intro-msg' in err assert 'retry-msg' not in err assert 'false-msg' in err def test_yes_env_output(capfd, monkeypatch): env_var = 'OVERRIDE_SOMETHING' monkeypatch.setenv(env_var, 'yes') assert yes(env_var_override=env_var) out, err = capfd.readouterr() assert out == '' assert env_var in err assert 'yes' in err def test_progress_percentage_multiline(capfd): pi = ProgressIndicatorPercent(1000, step=5, start=0, same_line=False, msg="%3.0f%%", file=sys.stderr) pi.show(0) out, err = capfd.readouterr() assert err == ' 0%\n' pi.show(420) out, err = capfd.readouterr() assert err == ' 42%\n' pi.show(1000) out, err = capfd.readouterr() assert err == '100%\n' pi.finish() out, err = capfd.readouterr() assert err == '' def test_progress_percentage_sameline(capfd): pi = ProgressIndicatorPercent(1000, step=5, start=0, same_line=True, msg="%3.0f%%", file=sys.stderr) pi.show(0) out, err = capfd.readouterr() assert err == ' 0%\r' pi.show(420) out, err = capfd.readouterr() assert err == ' 42%\r' pi.show(1000) out, err = capfd.readouterr() assert err == '100%\r' pi.finish() out, err = capfd.readouterr() assert err == ' ' * 4 + '\r' def test_progress_percentage_step(capfd): pi = ProgressIndicatorPercent(100, step=2, start=0, same_line=False, msg="%3.0f%%", file=sys.stderr) pi.show() out, err = capfd.readouterr() assert err == ' 0%\n' pi.show() out, err = capfd.readouterr() assert err == '' # no output at 1% as we have step == 2 pi.show() out, err = capfd.readouterr() assert err == ' 2%\n' def test_progress_endless(capfd): pi = ProgressIndicatorEndless(step=1, file=sys.stderr) pi.show() out, err = capfd.readouterr() assert err == '.' pi.show() out, err = capfd.readouterr() assert err == '.' pi.finish() out, err = capfd.readouterr() assert err == '\n' def test_progress_endless_step(capfd): pi = ProgressIndicatorEndless(step=2, file=sys.stderr) pi.show() out, err = capfd.readouterr() assert err == '' # no output here as we have step == 2 pi.show() out, err = capfd.readouterr() assert err == '.' pi.show() out, err = capfd.readouterr() assert err == '' # no output here as we have step == 2 pi.show() out, err = capfd.readouterr() assert err == '.' def test_format_line(): data = dict(foo='bar baz') assert format_line('', data) == '' assert format_line('{foo}', data) == 'bar baz' assert format_line('foo{foo}foo', data) == 'foobar bazfoo' def test_format_line_erroneous(): data = dict() with pytest.raises(PlaceholderError): assert format_line('{invalid}', data) with pytest.raises(PlaceholderError): assert format_line('{}', data) def test_safe_timestamps(): if SUPPORT_32BIT_PLATFORMS: # ns fit into int64 assert safe_ns(2 ** 64) <= 2 ** 63 - 1 assert safe_ns(-1) == 0 # s fit into int32 assert safe_s(2 ** 64) <= 2 ** 31 - 1 assert safe_s(-1) == 0 # datetime won't fall over its y10k problem beyond_y10k = 2 ** 100 with pytest.raises(OverflowError): datetime.utcfromtimestamp(beyond_y10k) assert datetime.utcfromtimestamp(safe_s(beyond_y10k)) > datetime(2038, 1, 1) assert datetime.utcfromtimestamp(safe_ns(beyond_y10k) / 1000000000) > datetime(2038, 1, 1) else: # ns fit into int64 assert safe_ns(2 ** 64) <= 2 ** 63 - 1 assert safe_ns(-1) == 0 # s are so that their ns conversion fits into int64 assert safe_s(2 ** 64) * 1000000000 <= 2 ** 63 - 1 assert safe_s(-1) == 0 # datetime won't fall over its y10k problem beyond_y10k = 2 ** 100 with pytest.raises(OverflowError): datetime.utcfromtimestamp(beyond_y10k) assert datetime.utcfromtimestamp(safe_s(beyond_y10k)) > datetime(2262, 1, 1) assert datetime.utcfromtimestamp(safe_ns(beyond_y10k) / 1000000000) > datetime(2262, 1, 1) borgbackup-1.0.13/borg/testsuite/key.py0000644000175000017500000002115113431513133020012 0ustar useruser00000000000000import os import re import shutil import tempfile from binascii import hexlify, unhexlify import msgpack import pytest from ..crypto import bytes_to_long, num_aes_blocks from ..key import PlaintextKey, PassphraseKey, KeyfileKey from ..key import UnsupportedManifestError, TAMRequiredError, TAMUnsupportedSuiteError, TAMInvalid from ..helpers import Location, StableDict from . import BaseTestCase class KeyTestCase(BaseTestCase): class MockArgs: location = Location(tempfile.mkstemp()[1]) keyfile2_key_file = """ BORG_KEY 0000000000000000000000000000000000000000000000000000000000000000 hqppdGVyYXRpb25zzgABhqCkaGFzaNoAIMyonNI+7Cjv0qHi0AOBM6bLGxACJhfgzVD2oq bIS9SFqWFsZ29yaXRobaZzaGEyNTakc2FsdNoAINNK5qqJc1JWSUjACwFEWGTdM7Nd0a5l 1uBGPEb+9XM9p3ZlcnNpb24BpGRhdGHaANAYDT5yfPpU099oBJwMomsxouKyx/OG4QIXK2 hQCG2L2L/9PUu4WIuKvGrsXoP7syemujNfcZws5jLp2UPva4PkQhQsrF1RYDEMLh2eF9Ol rwtkThq1tnh7KjWMG9Ijt7/aoQtq0zDYP/xaFF8XXSJxiyP5zjH5+spB6RL0oQHvbsliSh /cXJq7jrqmrJ1phd6dg4SHAM/i+hubadZoS6m25OQzYAW09wZD/phG8OVa698Z5ed3HTaT SmrtgJL3EoOKgUI9d6BLE4dJdBqntifo""".strip() keyfile2_cdata = unhexlify(re.sub('\W', '', """ 0055f161493fcfc16276e8c31493c4641e1eb19a79d0326fad0291e5a9c98e5933 00000000000003e8d21eaf9b86c297a8cd56432e1915bb """)) keyfile2_id = unhexlify('c3fbf14bc001ebcc3cd86e696c13482ed071740927cd7cbe1b01b4bfcee49314') def setUp(self): self.tmppath = tempfile.mkdtemp() os.environ['BORG_KEYS_DIR'] = self.tmppath def tearDown(self): shutil.rmtree(self.tmppath) class MockRepository: class _Location: orig = '/some/place' def canonical_path(self): return self.orig _location = _Location() id = bytes(32) def test_plaintext(self): key = PlaintextKey.create(None, None) data = b'foo' self.assert_equal(hexlify(key.id_hash(data)), b'2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae') self.assert_equal(data, key.decrypt(key.id_hash(data), key.encrypt(data))) def test_keyfile(self): os.environ['BORG_PASSPHRASE'] = 'test' key = KeyfileKey.create(self.MockRepository(), self.MockArgs()) self.assert_equal(bytes_to_long(key.enc_cipher.iv, 8), 0) manifest = key.encrypt(b'XXX') self.assert_equal(key.extract_nonce(manifest), 0) manifest2 = key.encrypt(b'XXX') self.assert_not_equal(manifest, manifest2) self.assert_equal(key.decrypt(None, manifest), key.decrypt(None, manifest2)) self.assert_equal(key.extract_nonce(manifest2), 1) iv = key.extract_nonce(manifest) key2 = KeyfileKey.detect(self.MockRepository(), manifest) self.assert_equal(bytes_to_long(key2.enc_cipher.iv, 8), iv + num_aes_blocks(len(manifest) - KeyfileKey.PAYLOAD_OVERHEAD)) # Key data sanity check self.assert_equal(len(set([key2.id_key, key2.enc_key, key2.enc_hmac_key])), 3) self.assert_equal(key2.chunk_seed == 0, False) data = b'foo' self.assert_equal(data, key2.decrypt(key.id_hash(data), key.encrypt(data))) def test_keyfile2(self): with open(os.path.join(os.environ['BORG_KEYS_DIR'], 'keyfile'), 'w') as fd: fd.write(self.keyfile2_key_file) os.environ['BORG_PASSPHRASE'] = 'passphrase' key = KeyfileKey.detect(self.MockRepository(), self.keyfile2_cdata) self.assert_equal(key.decrypt(self.keyfile2_id, self.keyfile2_cdata), b'payload') def test_passphrase(self): os.environ['BORG_PASSPHRASE'] = 'test' key = PassphraseKey.create(self.MockRepository(), None) self.assert_equal(bytes_to_long(key.enc_cipher.iv, 8), 0) self.assert_equal(hexlify(key.id_key), b'793b0717f9d8fb01c751a487e9b827897ceea62409870600013fbc6b4d8d7ca6') self.assert_equal(hexlify(key.enc_hmac_key), b'b885a05d329a086627412a6142aaeb9f6c54ab7950f996dd65587251f6bc0901') self.assert_equal(hexlify(key.enc_key), b'2ff3654c6daf7381dbbe718d2b20b4f1ea1e34caa6cc65f6bb3ac376b93fed2a') self.assert_equal(key.chunk_seed, -775740477) manifest = key.encrypt(b'XXX') self.assert_equal(key.extract_nonce(manifest), 0) manifest2 = key.encrypt(b'XXX') self.assert_not_equal(manifest, manifest2) self.assert_equal(key.decrypt(None, manifest), key.decrypt(None, manifest2)) self.assert_equal(key.extract_nonce(manifest2), 1) iv = key.extract_nonce(manifest) key2 = PassphraseKey.detect(self.MockRepository(), manifest) self.assert_equal(bytes_to_long(key2.enc_cipher.iv, 8), iv + num_aes_blocks(len(manifest) - PassphraseKey.PAYLOAD_OVERHEAD)) self.assert_equal(key.id_key, key2.id_key) self.assert_equal(key.enc_hmac_key, key2.enc_hmac_key) self.assert_equal(key.enc_key, key2.enc_key) self.assert_equal(key.chunk_seed, key2.chunk_seed) data = b'foo' self.assert_equal(hexlify(key.id_hash(data)), b'818217cf07d37efad3860766dcdf1d21e401650fed2d76ed1d797d3aae925990') self.assert_equal(data, key2.decrypt(key2.id_hash(data), key.encrypt(data))) class TestTAM: @pytest.fixture def key(self, monkeypatch): monkeypatch.setenv('BORG_PASSPHRASE', 'test') return KeyfileKey.create(KeyTestCase.MockRepository(), KeyTestCase.MockArgs()) def test_unpack_future(self, key): blob = b'\xc1\xc1\xc1\xc1foobar' with pytest.raises(UnsupportedManifestError): key.unpack_and_verify_manifest(blob) blob = b'\xc1\xc1\xc1' with pytest.raises((ValueError, msgpack.UnpackException)): key.unpack_and_verify_manifest(blob) def test_missing_when_required(self, key): blob = msgpack.packb({}) with pytest.raises(TAMRequiredError): key.unpack_and_verify_manifest(blob) def test_missing(self, key): blob = msgpack.packb({}) key.tam_required = False unpacked, verified = key.unpack_and_verify_manifest(blob) assert unpacked == {} assert not verified def test_unknown_type_when_required(self, key): blob = msgpack.packb({ 'tam': { 'type': 'HMAC_VOLLBIT', }, }) with pytest.raises(TAMUnsupportedSuiteError): key.unpack_and_verify_manifest(blob) def test_unknown_type(self, key): blob = msgpack.packb({ 'tam': { 'type': 'HMAC_VOLLBIT', }, }) key.tam_required = False unpacked, verified = key.unpack_and_verify_manifest(blob) assert unpacked == {} assert not verified @pytest.mark.parametrize('tam, exc', ( ({}, TAMUnsupportedSuiteError), ({'type': b'\xff'}, TAMUnsupportedSuiteError), (None, TAMInvalid), (1234, TAMInvalid), )) def test_invalid(self, key, tam, exc): blob = msgpack.packb({ 'tam': tam, }) with pytest.raises(exc): key.unpack_and_verify_manifest(blob) @pytest.mark.parametrize('hmac, salt', ( ({}, bytes(64)), (bytes(64), {}), (None, bytes(64)), (bytes(64), None), )) def test_wrong_types(self, key, hmac, salt): data = { 'tam': { 'type': 'HKDF_HMAC_SHA512', 'hmac': hmac, 'salt': salt }, } tam = data['tam'] if hmac is None: del tam['hmac'] if salt is None: del tam['salt'] blob = msgpack.packb(data) with pytest.raises(TAMInvalid): key.unpack_and_verify_manifest(blob) def test_round_trip(self, key): data = {'foo': 'bar'} blob = key.pack_and_authenticate_metadata(data) assert blob.startswith(b'\x82') unpacked = msgpack.unpackb(blob) assert unpacked[b'tam'][b'type'] == b'HKDF_HMAC_SHA512' unpacked, verified = key.unpack_and_verify_manifest(blob) assert verified assert unpacked[b'foo'] == b'bar' assert b'tam' not in unpacked @pytest.mark.parametrize('which', (b'hmac', b'salt')) def test_tampered(self, key, which): data = {'foo': 'bar'} blob = key.pack_and_authenticate_metadata(data) assert blob.startswith(b'\x82') unpacked = msgpack.unpackb(blob, object_hook=StableDict) assert len(unpacked[b'tam'][which]) == 64 unpacked[b'tam'][which] = unpacked[b'tam'][which][0:32] + bytes(32) assert len(unpacked[b'tam'][which]) == 64 blob = msgpack.packb(unpacked) with pytest.raises(TAMInvalid): key.unpack_and_verify_manifest(blob) borgbackup-1.0.13/borg/testsuite/locking.py0000644000175000017500000001507313431513133020656 0ustar useruser00000000000000import time import pytest from ..locking import get_id, TimeoutTimer, ExclusiveLock, Lock, LockRoster, \ ADD, REMOVE, SHARED, EXCLUSIVE, LockTimeout ID1 = "foo", 1, 1 ID2 = "bar", 2, 2 def test_id(): hostname, pid, tid = get_id() assert isinstance(hostname, str) assert isinstance(pid, int) assert isinstance(tid, int) assert len(hostname) > 0 assert pid > 0 class TestTimeoutTimer: def test_timeout(self): timeout = 0.5 t = TimeoutTimer(timeout).start() assert not t.timed_out() time.sleep(timeout * 1.5) assert t.timed_out() def test_notimeout_sleep(self): timeout, sleep = None, 0.5 t = TimeoutTimer(timeout, sleep).start() assert not t.timed_out_or_sleep() assert time.time() >= t.start_time + 1 * sleep assert not t.timed_out_or_sleep() assert time.time() >= t.start_time + 2 * sleep @pytest.fixture() def lockpath(tmpdir): return str(tmpdir.join('lock')) class TestExclusiveLock: def test_checks(self, lockpath): with ExclusiveLock(lockpath, timeout=1) as lock: assert lock.is_locked() and lock.by_me() def test_acquire_break_reacquire(self, lockpath): lock = ExclusiveLock(lockpath, id=ID1).acquire() lock.break_lock() with ExclusiveLock(lockpath, id=ID2): pass def test_timeout(self, lockpath): with ExclusiveLock(lockpath, id=ID1): with pytest.raises(LockTimeout): ExclusiveLock(lockpath, id=ID2, timeout=0.1).acquire() def test_migrate_lock(self, lockpath): old_id, new_id = ID1, ID2 assert old_id[1] != new_id[1] # different PIDs (like when doing daemonize()) lock = ExclusiveLock(lockpath, id=old_id).acquire() assert lock.id == old_id # lock is for old id / PID old_unique_name = lock.unique_name assert lock.by_me() # we have the lock lock.migrate_lock(old_id, new_id) # fix the lock assert lock.id == new_id # lock corresponds to the new id / PID new_unique_name = lock.unique_name assert lock.by_me() # we still have the lock assert old_unique_name != new_unique_name # locking filename is different now class TestLock: def test_shared(self, lockpath): lock1 = Lock(lockpath, exclusive=False, id=ID1).acquire() lock2 = Lock(lockpath, exclusive=False, id=ID2).acquire() assert len(lock1._roster.get(SHARED)) == 2 assert len(lock1._roster.get(EXCLUSIVE)) == 0 assert not lock1._roster.empty(SHARED, EXCLUSIVE) assert lock1._roster.empty(EXCLUSIVE) lock1.release() lock2.release() def test_exclusive(self, lockpath): with Lock(lockpath, exclusive=True, id=ID1) as lock: assert len(lock._roster.get(SHARED)) == 0 assert len(lock._roster.get(EXCLUSIVE)) == 1 assert not lock._roster.empty(SHARED, EXCLUSIVE) def test_upgrade(self, lockpath): with Lock(lockpath, exclusive=False) as lock: lock.upgrade() lock.upgrade() # NOP assert len(lock._roster.get(SHARED)) == 0 assert len(lock._roster.get(EXCLUSIVE)) == 1 assert not lock._roster.empty(SHARED, EXCLUSIVE) def test_downgrade(self, lockpath): with Lock(lockpath, exclusive=True) as lock: lock.downgrade() lock.downgrade() # NOP assert len(lock._roster.get(SHARED)) == 1 assert len(lock._roster.get(EXCLUSIVE)) == 0 def test_got_exclusive_lock(self, lockpath): lock = Lock(lockpath, exclusive=True, id=ID1) assert not lock.got_exclusive_lock() lock.acquire() assert lock.got_exclusive_lock() lock.release() assert not lock.got_exclusive_lock() def test_break(self, lockpath): lock = Lock(lockpath, exclusive=True, id=ID1).acquire() lock.break_lock() assert len(lock._roster.get(SHARED)) == 0 assert len(lock._roster.get(EXCLUSIVE)) == 0 with Lock(lockpath, exclusive=True, id=ID2): pass def test_timeout(self, lockpath): with Lock(lockpath, exclusive=False, id=ID1): with pytest.raises(LockTimeout): Lock(lockpath, exclusive=True, id=ID2, timeout=0.1).acquire() with Lock(lockpath, exclusive=True, id=ID1): with pytest.raises(LockTimeout): Lock(lockpath, exclusive=False, id=ID2, timeout=0.1).acquire() with Lock(lockpath, exclusive=True, id=ID1): with pytest.raises(LockTimeout): Lock(lockpath, exclusive=True, id=ID2, timeout=0.1).acquire() def test_migrate_lock(self, lockpath): old_id, new_id = ID1, ID2 assert old_id[1] != new_id[1] # different PIDs (like when doing daemonize()) lock = Lock(lockpath, id=old_id, exclusive=True).acquire() assert lock.id == old_id lock.migrate_lock(old_id, new_id) # fix the lock assert lock.id == new_id lock.release() lock = Lock(lockpath, id=old_id, exclusive=False).acquire() assert lock.id == old_id lock.migrate_lock(old_id, new_id) # fix the lock assert lock.id == new_id lock.release() @pytest.fixture() def rosterpath(tmpdir): return str(tmpdir.join('roster')) class TestLockRoster: def test_empty(self, rosterpath): roster = LockRoster(rosterpath) empty = roster.load() roster.save(empty) assert empty == {} def test_modify_get(self, rosterpath): roster1 = LockRoster(rosterpath, id=ID1) assert roster1.get(SHARED) == set() roster1.modify(SHARED, ADD) assert roster1.get(SHARED) == {ID1, } roster2 = LockRoster(rosterpath, id=ID2) roster2.modify(SHARED, ADD) assert roster2.get(SHARED) == {ID1, ID2, } roster1 = LockRoster(rosterpath, id=ID1) roster1.modify(SHARED, REMOVE) assert roster1.get(SHARED) == {ID2, } roster2 = LockRoster(rosterpath, id=ID2) roster2.modify(SHARED, REMOVE) assert roster2.get(SHARED) == set() def test_migrate_lock(self, rosterpath): old_id, new_id = ID1, ID2 assert old_id[1] != new_id[1] # different PIDs (like when doing daemonize()) roster = LockRoster(rosterpath, id=old_id) assert roster.id == old_id roster.modify(SHARED, ADD) assert roster.get(SHARED) == {old_id} roster.migrate_lock(SHARED, old_id, new_id) # fix the lock assert roster.id == new_id assert roster.get(SHARED) == {new_id} borgbackup-1.0.13/borg/testsuite/logger.py0000644000175000017500000000304713431513133020505 0ustar useruser00000000000000import logging from io import StringIO import pytest from ..logger import find_parent_module, create_logger, setup_logging logger = create_logger() @pytest.fixture() def io_logger(): io = StringIO() handler = setup_logging(stream=io, env_var=None) handler.setFormatter(logging.Formatter('%(name)s: %(message)s')) logger.setLevel(logging.DEBUG) return io def test_setup_logging(io_logger): logger.info('hello world') assert io_logger.getvalue() == "borg.testsuite.logger: hello world\n" def test_multiple_loggers(io_logger): logger = logging.getLogger(__name__) logger.info('hello world 1') assert io_logger.getvalue() == "borg.testsuite.logger: hello world 1\n" logger = logging.getLogger('borg.testsuite.logger') logger.info('hello world 2') assert io_logger.getvalue() == "borg.testsuite.logger: hello world 1\nborg.testsuite.logger: hello world 2\n" io_logger.truncate(0) io_logger.seek(0) logger = logging.getLogger('borg.testsuite.logger') logger.info('hello world 2') assert io_logger.getvalue() == "borg.testsuite.logger: hello world 2\n" def test_parent_module(): assert find_parent_module() == __name__ def test_lazy_logger(): # just calling all the methods of the proxy logger.setLevel(logging.DEBUG) logger.debug("debug") logger.info("info") logger.warning("warning") logger.error("error") logger.critical("critical") logger.log(logging.INFO, "info") try: raise Exception except Exception: logger.exception("exception") borgbackup-1.0.13/borg/testsuite/lrucache.py0000644000175000017500000000245613431513133021017 0ustar useruser00000000000000from ..lrucache import LRUCache import pytest from tempfile import TemporaryFile class TestLRUCache: def test_lrucache(self): c = LRUCache(2, dispose=lambda _: None) assert len(c) == 0 assert c.items() == set() for i, x in enumerate('abc'): c[x] = i assert len(c) == 2 assert c.items() == set([('b', 1), ('c', 2)]) assert 'a' not in c assert 'b' in c with pytest.raises(KeyError): c['a'] assert c['b'] == 1 assert c['c'] == 2 c['d'] = 3 assert len(c) == 2 assert c['c'] == 2 assert c['d'] == 3 del c['c'] assert len(c) == 1 with pytest.raises(KeyError): c['c'] assert c['d'] == 3 c.clear() assert c.items() == set() def test_dispose(self): c = LRUCache(2, dispose=lambda f: f.close()) f1 = TemporaryFile() f2 = TemporaryFile() f3 = TemporaryFile() c[1] = f1 c[2] = f2 assert not f2.closed c[3] = f3 assert 1 not in c assert f1.closed assert 2 in c assert not f2.closed del c[2] assert 2 not in c assert f2.closed c.clear() assert c.items() == set() assert f3.closed borgbackup-1.0.13/borg/testsuite/platform.py0000644000175000017500000001451513431513133021054 0ustar useruser00000000000000import os import shutil import sys import tempfile import unittest from ..platform import acl_get, acl_set from . import BaseTestCase ACCESS_ACL = """ user::rw- user:root:rw-:0 user:9999:r--:9999 group::r-- group:root:r--:0 group:9999:r--:9999 mask::rw- other::r-- """.strip().encode('ascii') DEFAULT_ACL = """ user::rw- user:root:r--:0 user:8888:r--:8888 group::r-- group:root:r--:0 group:8888:r--:8888 mask::rw- other::r-- """.strip().encode('ascii') def fakeroot_detected(): return 'FAKEROOTKEY' in os.environ @unittest.skipUnless(sys.platform.startswith('linux'), 'linux only test') @unittest.skipIf(fakeroot_detected(), 'not compatible with fakeroot') class PlatformLinuxTestCase(BaseTestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() def tearDown(self): shutil.rmtree(self.tmpdir) def get_acl(self, path, numeric_owner=False): item = {} acl_get(path, item, os.stat(path), numeric_owner=numeric_owner) return item def set_acl(self, path, access=None, default=None, numeric_owner=False): item = {b'acl_access': access, b'acl_default': default} acl_set(path, item, numeric_owner=numeric_owner) def test_access_acl(self): file = tempfile.NamedTemporaryFile() self.assert_equal(self.get_acl(file.name), {}) self.set_acl(file.name, access=b'user::rw-\ngroup::r--\nmask::rw-\nother::---\nuser:root:rw-:9999\ngroup:root:rw-:9999\n', numeric_owner=False) self.assert_in(b'user:root:rw-:0', self.get_acl(file.name)[b'acl_access']) self.assert_in(b'group:root:rw-:0', self.get_acl(file.name)[b'acl_access']) self.assert_in(b'user:0:rw-:0', self.get_acl(file.name, numeric_owner=True)[b'acl_access']) file2 = tempfile.NamedTemporaryFile() self.set_acl(file2.name, access=b'user::rw-\ngroup::r--\nmask::rw-\nother::---\nuser:root:rw-:9999\ngroup:root:rw-:9999\n', numeric_owner=True) self.assert_in(b'user:9999:rw-:9999', self.get_acl(file2.name)[b'acl_access']) self.assert_in(b'group:9999:rw-:9999', self.get_acl(file2.name)[b'acl_access']) def test_default_acl(self): self.assert_equal(self.get_acl(self.tmpdir), {}) self.set_acl(self.tmpdir, access=ACCESS_ACL, default=DEFAULT_ACL) self.assert_equal(self.get_acl(self.tmpdir)[b'acl_access'], ACCESS_ACL) self.assert_equal(self.get_acl(self.tmpdir)[b'acl_default'], DEFAULT_ACL) def test_non_ascii_acl(self): # Testing non-ascii ACL processing to see whether our code is robust. # I have no idea whether non-ascii ACLs are allowed by the standard, # but in practice they seem to be out there and must not make our code explode. file = tempfile.NamedTemporaryFile() self.assert_equal(self.get_acl(file.name), {}) nothing_special = 'user::rw-\ngroup::r--\nmask::rw-\nother::---\n'.encode('ascii') # TODO: can this be tested without having an existing system user übel with uid 666 gid 666? user_entry = 'user:übel:rw-:666'.encode('utf-8') user_entry_numeric = 'user:666:rw-:666'.encode('ascii') group_entry = 'group:übel:rw-:666'.encode('utf-8') group_entry_numeric = 'group:666:rw-:666'.encode('ascii') acl = b'\n'.join([nothing_special, user_entry, group_entry]) self.set_acl(file.name, access=acl, numeric_owner=False) acl_access = self.get_acl(file.name, numeric_owner=False)[b'acl_access'] self.assert_in(user_entry, acl_access) self.assert_in(group_entry, acl_access) acl_access_numeric = self.get_acl(file.name, numeric_owner=True)[b'acl_access'] self.assert_in(user_entry_numeric, acl_access_numeric) self.assert_in(group_entry_numeric, acl_access_numeric) file2 = tempfile.NamedTemporaryFile() self.set_acl(file2.name, access=acl, numeric_owner=True) acl_access = self.get_acl(file2.name, numeric_owner=False)[b'acl_access'] self.assert_in(user_entry, acl_access) self.assert_in(group_entry, acl_access) acl_access_numeric = self.get_acl(file.name, numeric_owner=True)[b'acl_access'] self.assert_in(user_entry_numeric, acl_access_numeric) self.assert_in(group_entry_numeric, acl_access_numeric) def test_utils(self): from ..platform_linux import acl_use_local_uid_gid self.assert_equal(acl_use_local_uid_gid(b'user:nonexistent1234:rw-:1234'), b'user:1234:rw-') self.assert_equal(acl_use_local_uid_gid(b'group:nonexistent1234:rw-:1234'), b'group:1234:rw-') self.assert_equal(acl_use_local_uid_gid(b'user:root:rw-:0'), b'user:0:rw-') self.assert_equal(acl_use_local_uid_gid(b'group:root:rw-:0'), b'group:0:rw-') @unittest.skipUnless(sys.platform.startswith('darwin'), 'OS X only test') @unittest.skipIf(fakeroot_detected(), 'not compatible with fakeroot') class PlatformDarwinTestCase(BaseTestCase): def setUp(self): self.tmpdir = tempfile.mkdtemp() def tearDown(self): shutil.rmtree(self.tmpdir) def get_acl(self, path, numeric_owner=False): item = {} acl_get(path, item, os.stat(path), numeric_owner=numeric_owner) return item def set_acl(self, path, acl, numeric_owner=False): item = {b'acl_extended': acl} acl_set(path, item, numeric_owner=numeric_owner) def test_access_acl(self): file = tempfile.NamedTemporaryFile() file2 = tempfile.NamedTemporaryFile() self.assert_equal(self.get_acl(file.name), {}) self.set_acl(file.name, b'!#acl 1\ngroup:ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000000:staff:0:allow:read\nuser:FFFFEEEE-DDDD-CCCC-BBBB-AAAA00000000:root:0:allow:read\n', numeric_owner=False) self.assert_in(b'group:ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000014:staff:20:allow:read', self.get_acl(file.name)[b'acl_extended']) self.assert_in(b'user:FFFFEEEE-DDDD-CCCC-BBBB-AAAA00000000:root:0:allow:read', self.get_acl(file.name)[b'acl_extended']) self.set_acl(file2.name, b'!#acl 1\ngroup:ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000000:staff:0:allow:read\nuser:FFFFEEEE-DDDD-CCCC-BBBB-AAAA00000000:root:0:allow:read\n', numeric_owner=True) self.assert_in(b'group:ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000000:wheel:0:allow:read', self.get_acl(file2.name)[b'acl_extended']) self.assert_in(b'group:ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000000::0:allow:read', self.get_acl(file2.name, numeric_owner=True)[b'acl_extended']) borgbackup-1.0.13/borg/testsuite/repository.py0000644000175000017500000004630113431513133021445 0ustar useruser00000000000000import logging import os import shutil import sys import tempfile from unittest.mock import patch import pytest from ..hashindex import NSIndex from ..helpers import Location, IntegrityError from ..locking import Lock, LockFailed from ..remote import RemoteRepository, InvalidRPCMethod from ..repository import Repository, LoggedIO, TAG_DELETE, MAX_DATA_SIZE, MAGIC from . import BaseTestCase from .hashindex import H UNSPECIFIED = object() # for default values where we can't use None class RepositoryTestCaseBase(BaseTestCase): key_size = 32 exclusive = True def open(self, create=False, exclusive=UNSPECIFIED): if exclusive is UNSPECIFIED: exclusive = self.exclusive return Repository(os.path.join(self.tmppath, 'repository'), exclusive=exclusive, create=create) def setUp(self): self.tmppath = tempfile.mkdtemp() self.repository = self.open(create=True) self.repository.__enter__() def tearDown(self): self.repository.close() shutil.rmtree(self.tmppath) def reopen(self, exclusive=UNSPECIFIED): if self.repository: self.repository.close() self.repository = self.open(exclusive=exclusive) class RepositoryTestCase(RepositoryTestCaseBase): def test1(self): for x in range(100): self.repository.put(H(x), b'SOMEDATA') key50 = H(50) self.assert_equal(self.repository.get(key50), b'SOMEDATA') self.repository.delete(key50) self.assert_raises(Repository.ObjectNotFound, lambda: self.repository.get(key50)) self.repository.commit() self.repository.close() with self.open() as repository2: self.assert_raises(Repository.ObjectNotFound, lambda: repository2.get(key50)) for x in range(100): if x == 50: continue self.assert_equal(repository2.get(H(x)), b'SOMEDATA') def test2(self): """Test multiple sequential transactions """ self.repository.put(H(0), b'foo') self.repository.put(H(1), b'foo') self.repository.commit() self.repository.delete(H(0)) self.repository.put(H(1), b'bar') self.repository.commit() self.assert_equal(self.repository.get(H(1)), b'bar') def test_consistency(self): """Test cache consistency """ self.repository.put(H(0), b'foo') self.assert_equal(self.repository.get(H(0)), b'foo') self.repository.put(H(0), b'foo2') self.assert_equal(self.repository.get(H(0)), b'foo2') self.repository.put(H(0), b'bar') self.assert_equal(self.repository.get(H(0)), b'bar') self.repository.delete(H(0)) self.assert_raises(Repository.ObjectNotFound, lambda: self.repository.get(H(0))) def test_consistency2(self): """Test cache consistency2 """ self.repository.put(H(0), b'foo') self.assert_equal(self.repository.get(H(0)), b'foo') self.repository.commit() self.repository.put(H(0), b'foo2') self.assert_equal(self.repository.get(H(0)), b'foo2') self.repository.rollback() self.assert_equal(self.repository.get(H(0)), b'foo') def test_overwrite_in_same_transaction(self): """Test cache consistency2 """ self.repository.put(H(0), b'foo') self.repository.put(H(0), b'foo2') self.repository.commit() self.assert_equal(self.repository.get(H(0)), b'foo2') def test_single_kind_transactions(self): # put self.repository.put(H(0), b'foo') self.repository.commit() self.repository.close() # replace self.repository = self.open() with self.repository: self.repository.put(H(0), b'bar') self.repository.commit() # delete self.repository = self.open() with self.repository: self.repository.delete(H(0)) self.repository.commit() def test_list(self): for x in range(100): self.repository.put(H(x), b'SOMEDATA') all = self.repository.list() self.assert_equal(len(all), 100) first_half = self.repository.list(limit=50) self.assert_equal(len(first_half), 50) self.assert_equal(first_half, all[:50]) second_half = self.repository.list(marker=first_half[-1]) self.assert_equal(len(second_half), 50) self.assert_equal(second_half, all[50:]) self.assert_equal(len(self.repository.list(limit=50)), 50) def test_max_data_size(self): max_data = b'x' * MAX_DATA_SIZE self.repository.put(H(0), max_data) self.assert_equal(self.repository.get(H(0)), max_data) self.assert_raises(IntegrityError, lambda: self.repository.put(H(1), max_data + b'x')) class RepositoryCommitTestCase(RepositoryTestCaseBase): def add_keys(self): self.repository.put(H(0), b'foo') self.repository.put(H(1), b'bar') self.repository.put(H(3), b'bar') self.repository.commit() self.repository.put(H(1), b'bar2') self.repository.put(H(2), b'boo') self.repository.delete(H(3)) def test_uncommitted_garbage(self): # uncommitted garbage should be no problem, it is cleaned up automatically. # we just have to be careful with invalidation of cached FDs in LoggedIO. self.repository.put(H(0), b'foo') self.repository.commit() # write some crap to a uncommitted segment file last_segment = self.repository.io.get_latest_segment() with open(self.repository.io.segment_filename(last_segment + 1), 'wb') as f: f.write(MAGIC + b'crapcrapcrap') self.repository.close() # usually, opening the repo and starting a transaction should trigger a cleanup. self.repository = self.open() with self.repository: # the next 2 lines get the FD of the crap segment file cached: segment = self.repository.io.get_latest_segment() self.repository.io.get_fd(segment) # when the put triggers the start of a transaction, crap should be cleaned up: self.repository.put(H(0), b'bar') # this may trigger compact_segments() self.repository.commit() # the point here is that nothing blows up with an exception. def test_replay_of_missing_index(self): self.add_keys() for name in os.listdir(self.repository.path): if name.startswith('index.'): os.unlink(os.path.join(self.repository.path, name)) self.reopen() with self.repository: self.assert_equal(len(self.repository), 3) self.assert_equal(self.repository.check(), True) def test_crash_before_compact_segments(self): self.add_keys() self.repository.compact_segments = None try: self.repository.commit() except TypeError: pass self.reopen() with self.repository: self.assert_equal(len(self.repository), 3) self.assert_equal(self.repository.check(), True) def test_crash_before_write_index(self): self.add_keys() self.repository.write_index = None try: self.repository.commit() except TypeError: pass self.reopen() with self.repository: self.assert_equal(len(self.repository), 3) self.assert_equal(self.repository.check(), True) def test_replay_lock_upgrade_old(self): self.add_keys() for name in os.listdir(self.repository.path): if name.startswith('index.'): os.unlink(os.path.join(self.repository.path, name)) with patch.object(Lock, 'upgrade', side_effect=LockFailed) as upgrade: self.reopen(exclusive=None) # simulate old client that always does lock upgrades with self.repository: # the repo is only locked by a shared read lock, but to replay segments, # we need an exclusive write lock - check if the lock gets upgraded. self.assert_raises(LockFailed, lambda: len(self.repository)) upgrade.assert_called_once_with() def test_replay_lock_upgrade(self): self.add_keys() for name in os.listdir(self.repository.path): if name.startswith('index.'): os.unlink(os.path.join(self.repository.path, name)) with patch.object(Lock, 'upgrade', side_effect=LockFailed) as upgrade: self.reopen(exclusive=False) # current client usually does not do lock upgrade, except for replay with self.repository: # the repo is only locked by a shared read lock, but to replay segments, # we need an exclusive write lock - check if the lock gets upgraded. self.assert_raises(LockFailed, lambda: len(self.repository)) upgrade.assert_called_once_with() def test_crash_before_deleting_compacted_segments(self): self.add_keys() self.repository.io.delete_segment = None try: self.repository.commit() except TypeError: pass self.reopen() with self.repository: self.assert_equal(len(self.repository), 3) self.assert_equal(self.repository.check(), True) self.assert_equal(len(self.repository), 3) def test_ignores_commit_tag_in_data(self): self.repository.put(H(0), LoggedIO.COMMIT) self.reopen() with self.repository: io = self.repository.io assert not io.is_committed_segment(io.get_latest_segment()) def test_moved_deletes_are_tracked(self): self.repository.put(H(1), b'1') self.repository.put(H(2), b'2') self.repository.commit() self.repository.delete(H(1)) self.repository.commit() last_segment = self.repository.io.get_latest_segment() num_deletes = 0 for tag, key, offset, data in self.repository.io.iter_objects(last_segment, include_data=True): if tag == TAG_DELETE: assert key == H(1) num_deletes += 1 assert num_deletes == 1 assert last_segment in self.repository.compact self.repository.put(H(3), b'3') self.repository.commit() assert last_segment not in self.repository.compact assert not self.repository.io.segment_exists(last_segment) last_segment = self.repository.io.get_latest_segment() for tag, key, offset in self.repository.io.iter_objects(last_segment): assert tag != TAG_DELETE class RepositoryAppendOnlyTestCase(RepositoryTestCaseBase): def open(self, create=False): return Repository(os.path.join(self.tmppath, 'repository'), exclusive=True, create=create, append_only=True) def test_destroy_append_only(self): # Can't destroy append only repo (via the API) with self.assert_raises(ValueError): self.repository.destroy() assert self.repository.append_only def test_append_only(self): def segments_in_repository(): return len(list(self.repository.io.segment_iterator())) self.repository.put(H(0), b'foo') self.repository.commit() self.repository.append_only = False assert segments_in_repository() == 1 self.repository.put(H(0), b'foo') self.repository.commit() # normal: compact squashes the data together, only one segment assert segments_in_repository() == 1 self.repository.append_only = True assert segments_in_repository() == 1 self.repository.put(H(0), b'foo') self.repository.commit() # append only: does not compact, only new segments written assert segments_in_repository() == 2 class RepositoryCheckTestCase(RepositoryTestCaseBase): def list_indices(self): return [name for name in os.listdir(os.path.join(self.tmppath, 'repository')) if name.startswith('index.')] def check(self, repair=False, status=True): self.assert_equal(self.repository.check(repair=repair), status) # Make sure no tmp files are left behind self.assert_equal([name for name in os.listdir(os.path.join(self.tmppath, 'repository')) if 'tmp' in name], [], 'Found tmp files') def get_objects(self, *ids): for id_ in ids: self.repository.get(H(id_)) def add_objects(self, segments): for ids in segments: for id_ in ids: self.repository.put(H(id_), b'data') self.repository.commit() def get_head(self): return sorted(int(n) for n in os.listdir(os.path.join(self.tmppath, 'repository', 'data', '0')) if n.isdigit())[-1] def open_index(self): return NSIndex.read(os.path.join(self.tmppath, 'repository', 'index.{}'.format(self.get_head()))) def corrupt_object(self, id_): idx = self.open_index() segment, offset = idx[H(id_)] with open(os.path.join(self.tmppath, 'repository', 'data', '0', str(segment)), 'r+b') as fd: fd.seek(offset) fd.write(b'BOOM') def delete_segment(self, segment): os.unlink(os.path.join(self.tmppath, 'repository', 'data', '0', str(segment))) def delete_index(self): os.unlink(os.path.join(self.tmppath, 'repository', 'index.{}'.format(self.get_head()))) def rename_index(self, new_name): os.rename(os.path.join(self.tmppath, 'repository', 'index.{}'.format(self.get_head())), os.path.join(self.tmppath, 'repository', new_name)) def list_objects(self): return set(int(key) for key in self.repository.list()) def test_repair_corrupted_segment(self): self.add_objects([[1, 2, 3], [4, 5], [6]]) self.assert_equal(set([1, 2, 3, 4, 5, 6]), self.list_objects()) self.check(status=True) self.corrupt_object(5) self.assert_raises(IntegrityError, lambda: self.get_objects(5)) self.repository.rollback() # Make sure a regular check does not repair anything self.check(status=False) self.check(status=False) # Make sure a repair actually repairs the repo self.check(repair=True, status=True) self.get_objects(4) self.check(status=True) self.assert_equal(set([1, 2, 3, 4, 6]), self.list_objects()) def test_repair_missing_segment(self): self.add_objects([[1, 2, 3], [4, 5, 6]]) self.assert_equal(set([1, 2, 3, 4, 5, 6]), self.list_objects()) self.check(status=True) self.delete_segment(1) self.repository.rollback() self.check(repair=True, status=True) self.assert_equal(set([1, 2, 3]), self.list_objects()) def test_repair_missing_commit_segment(self): self.add_objects([[1, 2, 3], [4, 5, 6]]) self.delete_segment(1) self.assert_raises(Repository.ObjectNotFound, lambda: self.get_objects(4)) self.assert_equal(set([1, 2, 3]), self.list_objects()) def test_repair_corrupted_commit_segment(self): self.add_objects([[1, 2, 3], [4, 5, 6]]) with open(os.path.join(self.tmppath, 'repository', 'data', '0', '1'), 'r+b') as fd: fd.seek(-1, os.SEEK_END) fd.write(b'X') self.assert_raises(Repository.ObjectNotFound, lambda: self.get_objects(4)) self.check(status=True) self.get_objects(3) self.assert_equal(set([1, 2, 3]), self.list_objects()) def test_repair_no_commits(self): self.add_objects([[1, 2, 3]]) with open(os.path.join(self.tmppath, 'repository', 'data', '0', '0'), 'r+b') as fd: fd.seek(-1, os.SEEK_END) fd.write(b'X') self.assert_raises(Repository.CheckNeeded, lambda: self.get_objects(4)) self.check(status=False) self.check(status=False) self.assert_equal(self.list_indices(), ['index.0']) self.check(repair=True, status=True) self.assert_equal(self.list_indices(), ['index.1']) self.check(status=True) self.get_objects(3) self.assert_equal(set([1, 2, 3]), self.list_objects()) def test_repair_missing_index(self): self.add_objects([[1, 2, 3], [4, 5, 6]]) self.delete_index() self.check(status=True) self.get_objects(4) self.assert_equal(set([1, 2, 3, 4, 5, 6]), self.list_objects()) def test_repair_index_too_new(self): self.add_objects([[1, 2, 3], [4, 5, 6]]) self.assert_equal(self.list_indices(), ['index.1']) self.rename_index('index.100') self.check(status=True) self.assert_equal(self.list_indices(), ['index.1']) self.get_objects(4) self.assert_equal(set([1, 2, 3, 4, 5, 6]), self.list_objects()) def test_crash_before_compact(self): self.repository.put(H(0), b'data') self.repository.put(H(0), b'data2') # Simulate a crash before compact with patch.object(Repository, 'compact_segments') as compact: self.repository.commit() compact.assert_called_once_with(save_space=False) self.reopen() with self.repository: self.check(repair=True) self.assert_equal(self.repository.get(H(0)), b'data2') class RemoteRepositoryTestCase(RepositoryTestCase): def open(self, create=False): return RemoteRepository(Location('__testsuite__:' + os.path.join(self.tmppath, 'repository')), exclusive=True, create=create) def test_invalid_rpc(self): self.assert_raises(InvalidRPCMethod, lambda: self.repository.call('__init__', None)) def test_ssh_cmd(self): assert self.repository.ssh_cmd(Location('example.com:foo')) == ['ssh', 'example.com'] assert self.repository.ssh_cmd(Location('ssh://example.com/foo')) == ['ssh', 'example.com'] assert self.repository.ssh_cmd(Location('ssh://user@example.com/foo')) == ['ssh', 'user@example.com'] assert self.repository.ssh_cmd(Location('ssh://user@example.com:1234/foo')) == ['ssh', '-p', '1234', 'user@example.com'] os.environ['BORG_RSH'] = 'ssh --foo' assert self.repository.ssh_cmd(Location('example.com:foo')) == ['ssh', '--foo', 'example.com'] def test_borg_cmd(self): class MockArgs: remote_path = 'borg' umask = 0o077 assert self.repository.borg_cmd(None, testing=True) == [sys.executable, '-m', 'borg.archiver', 'serve'] args = MockArgs() # XXX without next line we get spurious test fails when using pytest-xdist, root cause unknown: logging.getLogger().setLevel(logging.INFO) # note: test logger is on info log level, so --info gets added automagically assert self.repository.borg_cmd(args, testing=False) == ['borg', 'serve', '--umask=077', '--info'] args.remote_path = 'borg-0.28.2' assert self.repository.borg_cmd(args, testing=False) == ['borg-0.28.2', 'serve', '--umask=077', '--info'] class RemoteRepositoryCheckTestCase(RepositoryCheckTestCase): def open(self, create=False): return RemoteRepository(Location('__testsuite__:' + os.path.join(self.tmppath, 'repository')), exclusive=True, create=create) def test_crash_before_compact(self): # skip this test, we can't mock-patch a Repository class in another process! pass borgbackup-1.0.13/borg/testsuite/shellpattern.py0000644000175000017500000000521213431513133021727 0ustar useruser00000000000000import re import pytest from .. import shellpattern def check(path, pattern): compiled = re.compile(shellpattern.translate(pattern)) return bool(compiled.match(path)) @pytest.mark.parametrize("path, patterns", [ # Literal string ("foo/bar", ["foo/bar"]), ("foo\\bar", ["foo\\bar"]), # Non-ASCII ("foo/c/\u0152/e/bar", ["foo/*/\u0152/*/bar", "*/*/\u0152/*/*", "**/\u0152/*/*"]), ("\u00e4\u00f6\u00dc", ["???", "*", "\u00e4\u00f6\u00dc", "[\u00e4][\u00f6][\u00dc]"]), # Question mark ("foo", ["fo?"]), ("foo", ["f?o"]), ("foo", ["f??"]), ("foo", ["?oo"]), ("foo", ["?o?"]), ("foo", ["??o"]), ("foo", ["???"]), # Single asterisk ("", ["*"]), ("foo", ["*", "**", "***"]), ("foo", ["foo*"]), ("foobar", ["foo*"]), ("foobar", ["foo*bar"]), ("foobarbaz", ["foo*baz"]), ("bar", ["*bar"]), ("foobar", ["*bar"]), ("foo/bar", ["foo/*bar"]), ("foo/bar", ["foo/*ar"]), ("foo/bar", ["foo/*r"]), ("foo/bar", ["foo/*"]), ("foo/bar", ["foo*/bar"]), ("foo/bar", ["fo*/bar"]), ("foo/bar", ["f*/bar"]), ("foo/bar", ["*/bar"]), # Double asterisk (matches 0..n directory layers) ("foo/bar", ["foo/**/bar"]), ("foo/1/bar", ["foo/**/bar"]), ("foo/1/22/333/bar", ["foo/**/bar"]), ("foo/", ["foo/**/"]), ("foo/1/", ["foo/**/"]), ("foo/1/22/333/", ["foo/**/"]), ("bar", ["**/bar"]), ("1/bar", ["**/bar"]), ("1/22/333/bar", ["**/bar"]), ("foo/bar/baz", ["foo/**/*"]), # Set ("foo1", ["foo[12]"]), ("foo2", ["foo[12]"]), ("foo2/bar", ["foo[12]/*"]), ("f??f", ["f??f", "f[?][?]f"]), ("foo]", ["foo[]]"]), # Inverted set ("foo3", ["foo[!12]"]), ("foo^", ["foo[^!]"]), ("foo!", ["foo[^!]"]), ]) def test_match(path, patterns): for p in patterns: assert check(path, p) @pytest.mark.parametrize("path, patterns", [ ("", ["?", "[]"]), ("foo", ["foo?"]), ("foo", ["?foo"]), ("foo", ["f?oo"]), # do not match path separator ("foo/ar", ["foo?ar"]), # do not match/cross over os.path.sep ("foo/bar", ["*"]), ("foo/bar", ["foo*bar"]), ("foo/bar", ["foo*ar"]), ("foo/bar", ["fo*bar"]), ("foo/bar", ["fo*ar"]), # Double asterisk ("foobar", ["foo/**/bar"]), # Two asterisks without slash do not match directory separator ("foo/bar", ["**"]), # Double asterisk not matching filename ("foo/bar", ["**/"]), # Set ("foo3", ["foo[12]"]), # Inverted set ("foo1", ["foo[!12]"]), ("foo2", ["foo[!12]"]), ]) def test_mismatch(path, patterns): for p in patterns: assert not check(path, p) borgbackup-1.0.13/borg/testsuite/upgrader.py0000644000175000017500000001461713431513133021044 0ustar useruser00000000000000import os import tarfile import pytest from ..upgrader import AtticRepositoryUpgrader, AtticKeyfileKey from ..helpers import get_keys_dir from ..key import KeyfileKey from ..archiver import UMASK_DEFAULT from ..repository import Repository # tar with a repo and repo keyfile from attic ATTIC_TAR = os.path.join(os.path.dirname(__file__), 'attic.tar.gz') def untar(tarfname, path, what): """ extract tar archive to , all stuff starting with . return path to . """ def files(members): for tarinfo in members: if tarinfo.name.startswith(what): yield tarinfo with tarfile.open(tarfname, 'r') as tf: tf.extractall(path, members=files(tf)) return os.path.join(path, what) def repo_valid(path): """ utility function to check if borg can open a repository :param path: the path to the repository :returns: if borg can check the repository """ with Repository(str(path), exclusive=True, create=False) as repository: # can't check raises() because check() handles the error return repository.check() def key_valid(path): """ check that the new keyfile is alright :param path: the path to the key file :returns: if the file starts with the borg magic string """ keyfile = os.path.join(get_keys_dir(), os.path.basename(path)) with open(keyfile, 'r') as f: return f.read().startswith(KeyfileKey.FILE_ID) @pytest.fixture() def attic_repo(tmpdir): """ create an attic repo with some stuff in it :param tmpdir: path to the repository to be created :returns: path to attic repository """ # there is some stuff in that repo, copied from `RepositoryTestCase.test1` return untar(ATTIC_TAR, str(tmpdir), 'repo') @pytest.fixture(params=[True, False]) def inplace(request): return request.param def test_convert_segments(attic_repo, inplace): """test segment conversion this will load the given attic repository, list all the segments then convert them one at a time. we need to close the repo before conversion otherwise we have errors from borg :param attic_repo: a populated attic repository (fixture) """ repo_path = attic_repo # check should fail because of magic number assert not repo_valid(repo_path) repository = AtticRepositoryUpgrader(repo_path, create=False) with repository: segments = [filename for i, filename in repository.io.segment_iterator()] repository.convert_segments(segments, dryrun=False, inplace=inplace) repository.convert_cache(dryrun=False) assert repo_valid(repo_path) @pytest.fixture() def attic_key_file(tmpdir, monkeypatch): """ create an attic key file from the given repo, in the keys subdirectory of the given tmpdir :param tmpdir: a temporary directory (a builtin fixture) :returns: path to key file """ keys_dir = untar(ATTIC_TAR, str(tmpdir), 'keys') # we use the repo dir for the created keyfile, because we do # not want to clutter existing keyfiles monkeypatch.setenv('ATTIC_KEYS_DIR', keys_dir) # we use the same directory for the converted files, which # will clutter the previously created one, which we don't care # about anyways. in real runs, the original key will be retained. monkeypatch.setenv('BORG_KEYS_DIR', keys_dir) monkeypatch.setenv('ATTIC_PASSPHRASE', 'test') return os.path.join(keys_dir, 'repo') def test_keys(attic_repo, attic_key_file): """test key conversion test that we can convert the given key to a properly formatted borg key. assumes that the ATTIC_KEYS_DIR and BORG_KEYS_DIR have been properly populated by the attic_key_file fixture. :param attic_repo: path to an attic repository (fixture defined above) :param attic_key_file: path to an attic key file (fixture defined above) """ keyfile_path = attic_key_file assert not key_valid(keyfile_path) # not upgraded yet with AtticRepositoryUpgrader(attic_repo, create=False) as repository: keyfile = AtticKeyfileKey.find_key_file(repository) AtticRepositoryUpgrader.convert_keyfiles(keyfile, dryrun=False) assert key_valid(keyfile_path) def test_convert_all(attic_repo, attic_key_file, inplace): """test all conversion steps this runs everything. mostly redundant test, since everything is done above. yet we expect a NotImplementedError because we do not convert caches yet. :param attic_repo: path to an attic repository (fixture defined above) :param attic_key_file: path to an attic key file (fixture defined above) """ repo_path = attic_repo # check should fail because of magic number assert not repo_valid(repo_path) def stat_segment(path): return os.stat(os.path.join(path, 'data', '0', '0')) def first_inode(path): return stat_segment(path).st_ino orig_inode = first_inode(repo_path) with AtticRepositoryUpgrader(repo_path, create=False) as repository: # replicate command dispatch, partly os.umask(UMASK_DEFAULT) backup = repository.upgrade(dryrun=False, inplace=inplace) if inplace: assert backup is None assert first_inode(repository.path) == orig_inode else: assert backup assert first_inode(repository.path) != first_inode(backup) # i have seen cases where the copied tree has world-readable # permissions, which is wrong assert stat_segment(backup).st_mode & UMASK_DEFAULT == 0 assert key_valid(attic_key_file) assert repo_valid(repo_path) def test_hardlink(tmpdir, inplace): """test that we handle hard links properly that is, if we are in "inplace" mode, hardlinks should *not* change (ie. we write to the file directly, so we do not rewrite the whole file, and we do not re-create the file). if we are *not* in inplace mode, then the inode should change, as we are supposed to leave the original inode alone.""" a = str(tmpdir.join('a')) with open(a, 'wb') as tmp: tmp.write(b'aXXX') b = str(tmpdir.join('b')) os.link(a, b) AtticRepositoryUpgrader.header_replace(b, b'a', b'b', inplace=inplace) if not inplace: assert os.stat(a).st_ino != os.stat(b).st_ino else: assert os.stat(a).st_ino == os.stat(b).st_ino with open(b, 'rb') as tmp: assert tmp.read() == b'bXXX' borgbackup-1.0.13/borg/testsuite/xattr.py0000644000175000017500000000556513431513133020377 0ustar useruser00000000000000import os import tempfile import unittest import pytest from ..xattr import is_enabled, getxattr, setxattr, listxattr, buffer, split_lstring from . import BaseTestCase @unittest.skipUnless(is_enabled(), 'xattr not enabled on filesystem') class XattrTestCase(BaseTestCase): def setUp(self): self.tmpfile = tempfile.NamedTemporaryFile() self.symlink = self.tmpfile.name + '.symlink' os.symlink(self.tmpfile.name, self.symlink) def tearDown(self): os.unlink(self.symlink) def assert_equal_se(self, is_x, want_x): # check 2 xattr lists for equality, but ignore security.selinux attr is_x = set(is_x) - {'security.selinux'} want_x = set(want_x) self.assert_equal(is_x, want_x) def test(self): self.assert_equal_se(listxattr(self.tmpfile.name), []) self.assert_equal_se(listxattr(self.tmpfile.fileno()), []) self.assert_equal_se(listxattr(self.symlink), []) setxattr(self.tmpfile.name, 'user.foo', b'bar') setxattr(self.tmpfile.fileno(), 'user.bar', b'foo') setxattr(self.tmpfile.name, 'user.empty', None) self.assert_equal_se(listxattr(self.tmpfile.name), ['user.foo', 'user.bar', 'user.empty']) self.assert_equal_se(listxattr(self.tmpfile.fileno()), ['user.foo', 'user.bar', 'user.empty']) self.assert_equal_se(listxattr(self.symlink), ['user.foo', 'user.bar', 'user.empty']) self.assert_equal_se(listxattr(self.symlink, follow_symlinks=False), []) self.assert_equal(getxattr(self.tmpfile.name, 'user.foo'), b'bar') self.assert_equal(getxattr(self.tmpfile.fileno(), 'user.foo'), b'bar') self.assert_equal(getxattr(self.symlink, 'user.foo'), b'bar') self.assert_equal(getxattr(self.tmpfile.name, 'user.empty'), None) def test_listxattr_buffer_growth(self): # make it work even with ext4, which imposes rather low limits buffer.resize(size=64, init=True) # xattr raw key list will be size 9 * (10 + 1), which is > 64 keys = ['user.attr%d' % i for i in range(9)] for key in keys: setxattr(self.tmpfile.name, key, b'x') got_keys = listxattr(self.tmpfile.name) self.assert_equal_se(got_keys, keys) self.assert_equal(len(buffer), 128) def test_getxattr_buffer_growth(self): # make it work even with ext4, which imposes rather low limits buffer.resize(size=64, init=True) value = b'x' * 126 setxattr(self.tmpfile.name, 'user.big', value) got_value = getxattr(self.tmpfile.name, 'user.big') self.assert_equal(value, got_value) self.assert_equal(len(buffer), 128) @pytest.mark.parametrize('lstring, splitted', ( (b'', []), (b'\x00', [b'']), (b'\x01a', [b'a']), (b'\x01a\x02cd', [b'a', b'cd']), )) def test_split_lstring(lstring, splitted): assert split_lstring(lstring) == splitted borgbackup-1.0.13/borg/upgrader.py0000644000175000017500000003231013431513133017001 0ustar useruser00000000000000import datetime import os import shutil import time from .logger import create_logger logger = create_logger() from .helpers import get_keys_dir, get_cache_dir, ProgressIndicatorPercent, bin_to_hex from .locking import Lock from .repository import Repository, MAGIC from .key import KeyfileKey, KeyfileNotFoundError ATTIC_MAGIC = b'ATTICSEG' class AtticRepositoryUpgrader(Repository): def __init__(self, *args, **kw): kw['lock'] = False # do not create borg lock files (now) in attic repo super().__init__(*args, **kw) def upgrade(self, dryrun=True, inplace=False, progress=False): """convert an attic repository to a borg repository those are the files that need to be upgraded here, from most important to least important: segments, key files, and various caches, the latter being optional, as they will be rebuilt if missing. we nevertheless do the order in reverse, as we prefer to do the fast stuff first, to improve interactivity. """ with self: backup = None if not inplace: backup = '{}.upgrade-{:%Y-%m-%d-%H:%M:%S}'.format(self.path, datetime.datetime.now()) logger.info('making a hardlink copy in %s', backup) if not dryrun: shutil.copytree(self.path, backup, copy_function=os.link) logger.info("opening attic repository with borg and converting") # now lock the repo, after we have made the copy self.lock = Lock(os.path.join(self.path, 'lock'), exclusive=True, timeout=1.0).acquire() segments = [filename for i, filename in self.io.segment_iterator()] try: keyfile = self.find_attic_keyfile() except KeyfileNotFoundError: logger.warning("no key file found for repository") else: self.convert_keyfiles(keyfile, dryrun) # partial open: just hold on to the lock self.lock = Lock(os.path.join(self.path, 'lock'), exclusive=True).acquire() try: self.convert_cache(dryrun) self.convert_repo_index(dryrun=dryrun, inplace=inplace) self.convert_segments(segments, dryrun=dryrun, inplace=inplace, progress=progress) self.borg_readme() finally: self.lock.release() self.lock = None return backup def borg_readme(self): readme = os.path.join(self.path, 'README') os.remove(readme) with open(readme, 'w') as fd: fd.write('This is a Borg repository\n') @staticmethod def convert_segments(segments, dryrun=True, inplace=False, progress=False): """convert repository segments from attic to borg replacement pattern is `s/ATTICSEG/BORG_SEG/` in files in `$ATTIC_REPO/data/**`. luckily the magic string length didn't change so we can just replace the 8 first bytes of all regular files in there.""" logger.info("converting %d segments..." % len(segments)) segment_count = len(segments) pi = ProgressIndicatorPercent(total=segment_count, msg="Converting segments %3.0f%%", same_line=True) for i, filename in enumerate(segments): if progress: pi.show(i) if dryrun: time.sleep(0.001) else: AtticRepositoryUpgrader.header_replace(filename, ATTIC_MAGIC, MAGIC, inplace=inplace) if progress: pi.finish() @staticmethod def header_replace(filename, old_magic, new_magic, inplace=True): with open(filename, 'r+b') as segment: segment.seek(0) # only write if necessary if segment.read(len(old_magic)) == old_magic: if inplace: segment.seek(0) segment.write(new_magic) else: # rename the hardlink and rewrite the file. this works # because the file is still open. so even though the file # is renamed, we can still read it until it is closed. os.rename(filename, filename + '.tmp') with open(filename, 'wb') as new_segment: new_segment.write(new_magic) new_segment.write(segment.read()) # the little dance with the .tmp file is necessary # because Windows won't allow overwriting an open file. os.unlink(filename + '.tmp') def find_attic_keyfile(self): """find the attic keyfiles the keyfiles are loaded by `KeyfileKey.find_key_file()`. that finds the keys with the right identifier for the repo. this is expected to look into $HOME/.attic/keys or $ATTIC_KEYS_DIR for key files matching the given Borg repository. it is expected to raise an exception (KeyfileNotFoundError) if no key is found. whether that exception is from Borg or Attic is unclear. this is split in a separate function in case we want to use the attic code here directly, instead of our local implementation.""" return AtticKeyfileKey.find_key_file(self) @staticmethod def convert_keyfiles(keyfile, dryrun): """convert key files from attic to borg replacement pattern is `s/ATTIC KEY/BORG_KEY/` in `get_keys_dir()`, that is `$ATTIC_KEYS_DIR` or `$HOME/.attic/keys`, and moved to `$BORG_KEYS_DIR` or `$HOME/.config/borg/keys`. no need to decrypt to convert. we need to rewrite the whole key file because magic string length changed, but that's not a problem because the keyfiles are small (compared to, say, all the segments).""" logger.info("converting keyfile %s" % keyfile) with open(keyfile, 'r') as f: data = f.read() data = data.replace(AtticKeyfileKey.FILE_ID, KeyfileKey.FILE_ID, 1) keyfile = os.path.join(get_keys_dir(), os.path.basename(keyfile)) logger.info("writing borg keyfile to %s" % keyfile) if not dryrun: with open(keyfile, 'w') as f: f.write(data) def convert_repo_index(self, dryrun, inplace): """convert some repo files those are all hash indexes, so we need to `s/ATTICIDX/BORG_IDX/` in a few locations: * the repository index (in `$ATTIC_REPO/index.%d`, where `%d` is the `Repository.get_index_transaction_id()`), which we should probably update, with a lock, see `Repository.open()`, which i'm not sure we should use because it may write data on `Repository.close()`... """ transaction_id = self.get_index_transaction_id() if transaction_id is None: logger.warning('no index file found for repository %s' % self.path) else: index = os.path.join(self.path, 'index.%d' % transaction_id) logger.info("converting repo index %s" % index) if not dryrun: AtticRepositoryUpgrader.header_replace(index, b'ATTICIDX', b'BORG_IDX', inplace=inplace) def convert_cache(self, dryrun): """convert caches from attic to borg those are all hash indexes, so we need to `s/ATTICIDX/BORG_IDX/` in a few locations: * the `files` and `chunks` cache (in `$ATTIC_CACHE_DIR` or `$HOME/.cache/attic//`), which we could just drop, but if we'd want to convert, we could open it with the `Cache.open()`, edit in place and then `Cache.close()` to make sure we have locking right """ # copy of attic's get_cache_dir() attic_cache_dir = os.environ.get('ATTIC_CACHE_DIR', os.path.join(os.path.expanduser('~'), '.cache', 'attic')) attic_cache_dir = os.path.join(attic_cache_dir, bin_to_hex(self.id)) borg_cache_dir = os.path.join(get_cache_dir(), bin_to_hex(self.id)) def copy_cache_file(path): """copy the given attic cache path into the borg directory does nothing if dryrun is True. also expects attic_cache_dir and borg_cache_dir to be set in the parent scope, to the directories path including the repository identifier. :params path: the basename of the cache file to copy (example: "files" or "chunks") as a string :returns: the borg file that was created or None if no Attic cache file was found. """ attic_file = os.path.join(attic_cache_dir, path) if os.path.exists(attic_file): borg_file = os.path.join(borg_cache_dir, path) if os.path.exists(borg_file): logger.warning("borg cache file already exists in %s, not copying from Attic", borg_file) else: logger.info("copying attic cache file from %s to %s" % (attic_file, borg_file)) if not dryrun: shutil.copyfile(attic_file, borg_file) return borg_file else: logger.warning("no %s cache file found in %s" % (path, attic_file)) return None # XXX: untested, because generating cache files is a PITA, see # Archiver.do_create() for proof if os.path.exists(attic_cache_dir): if not os.path.exists(borg_cache_dir): os.makedirs(borg_cache_dir) # file that we don't have a header to convert, just copy for cache in ['config', 'files']: copy_cache_file(cache) # we need to convert the headers of those files, copy first for cache in ['chunks']: cache = copy_cache_file(cache) logger.info("converting cache %s" % cache) if not dryrun: AtticRepositoryUpgrader.header_replace(cache, b'ATTICIDX', b'BORG_IDX') class AtticKeyfileKey(KeyfileKey): """backwards compatible Attic key file parser""" FILE_ID = 'ATTIC KEY' # verbatim copy from attic @staticmethod def get_keys_dir(): """Determine where to repository keys and cache""" return os.environ.get('ATTIC_KEYS_DIR', os.path.join(os.path.expanduser('~'), '.attic', 'keys')) @classmethod def find_key_file(cls, repository): """copy of attic's `find_key_file`_ this has two small modifications: 1. it uses the above `get_keys_dir`_ instead of the global one, assumed to be borg's 2. it uses `repository.path`_ instead of `repository._location.canonical_path`_ because we can't assume the repository has been opened by the archiver yet """ get_keys_dir = cls.get_keys_dir id = bin_to_hex(repository.id) keys_dir = get_keys_dir() if not os.path.exists(keys_dir): raise KeyfileNotFoundError(repository.path, keys_dir) for name in os.listdir(keys_dir): filename = os.path.join(keys_dir, name) with open(filename, 'r') as fd: line = fd.readline().strip() if line and line.startswith(cls.FILE_ID) and line[10:] == id: return filename raise KeyfileNotFoundError(repository.path, keys_dir) class BorgRepositoryUpgrader(Repository): def upgrade(self, dryrun=True, inplace=False, progress=False): """convert an old borg repository to a current borg repository """ logger.info("converting borg 0.xx to borg current") with self: try: keyfile = self.find_borg0xx_keyfile() except KeyfileNotFoundError: logger.warning("no key file found for repository") else: self.move_keyfiles(keyfile, dryrun) def find_borg0xx_keyfile(self): return Borg0xxKeyfileKey.find_key_file(self) def move_keyfiles(self, keyfile, dryrun): filename = os.path.basename(keyfile) new_keyfile = os.path.join(get_keys_dir(), filename) try: os.rename(keyfile, new_keyfile) except FileExistsError: # likely the attic -> borg upgrader already put it in the final location pass class Borg0xxKeyfileKey(KeyfileKey): """backwards compatible borg 0.xx key file parser""" @staticmethod def get_keys_dir(): return os.environ.get('BORG_KEYS_DIR', os.path.join(os.path.expanduser('~'), '.borg', 'keys')) @classmethod def find_key_file(cls, repository): get_keys_dir = cls.get_keys_dir id = bin_to_hex(repository.id) keys_dir = get_keys_dir() if not os.path.exists(keys_dir): raise KeyfileNotFoundError(repository.path, keys_dir) for name in os.listdir(keys_dir): filename = os.path.join(keys_dir, name) with open(filename, 'r') as fd: line = fd.readline().strip() if line and line.startswith(cls.FILE_ID) and line[len(cls.FILE_ID) + 1:] == id: return filename raise KeyfileNotFoundError(repository.path, keys_dir) borgbackup-1.0.13/borg/xattr.py0000644000175000017500000003411313431513133016335 0ustar useruser00000000000000"""A basic extended attributes (xattr) implementation for Linux, FreeBSD and MacOS X.""" import errno import os import re import subprocess import sys import tempfile from ctypes import CDLL, create_string_buffer, c_ssize_t, c_size_t, c_char_p, c_int, c_uint32, get_errno from ctypes.util import find_library from distutils.version import LooseVersion from .helpers import Buffer, prepare_subprocess_env try: ENOATTR = errno.ENOATTR except AttributeError: # on some platforms, ENOATTR is missing, use ENODATA there ENOATTR = errno.ENODATA buffer = Buffer(create_string_buffer, limit=2**24) def is_enabled(path=None): """Determine if xattr is enabled on the filesystem """ with tempfile.NamedTemporaryFile(dir=path, prefix='borg-tmp') as fd: try: setxattr(fd.fileno(), 'user.name', b'value') except OSError: return False return getxattr(fd.fileno(), 'user.name') == b'value' def get_all(path, follow_symlinks=True): try: result = {} names = listxattr(path, follow_symlinks=follow_symlinks) for name in names: try: result[name] = getxattr(path, name, follow_symlinks=follow_symlinks) except OSError as e: # if we get ENOATTR, a race has happened: xattr names were deleted after list. # we just ignore the now missing ones. if you want consistency, do snapshots. if e.errno != ENOATTR: raise return result except OSError as e: if e.errno in (errno.ENOTSUP, errno.EPERM): return {} libc_name = find_library('c') if libc_name is None: # find_library didn't work, maybe we are on some minimal system that misses essential # tools used by find_library, like ldconfig, gcc/cc, objdump. # so we can only try some "usual" names for the C library: if sys.platform.startswith('linux'): libc_name = 'libc.so.6' elif sys.platform.startswith(('freebsd', 'netbsd')): libc_name = 'libc.so' elif sys.platform == 'darwin': libc_name = 'libc.dylib' else: msg = "Can't find C library. No fallback known. Try installing ldconfig, gcc/cc or objdump." print(msg, file=sys.stderr) # logger isn't initialized at this stage raise Exception(msg) # If we are running with fakeroot on Linux, then use the xattr functions of fakeroot. This is needed by # the 'test_extract_capabilities' test, but also allows xattrs to work with fakeroot on Linux in normal use. # TODO: Check whether fakeroot supports xattrs on all platforms supported below. # TODO: If that's the case then we can make Borg fakeroot-xattr-compatible on these as well. try: XATTR_FAKEROOT = False if sys.platform.startswith('linux'): LD_PRELOAD = os.environ.get('LD_PRELOAD', '') preloads = re.split("[ :]", LD_PRELOAD) for preload in preloads: if preload.startswith("libfakeroot"): env = prepare_subprocess_env(system=True) fakeroot_output = subprocess.check_output(['fakeroot', '-v'], env=env) fakeroot_version = LooseVersion(fakeroot_output.decode('ascii').split()[-1]) if fakeroot_version >= LooseVersion("1.20.2"): # 1.20.2 has been confirmed to have xattr support # 1.18.2 has been confirmed not to have xattr support # Versions in-between are unknown libc_name = preload XATTR_FAKEROOT = True break except: pass try: libc = CDLL(libc_name, use_errno=True) except OSError as e: msg = "Can't find C library [%s]. Try installing ldconfig, gcc/cc or objdump." % e raise Exception(msg) def split_string0(buf): """split a list of zero-terminated strings into python not-zero-terminated bytes""" return buf.split(b'\0')[:-1] def split_lstring(buf): """split a list of length-prefixed strings into python not-length-prefixed bytes""" result = [] mv = memoryview(buf) while mv: length = mv[0] result.append(bytes(mv[1:1 + length])) mv = mv[1 + length:] return result class BufferTooSmallError(Exception): """the buffer given to an xattr function was too small for the result.""" def _check(rv, path=None, detect_buffer_too_small=False): if rv < 0: e = get_errno() if detect_buffer_too_small and e == errno.ERANGE: # listxattr and getxattr signal with ERANGE that they need a bigger result buffer. # setxattr signals this way that e.g. a xattr key name is too long / inacceptable. raise BufferTooSmallError else: try: msg = os.strerror(e) except ValueError: msg = '' if isinstance(path, int): path = '' % path raise OSError(e, msg, path) if detect_buffer_too_small and rv >= len(buffer): # freebsd does not error with ERANGE if the buffer is too small, # it just fills the buffer, truncates and returns. # so, we play sure and just assume that result is truncated if # it happens to be a full buffer. raise BufferTooSmallError return rv def _listxattr_inner(func, path): if isinstance(path, str): path = os.fsencode(path) size = len(buffer) while True: buf = buffer.get(size) try: n = _check(func(path, buf, size), path, detect_buffer_too_small=True) except BufferTooSmallError: size *= 2 else: return n, buf.raw def _getxattr_inner(func, path, name): if isinstance(path, str): path = os.fsencode(path) name = os.fsencode(name) size = len(buffer) while True: buf = buffer.get(size) try: n = _check(func(path, name, buf, size), path, detect_buffer_too_small=True) except BufferTooSmallError: size *= 2 else: return n, buf.raw def _setxattr_inner(func, path, name, value): if isinstance(path, str): path = os.fsencode(path) name = os.fsencode(name) value = value and os.fsencode(value) size = len(value) if value else 0 _check(func(path, name, value, size), path, detect_buffer_too_small=False) if sys.platform.startswith('linux'): # pragma: linux only libc.listxattr.argtypes = (c_char_p, c_char_p, c_size_t) libc.listxattr.restype = c_ssize_t libc.llistxattr.argtypes = (c_char_p, c_char_p, c_size_t) libc.llistxattr.restype = c_ssize_t libc.flistxattr.argtypes = (c_int, c_char_p, c_size_t) libc.flistxattr.restype = c_ssize_t libc.setxattr.argtypes = (c_char_p, c_char_p, c_char_p, c_size_t, c_int) libc.setxattr.restype = c_int libc.lsetxattr.argtypes = (c_char_p, c_char_p, c_char_p, c_size_t, c_int) libc.lsetxattr.restype = c_int libc.fsetxattr.argtypes = (c_int, c_char_p, c_char_p, c_size_t, c_int) libc.fsetxattr.restype = c_int libc.getxattr.argtypes = (c_char_p, c_char_p, c_char_p, c_size_t) libc.getxattr.restype = c_ssize_t libc.lgetxattr.argtypes = (c_char_p, c_char_p, c_char_p, c_size_t) libc.lgetxattr.restype = c_ssize_t libc.fgetxattr.argtypes = (c_int, c_char_p, c_char_p, c_size_t) libc.fgetxattr.restype = c_ssize_t def listxattr(path, *, follow_symlinks=True): def func(path, buf, size): if isinstance(path, int): return libc.flistxattr(path, buf, size) else: if follow_symlinks: return libc.listxattr(path, buf, size) else: return libc.llistxattr(path, buf, size) n, buf = _listxattr_inner(func, path) return [os.fsdecode(name) for name in split_string0(buf[:n]) if name and not name.startswith(b'system.posix_acl_')] def getxattr(path, name, *, follow_symlinks=True): def func(path, name, buf, size): if isinstance(path, int): return libc.fgetxattr(path, name, buf, size) else: if follow_symlinks: return libc.getxattr(path, name, buf, size) else: return libc.lgetxattr(path, name, buf, size) n, buf = _getxattr_inner(func, path, name) return buf[:n] or None def setxattr(path, name, value, *, follow_symlinks=True): def func(path, name, value, size): flags = 0 if isinstance(path, int): return libc.fsetxattr(path, name, value, size, flags) else: if follow_symlinks: return libc.setxattr(path, name, value, size, flags) else: return libc.lsetxattr(path, name, value, size, flags) _setxattr_inner(func, path, name, value) elif sys.platform == 'darwin': # pragma: darwin only libc.listxattr.argtypes = (c_char_p, c_char_p, c_size_t, c_int) libc.listxattr.restype = c_ssize_t libc.flistxattr.argtypes = (c_int, c_char_p, c_size_t, c_int) libc.flistxattr.restype = c_ssize_t libc.setxattr.argtypes = (c_char_p, c_char_p, c_char_p, c_size_t, c_uint32, c_int) libc.setxattr.restype = c_int libc.fsetxattr.argtypes = (c_int, c_char_p, c_char_p, c_size_t, c_uint32, c_int) libc.fsetxattr.restype = c_int libc.getxattr.argtypes = (c_char_p, c_char_p, c_char_p, c_size_t, c_uint32, c_int) libc.getxattr.restype = c_ssize_t libc.fgetxattr.argtypes = (c_int, c_char_p, c_char_p, c_size_t, c_uint32, c_int) libc.fgetxattr.restype = c_ssize_t XATTR_NOFLAGS = 0x0000 XATTR_NOFOLLOW = 0x0001 def listxattr(path, *, follow_symlinks=True): def func(path, buf, size): if isinstance(path, int): return libc.flistxattr(path, buf, size, XATTR_NOFLAGS) else: if follow_symlinks: return libc.listxattr(path, buf, size, XATTR_NOFLAGS) else: return libc.listxattr(path, buf, size, XATTR_NOFOLLOW) n, buf = _listxattr_inner(func, path) return [os.fsdecode(name) for name in split_string0(buf[:n]) if name] def getxattr(path, name, *, follow_symlinks=True): def func(path, name, buf, size): if isinstance(path, int): return libc.fgetxattr(path, name, buf, size, 0, XATTR_NOFLAGS) else: if follow_symlinks: return libc.getxattr(path, name, buf, size, 0, XATTR_NOFLAGS) else: return libc.getxattr(path, name, buf, size, 0, XATTR_NOFOLLOW) n, buf = _getxattr_inner(func, path, name) return buf[:n] or None def setxattr(path, name, value, *, follow_symlinks=True): def func(path, name, value, size): if isinstance(path, int): return libc.fsetxattr(path, name, value, size, 0, XATTR_NOFLAGS) else: if follow_symlinks: return libc.setxattr(path, name, value, size, 0, XATTR_NOFLAGS) else: return libc.setxattr(path, name, value, size, 0, XATTR_NOFOLLOW) _setxattr_inner(func, path, name, value) elif sys.platform.startswith('freebsd'): # pragma: freebsd only libc.extattr_list_fd.argtypes = (c_int, c_int, c_char_p, c_size_t) libc.extattr_list_fd.restype = c_ssize_t libc.extattr_list_link.argtypes = (c_char_p, c_int, c_char_p, c_size_t) libc.extattr_list_link.restype = c_ssize_t libc.extattr_list_file.argtypes = (c_char_p, c_int, c_char_p, c_size_t) libc.extattr_list_file.restype = c_ssize_t libc.extattr_get_fd.argtypes = (c_int, c_int, c_char_p, c_char_p, c_size_t) libc.extattr_get_fd.restype = c_ssize_t libc.extattr_get_link.argtypes = (c_char_p, c_int, c_char_p, c_char_p, c_size_t) libc.extattr_get_link.restype = c_ssize_t libc.extattr_get_file.argtypes = (c_char_p, c_int, c_char_p, c_char_p, c_size_t) libc.extattr_get_file.restype = c_ssize_t libc.extattr_set_fd.argtypes = (c_int, c_int, c_char_p, c_char_p, c_size_t) libc.extattr_set_fd.restype = c_int libc.extattr_set_link.argtypes = (c_char_p, c_int, c_char_p, c_char_p, c_size_t) libc.extattr_set_link.restype = c_int libc.extattr_set_file.argtypes = (c_char_p, c_int, c_char_p, c_char_p, c_size_t) libc.extattr_set_file.restype = c_int ns = EXTATTR_NAMESPACE_USER = 0x0001 def listxattr(path, *, follow_symlinks=True): def func(path, buf, size): if isinstance(path, int): return libc.extattr_list_fd(path, ns, buf, size) else: if follow_symlinks: return libc.extattr_list_file(path, ns, buf, size) else: return libc.extattr_list_link(path, ns, buf, size) n, buf = _listxattr_inner(func, path) return [os.fsdecode(name) for name in split_lstring(buf[:n]) if name] def getxattr(path, name, *, follow_symlinks=True): def func(path, name, buf, size): if isinstance(path, int): return libc.extattr_get_fd(path, ns, name, buf, size) else: if follow_symlinks: return libc.extattr_get_file(path, ns, name, buf, size) else: return libc.extattr_get_link(path, ns, name, buf, size) n, buf = _getxattr_inner(func, path, name) return buf[:n] or None def setxattr(path, name, value, *, follow_symlinks=True): def func(path, name, value, size): if isinstance(path, int): return libc.extattr_set_fd(path, ns, name, value, size) else: if follow_symlinks: return libc.extattr_set_file(path, ns, name, value, size) else: return libc.extattr_set_link(path, ns, name, value, size) _setxattr_inner(func, path, name, value) else: # pragma: unknown platform only def listxattr(path, *, follow_symlinks=True): return [] def getxattr(path, name, *, follow_symlinks=True): pass def setxattr(path, name, value, *, follow_symlinks=True): pass borgbackup-1.0.13/borgbackup.egg-info/0000755000175000017500000000000013431514213017477 5ustar useruser00000000000000borgbackup-1.0.13/borgbackup.egg-info/PKG-INFO0000644000175000017500000002063313431514213020600 0ustar useruser00000000000000Metadata-Version: 2.1 Name: borgbackup Version: 1.0.13 Summary: Deduplicated, encrypted, authenticated and compressed backups Home-page: https://borgbackup.readthedocs.io/ Author: The Borg Collective (see AUTHORS file) Author-email: borgbackup@python.org License: BSD Description: What is BorgBackup? ------------------- BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets. See the `installation manual`_ or, if you have already downloaded Borg, ``docs/installation.rst`` to get started with Borg. .. _installation manual: https://borgbackup.readthedocs.org/en/stable/installation.html Main features ~~~~~~~~~~~~~ **Space efficient storage** Deduplication based on content-defined chunking is used to reduce the number of bytes stored: each file is split into a number of variable length chunks and only chunks that have never been seen before are added to the repository. A chunk is considered duplicate if its id_hash value is identical. A cryptographically strong hash or MAC function is used as id_hash, e.g. (hmac-)sha256. To deduplicate, all the chunks in the same repository are considered, no matter whether they come from different machines, from previous backups, from the same backup or even from the same single file. Compared to other deduplication approaches, this method does NOT depend on: * file/directory names staying the same: So you can move your stuff around without killing the deduplication, even between machines sharing a repo. * complete files or time stamps staying the same: If a big file changes a little, only a few new chunks need to be stored - this is great for VMs or raw disks. * The absolute position of a data chunk inside a file: Stuff may get shifted and will still be found by the deduplication algorithm. **Speed** * performance critical code (chunking, compression, encryption) is implemented in C/Cython * local caching of files/chunks index data * quick detection of unmodified files **Data encryption** All data can be protected using 256-bit AES encryption, data integrity and authenticity is verified using HMAC-SHA256. Data is encrypted clientside. **Compression** All data can be compressed by lz4 (super fast, low compression), zlib (medium speed and compression) or lzma (low speed, high compression). **Off-site backups** Borg can store data on any remote host accessible over SSH. If Borg is installed on the remote host, big performance gains can be achieved compared to using a network filesystem (sshfs, nfs, ...). **Backups mountable as filesystems** Backup archives are mountable as userspace filesystems for easy interactive backup examination and restores (e.g. by using a regular file manager). **Easy installation on multiple platforms** We offer single-file binaries that do not require installing anything - you can just run them on these platforms: * Linux * Mac OS X * FreeBSD * OpenBSD and NetBSD (no xattrs/ACLs support or binaries yet) * Cygwin (not supported, no binaries yet) * Linux Subsystem of Windows 10 (not supported) **Free and Open Source Software** * security and functionality can be audited independently * licensed under the BSD (3-clause) license, see `License`_ for the complete license Easy to use ~~~~~~~~~~~ Initialize a new backup repository and create a backup archive:: $ borg init /path/to/repo $ borg create /path/to/repo::Saturday1 ~/Documents Now doing another backup, just to show off the great deduplication:: $ borg create -v --stats /path/to/repo::Saturday2 ~/Documents ----------------------------------------------------------------------------- Archive name: Saturday2 Archive fingerprint: 622b7c53c... Time (start): Sat, 2016-02-27 14:48:13 Time (end): Sat, 2016-02-27 14:48:14 Duration: 0.88 seconds Number of files: 163 ----------------------------------------------------------------------------- Original size Compressed size Deduplicated size This archive: 6.85 MB 6.85 MB 30.79 kB <-- ! All archives: 13.69 MB 13.71 MB 6.88 MB Unique chunks Total chunks Chunk index: 167 330 ----------------------------------------------------------------------------- For a graphical frontend refer to our complementary project `BorgWeb `_. Helping, Donations and Bounties ------------------------------- Your help is always welcome! Spread the word, give feedback, help with documentation, testing or development. You can also give monetary support to the project, see there for details: https://borgbackup.readthedocs.io/en/stable/support.html#bounties-and-fundraisers Links ----- * `Main Web Site `_ * `Releases `_, `PyPI packages `_ and `ChangeLog `_ * `GitHub `_ and `Issue Tracker `_. * `Web-Chat (IRC) `_ and `Mailing List `_ * `License `_ * `Security contact `_ Compatibility notes ------------------- EXPECT THAT WE WILL BREAK COMPATIBILITY REPEATEDLY WHEN MAJOR RELEASE NUMBER CHANGES (like when going from 0.x.y to 1.0.0 or from 1.x.y to 2.0.0). NOT RELEASED DEVELOPMENT VERSIONS HAVE UNKNOWN COMPATIBILITY PROPERTIES. THIS IS SOFTWARE IN DEVELOPMENT, DECIDE YOURSELF WHETHER IT FITS YOUR NEEDS. Security issues should be reported to the `Security contact`_ (or see ``docs/suppport.rst`` in the source distribution). Platform: Linux Platform: MacOS X Platform: FreeBSD Platform: OpenBSD Platform: NetBSD Classifier: Development Status :: 4 - Beta Classifier: Environment :: Console Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: POSIX :: BSD :: FreeBSD Classifier: Operating System :: POSIX :: BSD :: OpenBSD Classifier: Operating System :: POSIX :: BSD :: NetBSD Classifier: Operating System :: MacOS :: MacOS X Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Topic :: Security :: Cryptography Classifier: Topic :: System :: Archiving :: Backup Provides-Extra: fuse borgbackup-1.0.13/borgbackup.egg-info/SOURCES.txt0000644000175000017500000000534113431514213021366 0ustar useruser00000000000000AUTHORS CHANGES.rst LICENSE MANIFEST.in README.rst conftest.py setup.cfg setup.py tox.ini borg/__init__.py borg/__main__.py borg/_chunker.c borg/_hashindex.c borg/_version.py borg/archive.py borg/archiver.py borg/cache.py borg/chunker.c borg/chunker.pyx borg/compress.c borg/compress.pyx borg/crypto.c borg/crypto.pyx borg/fuse.py borg/hashindex.c borg/hashindex.pyx borg/helpers.py borg/key.py borg/keymanager.py borg/locking.py borg/logger.py borg/lrucache.py borg/paperkey.html borg/platform.py borg/platform_darwin.c borg/platform_darwin.pyx borg/platform_freebsd.c borg/platform_freebsd.pyx borg/platform_linux.c borg/platform_linux.pyx borg/remote.py borg/repository.py borg/shellpattern.py borg/upgrader.py borg/xattr.py borg/testsuite/__init__.py borg/testsuite/archive.py borg/testsuite/archiver.py borg/testsuite/attic.tar.gz borg/testsuite/benchmark.py borg/testsuite/chunker.py borg/testsuite/compress.py borg/testsuite/crypto.py borg/testsuite/hashindex.py borg/testsuite/helpers.py borg/testsuite/key.py borg/testsuite/locking.py borg/testsuite/logger.py borg/testsuite/lrucache.py borg/testsuite/platform.py borg/testsuite/repository.py borg/testsuite/shellpattern.py borg/testsuite/upgrader.py borg/testsuite/xattr.py borgbackup.egg-info/PKG-INFO borgbackup.egg-info/SOURCES.txt borgbackup.egg-info/dependency_links.txt borgbackup.egg-info/entry_points.txt borgbackup.egg-info/requires.txt borgbackup.egg-info/top_level.txt docs/Makefile docs/authors.rst docs/book.rst docs/changes.rst docs/conf.py docs/deployment.rst docs/development.rst docs/faq.rst docs/global.rst.inc docs/index.rst docs/installation.rst docs/internals.rst docs/introduction.rst docs/quickstart.rst docs/resources.rst docs/support.rst docs/usage.rst docs/_static/favicon.ico docs/_static/logo.png docs/_static/logo_font.txt docs/borg_theme/css/borg.css docs/misc/create_chunker-params.txt docs/misc/internals-picture.txt docs/misc/prune-example.txt docs/misc/asciinema/install_and_basics.json docs/misc/asciinema/install_and_basics.txt docs/usage/break-lock.rst.inc docs/usage/change-passphrase.rst.inc docs/usage/check.rst.inc docs/usage/create.rst.inc docs/usage/delete.rst.inc docs/usage/extract.rst.inc docs/usage/help.rst.inc docs/usage/info.rst.inc docs/usage/init.rst.inc docs/usage/key_export.rst.inc docs/usage/key_import.rst.inc docs/usage/list.rst.inc docs/usage/migrate-to-repokey.rst.inc docs/usage/mount.rst.inc docs/usage/prune.rst.inc docs/usage/rename.rst.inc docs/usage/serve.rst.inc docs/usage/umount.rst.inc docs/usage/upgrade.rst.inc requirements.d/development.lock.txt requirements.d/development.txt requirements.d/docs.txt requirements.d/fuse.txt scripts/borg.exe.spec scripts/glibc_check.py scripts/hash_sizes.py scripts/sdist-sign scripts/sign-binaries scripts/upload-pypiborgbackup-1.0.13/borgbackup.egg-info/dependency_links.txt0000644000175000017500000000000113431514213023545 0ustar useruser00000000000000 borgbackup-1.0.13/borgbackup.egg-info/entry_points.txt0000644000175000017500000000005513431514213022775 0ustar useruser00000000000000[console_scripts] borg = borg.archiver:main borgbackup-1.0.13/borgbackup.egg-info/requires.txt0000644000175000017500000000006013431514213022073 0ustar useruser00000000000000msgpack-python<0.5.0,>=0.4.6 [fuse] llfuse<2.0 borgbackup-1.0.13/borgbackup.egg-info/top_level.txt0000644000175000017500000000000513431514213022224 0ustar useruser00000000000000borg borgbackup-1.0.13/conftest.py0000644000175000017500000000413113431513133016064 0ustar useruser00000000000000import os import os.path import sys import pytest # needed to get pretty assertion failures in unit tests: if hasattr(pytest, 'register_assert_rewrite'): pytest.register_assert_rewrite('borg.testsuite') # This is a hack to fix path problems because "borg" (the package) is in the source root. # When importing the conftest an "import borg" can incorrectly import the borg from the # source root instead of the one installed in the environment. # The workaround is to remove entries pointing there from the path and check whether "borg" # is still importable. If it is not, then it has not been installed in the environment # and the entries are put back. original_path = list(sys.path) for entry in original_path: if entry == '' or entry == os.path.dirname(__file__): sys.path.remove(entry) try: import borg except ImportError: sys.path = original_path # note: if anything from borg needs to be imported, do it below this line. import borg.cache @pytest.fixture(autouse=True) def clean_env(tmpdir_factory, monkeypatch): # avoid that we access / modify the user's normal .config / .cache directory: monkeypatch.setenv('XDG_CONFIG_HOME', tmpdir_factory.mktemp('xdg-config-home')) monkeypatch.setenv('XDG_CACHE_HOME', tmpdir_factory.mktemp('xdg-cache-home')) # also avoid to use anything from the outside environment: keys = [key for key in os.environ if key.startswith('BORG_')] for key in keys: monkeypatch.delenv(key, raising=False) class DefaultPatches: def __init__(self, request): self.org_cache_wipe_cache = borg.cache.Cache.wipe_cache def wipe_should_not_be_called(*a, **kw): raise AssertionError("Cache wipe was triggered, if this is part of the test add @pytest.mark.allow_cache_wipe") if 'allow_cache_wipe' not in request.keywords: borg.cache.Cache.wipe_cache = wipe_should_not_be_called request.addfinalizer(self.undo) def undo(self): borg.cache.Cache.wipe_cache = self.org_cache_wipe_cache @pytest.fixture(autouse=True) def default_patches(request): return DefaultPatches(request) borgbackup-1.0.13/docs/0000755000175000017500000000000013431514213014616 5ustar useruser00000000000000borgbackup-1.0.13/docs/Makefile0000644000175000017500000001074613431513133016266 0ustar useruser00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " text to make text files" @echo " man to make manual pages" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/borg.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/borg.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/borg" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/borg" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." make -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." borgbackup-1.0.13/docs/_static/0000755000175000017500000000000013431514213016244 5ustar useruser00000000000000borgbackup-1.0.13/docs/_static/favicon.ico0000644000175000017500000011401613431513133020370 0ustar useruser00000000000000RR m6RRFmRR"&u(R PP aaaa PPKL<M>>>>>>?BuL0i9iuL0i9iuL0i9iCuL%|cs] cvu"uL4uL4uL4uL<>>>4B\\\\\3uL0 4uL064uL0-4uL`* 3e`Q=Lf9CflFLg9 glHLh9 hlJL%|L0L0L0PP aaaa PP?@@@@@@?(R(R "%*-03469<=>?BCFHJKLMQ\]`cefghilsuv|S\\\\\\\\' R\\\\\\\\\\(\\\\\\\\\\\\) !Q\\\\\"G\\\\\6(\\\\\U\\\\\\\;3\\\\N\\\Q9\\\\\D3\\\UG\\\\\6(\\\\\U\\\\\\\\;3\\\\\N\\\\P 9\\\\\D3\\\\\UG\\\\\6(\\\\\U\\\\\\\\\;3\\\\\\N\\\\\P 9\\\\\D3\\\\\\\#G\\\\\6(\\\\\UEGGGW\\\\\8.\\\\\T5,>\\\\\N 9\\\\\D.\\\\\L76G\\\\\6(\\\\\U<\\\\\PD\\\\\@\\\\\\9\\\\\DD\\\\\<G\\\\\6(\\\\\U<\\\\\PD\\\\\@\\\\\\9\\\\\DD\\\\\<G\\\\\6(\\\\\U<\\\\\PD\\\\\@\\\\\\9\\\\\DD\\\\\<G\\\\\6(\\\\\U  A\\\\\?D\\\\\@\\\\\\9\\\\\D"+++++D\\\\\<G\\\\\6(\\\\\U\\\\\\\\\AD\\\\\@\\\\\\9\\\\\DF\\\\\<D\\\\\<G\\\\\6(\\\\\U\\\\\\\\\D\\\\\@\\\\\\9\\\\\DF\\\\\<D\\\\\<G\\\\\6(\\\\\U\\\\\\\\\XD\\\\\@\\\\\\9\\\\\DF\\\\\<D\\\\\<G\\\\\6(\\\\\U<\\\\\M-\\\\\YH=R\\\\\L 9\\\\\DIJZ\\\\\/-\\\\\<*JJ[\\\\\(\\\\\U<\\\\\P0\\\\\\N\\\\\N 9\\\\\D#\\\\\\\@0\\\\<4\\\\\\\$(\\\\\U<\\\\\P1\\\\\N\\\\O 9\\\\\DB\\\\\K 1\\\<4\\\\\\%(\\\\\U<\\\\\P2\\\\N\\\Q 9\\\\\DC\\\V2\\<4\\\\\&(\\\\\UEGGGW\\\\\8(\\\\\U\\\\\\\\\:(\\\\\U\\\\\\\\:(\\\\\U\\\\\\\:borgbackup-1.0.13/docs/_static/logo.png0000644000175000017500000000314513431513133017715 0ustar useruser00000000000000PNG  IHDRd pHYs  IDATxkec QZd Dd`DH ERCSJ3+ 2B*%BAz@VH=3;Ϟggf?yfyhARN- $,KXd%`K2 d%`IX $,K2d%`KJIcF33 0hΩc&_Yl 2`ŖNSw* ..p.b /wt p[> N,M&bpkj|b0"rG ^^@8VK?ΰͽ߻IVLY7m} y:Q}v~h"XVr>%8%7 . p076\<6L 4cA;]ϥR.#ЬV"T>CfD3x)N ߬2y Y!\ 7x@4x2c9y c&\W pqvs[ n7FuOϹͣH>g+o^!l:\ J>bq8W =?xp}Έ}N# :_IѳПpbC ,.ɧFЌnnq9Uzw D3d-p9<$vUՓ0oK=8IpHIi; pZ~A8-MXœԒNH3ᒵ2F89xD!\O,2ON'jrb%xM%f:aR ZJ9t|kN8w'>Jќr&wZ $,KXd%`G?$޻[vGs;=2o)'0Q_{ >; \L 8Ͳ6'=InnG5L>8GshUH w_W9OWtՃcsr0w[m"\~x7f"ϯ-eڦwvXg"k-0RKvbp(٭H} p폇Wksjom ΡwffPN8dkk/]*G,K}v߱\C~bP[U1H6Xix+Rq޼ޤ<_+ $+cn)qn}$uoi!.>?TFv9CJ $,[ilN͹_4>rYhկ`90@봂}N a { color: rgba(255, 255, 255, 0.5); } .wy-side-nav-search > div.version { color: rgba(255, 255, 255, 0.5); } borgbackup-1.0.13/docs/changes.rst0000644000175000017500000024516413431513133016774 0ustar useruser00000000000000Important notes =============== This section provides information about security and corruption issues. .. _tam_vuln: Pre-1.0.9 manifest spoofing vulnerability (CVE-2016-10099) ---------------------------------------------------------- A flaw in the cryptographic authentication scheme in Borg allowed an attacker to spoof the manifest. The attack requires an attacker to be able to 1. insert files (with no additional headers) into backups 2. gain write access to the repository This vulnerability does not disclose plaintext to the attacker, nor does it affect the authenticity of existing archives. The vulnerability allows an attacker to create a spoofed manifest (the list of archives). Creating plausible fake archives may be feasible for small archives, but is unlikely for large archives. The fix adds a separate authentication tag to the manifest. For compatibility with prior versions this authentication tag is *not* required by default for existing repositories. Repositories created with 1.0.9 and later require it. Steps you should take: 1. Upgrade all clients to 1.0.9 or later. 2. Run ``borg upgrade --tam `` *on every client* for *each* repository. 3. This will list all archives, including archive IDs, for easy comparison with your logs. 4. Done. Prior versions can access and modify repositories with this measure enabled, however, to 1.0.9 or later their modifications are indiscernible from an attack and will raise an error until the below procedure is followed. We are aware that this can be be annoying in some circumstances, but don't see a way to fix the vulnerability otherwise. In case a version prior to 1.0.9 is used to modify a repository where above procedure was completed, and now you get an error message from other clients: 1. ``borg upgrade --tam --force `` once with *any* client suffices. This attack is mitigated by: - Noting/logging ``borg list``, ``borg info``, or ``borg create --stats``, which contain the archive IDs. We are not aware of others having discovered, disclosed or exploited this vulnerability. Vulnerability time line: * 2016-11-14: Vulnerability and fix discovered during review of cryptography by Marian Beermann (@enkore) * 2016-11-20: First patch * 2016-12-20: Released fixed version 1.0.9 * 2017-01-02: CVE was assigned * 2017-01-15: Released fixed version 1.1.0b3 (fix was previously only available from source) .. _attic013_check_corruption: Pre-1.0.9 potential data loss ----------------------------- If you have archives in your repository that were made with attic <= 0.13 (and later migrated to borg), running borg check would report errors in these archives. See issue #1837. The reason for this is a invalid (and useless) metadata key that was always added due to a bug in these old attic versions. If you run borg check --repair, things escalate quickly: all archive items with invalid metadata will be killed. Due to that attic bug, that means all items in all archives made with these old attic versions. Pre-1.0.4 potential repo corruption ----------------------------------- Some external errors (like network or disk I/O errors) could lead to corruption of the backup repository due to issue #1138. A sign that this happened is if "E" status was reported for a file that can not be explained by problems with the source file. If you still have logs from "borg create -v --list", you can check for "E" status. Here is what could cause corruption and what you can do now: 1) I/O errors (e.g. repo disk errors) while writing data to repo. This could lead to corrupted segment files. Fix:: # check for corrupt chunks / segments: borg check -v --repository-only REPO # repair the repo: borg check -v --repository-only --repair REPO # make sure everything is fixed: borg check -v --repository-only REPO 2) Unreliable network / unreliable connection to the repo. This could lead to archive metadata corruption. Fix:: # check for corrupt archives: borg check -v --archives-only REPO # delete the corrupt archives: borg delete --force REPO::CORRUPT_ARCHIVE # make sure everything is fixed: borg check -v --archives-only REPO 3) In case you want to do more intensive checking. The best check that everything is ok is to run a dry-run extraction:: borg extract -v --dry-run REPO::ARCHIVE .. _changelog: Changelog ========= Version 1.0.13 (2019-02-15) --------------------------- Please note: this is very likely the last 1.0.x release, please upgrade to 1.1.x. Bug fixes: - security fix: configure FUSE with "default_permissions", #3903. "default_permissions" is now enforced by borg by default to let the kernel check uid/gid/mode based permissions. "ignore_permissions" can be given to not enforce "default_permissions". - xattrs: fix borg exception handling on ENOSPC error, #3808. New features: - Read a passphrase from a file descriptor specified in the BORG_PASSPHRASE_FD environment variable. Other changes: - acl platform code: fix acl set return type - xattr: - add linux {list,get,set}xattr ctypes prototypes - fix darwin flistxattr ctypes prototype - testing / travis-ci: - fix the homebrew 1.9 issues on travis-ci, #4254 - travis OS X: use xcode 8.3 (not broken) - tox.ini: lock requirements - unbreak 1.0-maint on travis, fixes #4123 - vagrant: - misc. fixes - FUSE for macOS: upgrade 3.7.1 to 3.8.3 - Python: upgrade 3.5.5 to 3.5.6 - docs: - Update installation instructions for macOS - update release workflow using twine (docs, scripts), #4213 Version 1.0.12 (2018-04-08) --------------------------- Bug fixes: - repository: cleanup/write: invalidate cached FDs, tests - serve: fix exitcode, #2910 - extract: set bsdflags last (include immutable flag), #3263 - create --timestamp: set start time, #2957 - create: show excluded dir with "x" for tagged dirs / caches, #3189 - migrate locks to child PID when daemonize is used - Buffer: fix wrong thread-local storage use, #2951 - fix detection of non-local path, #3108 - fix LDLP restoration for subprocesses, #3077 - fix subprocess environments (xattr module's fakeroot version check, borg umount, BORG_PASSCOMMAND), #3050 - remote: deal with partial lines, #2637 - get rid of datetime.isoformat, use safe parse_timestamp to parse timestamps, #2994 - build: do .h file content checks in binary mode, fixes build issue for non-ascii header files on pure-ascii locale platforms, #3544 #3639 - remove platform.uname() call which caused library mismatch issues, #3732 - add exception handler around deprecated platform.linux_distribution() call Other changes: - require msgpack-python >= 0.4.6 and < 0.5.0, see #3753 - add parens for C preprocessor macro argument usages (did not cause malfunction) - ignore corrupt files cache, #2939 - replace "modulo" with "if" to check for wraparound in hashmap - keymanager: don't depend on optional readline module, #2980 - exclude broken pytest 3.3.0 release - exclude broken Cython 0.27(.0) release, #3066 - flake8: add some ignores - docs: - create: document exclusion through nodump - document good and problematic option placements, fix examples, #3356 - update docs about hardlinked symlinks limitation - faq: we do not implement futile attempts of ETA / progress displays - simplified rate limiting wrapper in FAQ - twitter account @borgbackup, #2948 - add note about metadata dedup and --no[ac]time, #2518 - change-passphrase only changes the passphrase, #2990 - clarify encrypted key format for borg key export, #3296 - document sshfs rename workaround, #3315 - update release checklist about security fixes - docs about how to verify a signed release, #3634 - chunk seed is generated per /repository/ - vagrant: - use FUSE for macOS 3.7.1 to build the macOS binary - use python 3.5.5 to build the binaries - add exe location to PATH when we build an exe - use https pypi url for wheezy - netbsd: bash is already installed - netbsd: fix netbsd version in PKG_PATH - use self-made FreeBSD 10.3 box, #3022 - backport fs_init (including related updates) from 1.1 - the boxcutter wheezy boxes are 404, use local ones - travis: - don't perform full Travis build on docs-only changes, #2531 - only short-circuit docs-only changes for pull requests Version 1.0.11 (2017-07-21) --------------------------- Bug fixes: - use limited unpacker for outer key (security precaution), #2174 - fix paperkey import bug Other changes: - change --checkpoint-interval default from 600s to 1800s, #2841. this improves efficiency for big repositories a lot. - docs: fix OpenSUSE command and add OpenSUSE section - tests: add tests for split_lstring and paperkey - vagrant: - fix openbsd shell - backport cpu/ram setup from master - add stretch64 VM Version 1.0.11rc1 (2017-06-27) ------------------------------ Bug fixes: - performance: rebuild hashtable if we have too few empty buckets, #2246. this fixes some sporadic, but severe performance breakdowns. - Archive: allocate zeros when needed, #2308 fixes huge memory usage of mount (8 MiB × number of archives) - IPv6 address support also: Location: more informative exception when parsing fails - borg single-file binary: use pyinstaller v3.2.1, #2396 this fixes that the prelink cronjob on some distros kills the borg binary by stripping away parts of it. - extract: - warning for unextracted big extended attributes, #2258 - also create parent dir for device files, if needed. - don't write to disk with --stdout, #2645 - archive check: detect and fix missing all-zero replacement chunks, #2180 - fix (de)compression exceptions, #2224 #2221 - files cache: update inode number, #2226 - borg rpc: use limited msgpack.Unpacker (security precaution), #2139 - Manifest: use limited msgpack.Unpacker (security precaution), #2175 - Location: accept //servername/share/path - fix ChunkIndex.__contains__ assertion for big-endian archs (harmless) - create: handle BackupOSError on a per-path level in one spot - fix error msg, there is no --keep-last in borg 1.0.x, #2282 - clamp (nano)second values to unproblematic range, #2304 - fuse / borg mount: - fix st_blocks to be an integer (not float) value - fix negative uid/gid crash (they could come into archives e.g. when backing up external drives under cygwin), #2674 - fix crash if empty (None) xattr is read - do pre-mount checks before opening repository - check llfuse is installed before asking for passphrase - borg rename: expand placeholders, #2386 - borg serve: fix forced command lines containing BORG_ env vars - fix error msg, it is --keep-within, not --within - fix borg key/debug/benchmark crashing without subcommand, #2240 - chunker: fix invalid use of types, don't do uint32_t >> 32 - document follow_symlinks requirements, check libc, #2507 New features: - added BORG_PASSCOMMAND environment variable, #2573 - add minimal version of in repository mandatory feature flags, #2134 This should allow us to make sure older borg versions can be cleanly prevented from doing operations that are no longer safe because of repository format evolution. This allows more fine grained control than just incrementing the manifest version. So for example a change that still allows new archives to be created but would corrupt the repository when an old version tries to delete an archive or check the repository would add the new feature to the check and delete set but leave it out of the write set. - borg delete --force --force to delete severely corrupted archives, #1975 Other changes: - embrace y2038 issue to support 32bit platforms - be more clear that this is a "beyond repair" case, #2427 - key file names: limit to 100 characters and remove colons from host name - upgrade FUSE for macOS to 3.5.8, #2346 - split up parsing and filtering for --keep-within, better error message, #2610 - docs: - fix caskroom link, #2299 - address SSH batch mode, #2202 #2270 - improve remote-path description - document snapshot usage, #2178 - document relative path usage, #1868 - one link per distro in the installation page - development: new branching model in git repository - kill api page - added FAQ section about backing up root partition - add bountysource badge, #2558 - create empty docs.txt reequirements, #2694 - README: how to help the project - note -v/--verbose requirement on affected options, #2542 - document borg init behaviour via append-only borg serve, #2440 - be clear about what buzhash is used for (chunking) and want it is not used for (deduplication)- also say already in the readme that we use a cryptohash for dedupe, so people don't worry, #2390 - add hint about chunker params to borg upgrade docs, #2421 - clarify borg upgrade docs, #2436 - quickstart: delete problematic BORG_PASSPRHASE use, #2623 - faq: specify "using inline shell scripts" - document pattern denial of service, #2624 - tests: - remove attic dependency of the tests, #2505 - travis: - enhance travis setuptools_scm situation - install fakeroot for Linux - add test for borg delete --force - enable remote tests on cygwin (the cygwin issue that caused these tests to break was fixed in cygwin at least since cygwin 2.8, maybe even since 2.7.0). - remove skipping the noatime tests on GNU/Hurd, #2710 - fix borg import issue, add comment, #2718 - include attic.tar.gz when installing the package also: add include_package_data=True Version 1.0.10 (2017-02-13) --------------------------- Bug fixes: - Manifest timestamps are now monotonically increasing, this fixes issues when the system clock jumps backwards or is set inconsistently across computers accessing the same repository, #2115 - Fixed testing regression in 1.0.10rc1 that lead to a hard dependency on py.test >= 3.0, #2112 New features: - "key export" can now generate a printable HTML page with both a QR code and a human-readable "paperkey" representation (and custom text) through the ``--qr-html`` option. The same functionality is also available through `paperkey.html `_, which is the same HTML page generated by ``--qr-html``. It works with existing "key export" files and key files. Other changes: - docs: - language clarification - "borg create --one-file-system" option does not respect mount points, but considers different file systems instead, #2141 - setup.py: build_api: sort file list for determinism Version 1.0.10rc1 (2017-01-29) ------------------------------ Bug fixes: - borg serve: fix transmission data loss of pipe writes, #1268 This affects only the cygwin platform (not Linux, BSD, OS X). - Avoid triggering an ObjectiveFS bug in xattr retrieval, #1992 - When running out of buffer memory when reading xattrs, only skip the current file, #1993 - Fixed "borg upgrade --tam" crashing with unencrypted repositories. Since :ref:`the issue ` is not relevant for unencrypted repositories, it now does nothing and prints an error, #1981. - Fixed change-passphrase crashing with unencrypted repositories, #1978 - Fixed "borg check repo::archive" indicating success if "archive" does not exist, #1997 - borg check: print non-exit-code warning if --last or --prefix aren't fulfilled - fix bad parsing of wrong repo location syntax - create: don't create hard link refs to failed files, mount: handle invalid hard link refs, #2092 - detect mingw byte order, #2073 - creating a new segment: use "xb" mode, #2099 - mount: umount on SIGINT/^C when in foreground, #2082 Other changes: - binary: use fixed AND freshly compiled pyinstaller bootloader, #2002 - xattr: ignore empty names returned by llistxattr(2) et al - Enable the fault handler: install handlers for the SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL signals to dump the Python traceback. - Also print a traceback on SIGUSR2. - borg change-passphrase: print key location (simplify making a backup of it) - officially support Python 3.6 (setup.py: add Python 3.6 qualifier) - tests: - vagrant / travis / tox: add Python 3.6 based testing - vagrant: fix openbsd repo, #2042 - vagrant: fix the freebsd64 machine, #2037 #2067 - vagrant: use python 3.5.3 to build binaries, #2078 - vagrant: use osxfuse 3.5.4 for tests / to build binaries vagrant: improve darwin64 VM settings - travis: fix osxfuse install (fixes OS X testing on Travis CI) - travis: require succeeding OS X tests, #2028 - travis: use latest pythons for OS X based testing - use pytest-xdist to parallelize testing - fix xattr test race condition, #2047 - setup.cfg: fix pytest deprecation warning, #2050 - docs: - language clarification - VM backup FAQ - borg create: document how to backup stdin, #2013 - borg upgrade: fix incorrect title levels - add CVE numbers for issues fixed in 1.0.9, #2106 - fix typos (taken from Debian package patch) - remote: include data hexdump in "unexpected RPC data" error message - remote: log SSH command line at debug level - API_VERSION: use numberspaces, #2023 - remove .github from pypi package, #2051 - add pip and setuptools to requirements file, #2030 - SyncFile: fix use of fd object after close (cosmetic) - Manifest.in: simplify, exclude \*.{so,dll,orig}, #2066 - ignore posix_fadvise errors in repository.py, #2095 (works around issues with docker on ARM) - make LoggedIO.close_segment reentrant, avoid reentrance Version 1.0.9 (2016-12-20) -------------------------- Security fixes: - A flaw in the cryptographic authentication scheme in Borg allowed an attacker to spoof the manifest. See :ref:`tam_vuln` above for the steps you should take. CVE-2016-10099 was assigned to this vulnerability. - borg check: When rebuilding the manifest (which should only be needed very rarely) duplicate archive names would be handled on a "first come first serve" basis, allowing an attacker to apparently replace archives. CVE-2016-10100 was assigned to this vulnerability. Bug fixes: - borg check: - rebuild manifest if it's corrupted - skip corrupted chunks during manifest rebuild - fix TypeError in integrity error handler, #1903, #1894 - fix location parser for archives with @ char (regression introduced in 1.0.8), #1930 - fix wrong duration/timestamps if system clock jumped during a create - fix progress display not updating if system clock jumps backwards - fix checkpoint interval being incorrect if system clock jumps Other changes: - docs: - add python3-devel as a dependency for cygwin-based installation - clarify extract is relative to current directory - FAQ: fix link to changelog - markup fixes - tests: - test_get\_(cache|keys)_dir: clean env state, #1897 - get back pytest's pretty assertion failures, #1938 - setup.py build_usage: - fixed build_usage not processing all commands - fixed build_usage not generating includes for debug commands Version 1.0.9rc1 (2016-11-27) ----------------------------- Bug fixes: - files cache: fix determination of newest mtime in backup set (which is used in cache cleanup and led to wrong "A" [added] status for unchanged files in next backup), #1860. - borg check: - fix incorrectly reporting attic 0.13 and earlier archives as corrupt - handle repo w/o objects gracefully and also bail out early if repo is *completely* empty, #1815. - fix tox/pybuild in 1.0-maint - at xattr module import time, loggers are not initialized yet New features: - borg umount exposed already existing umount code via the CLI api, so users can use it, which is more consistent than using borg to mount and fusermount -u (or umount) to un-mount, #1855. - implement borg create --noatime --noctime, fixes #1853 Other changes: - docs: - display README correctly on PyPI - improve cache / index docs, esp. files cache docs, fixes #1825 - different pattern matching for --exclude, #1779 - datetime formatting examples for {now} placeholder, #1822 - clarify passphrase mode attic repo upgrade, #1854 - clarify --umask usage, #1859 - clarify how to choose PR target branch - clarify prune behavior for different archive contents, #1824 - fix PDF issues, add logo, fix authors, headings, TOC - move security verification to support section - fix links in standalone README (:ref: tags) - add link to security contact in README - add FAQ about security - move fork differences to FAQ - add more details about resource usage - tests: skip remote tests on cygwin, #1268 - travis: - allow OS X failures until the brew cask osxfuse issue is fixed - caskroom osxfuse-beta gone, it's osxfuse now (3.5.3) - vagrant: - upgrade OSXfuse / FUSE for macOS to 3.5.3 - remove llfuse from tox.ini at a central place - do not try to install llfuse on centos6 - fix fuse test for darwin, #1546 - add windows virtual machine with cygwin - Vagrantfile cleanup / code deduplication Version 1.0.8 (2016-10-29) -------------------------- Bug fixes: - RemoteRepository: Fix busy wait in call_many, #940 New features: - implement borgmajor/borgminor/borgpatch placeholders, #1694 {borgversion} was already there (full version string). With the new placeholders you can now also get e.g. 1 or 1.0 or 1.0.8. Other changes: - avoid previous_location mismatch, #1741 due to the changed canonicalization for relative pathes in PR #1711 / #1655 (implement /./ relpath hack), there would be a changed repo location warning and the user would be asked if this is ok. this would break automation and require manual intervention, which is unwanted. thus, we automatically fix the previous_location config entry, if it only changed in the expected way, but still means the same location. - docs: - deployment.rst: do not use bare variables in ansible snippet - add clarification about append-only mode, #1689 - setup.py: add comment about requiring llfuse, #1726 - update usage.rst / api.rst - repo url / archive location docs + typo fix - quickstart: add a comment about other (remote) filesystems - vagrant / tests: - no chown when rsyncing (fixes boxes w/o vagrant group) - fix fuse permission issues on linux/freebsd, #1544 - skip fuse test for borg binary + fakeroot - ignore security.selinux xattrs, fixes tests on centos, #1735 Version 1.0.8rc1 (2016-10-17) ----------------------------- Bug fixes: - fix signal handling (SIGINT, SIGTERM, SIGHUP), #1620 #1593 Fixes e.g. leftover lock files for quickly repeated signals (e.g. Ctrl-C Ctrl-C) or lost connections or systemd sending SIGHUP. - progress display: adapt formatting to narrow screens, do not crash, #1628 - borg create --read-special - fix crash on broken symlink, #1584. also correctly processes broken symlinks. before this regressed to a crash (5b45385) a broken symlink would've been skipped. - process_symlink: fix missing backup_io() Fixes a chmod/chown/chgrp/unlink/rename/... crash race between getting dirents and dispatching to process_symlink. - yes(): abort on wrong answers, saying so, #1622 - fixed exception borg serve raised when connection was closed before reposiory was openend. add an error message for this. - fix read-from-closed-FD issue, #1551 (this seems not to get triggered in 1.0.x, but was discovered in master) - hashindex: fix iterators (always raise StopIteration when exhausted) (this seems not to get triggered in 1.0.x, but was discovered in master) - enable relative pathes in ssh:// repo URLs, via /./relpath hack, #1655 - allow repo pathes with colons, #1705 - update changed repo location immediately after acceptance, #1524 - fix debug get-obj / delete-obj crash if object not found and remote repo, #1684 - pyinstaller: use a spec file to build borg.exe binary, exclude osxfuse dylib on Mac OS X (avoids mismatch lib <-> driver), #1619 New features: - add "borg key export" / "borg key import" commands, #1555, so users are able to backup / restore their encryption keys more easily. Supported formats are the keyfile format used by borg internally and a special "paper" format with by line checksums for printed backups. For the paper format, the import is an interactive process which checks each line as soon as it is input. - add "borg debug-refcount-obj" to determine a repo objects' referrer counts, #1352 Other changes: - add "borg debug ..." subcommands (borg debug-* still works, but will be removed in borg 1.1) - setup.py: Add subcommand support to build_usage. - remote: change exception message for unexpected RPC data format to indicate dataflow direction. - improved messages / error reporting: - IntegrityError: add placeholder for message, so that the message we give appears not only in the traceback, but also in the (short) error message, #1572 - borg.key: include chunk id in exception msgs, #1571 - better messages for cache newer than repo, #1700 - vagrant (testing/build VMs): - upgrade OSXfuse / FUSE for macOS to 3.5.2 - update Debian Wheezy boxes, #1686 - openbsd / netbsd: use own boxes, fixes misc rsync installation and fuse/llfuse related testing issues, #1695 #1696 #1670 #1671 #1728 - docs: - add docs for "key export" and "key import" commands, #1641 - fix inconsistency in FAQ (pv-wrapper). - fix second block in "Easy to use" section not showing on GitHub, #1576 - add bestpractices badge - link reference docs and faq about BORG_FILES_CACHE_TTL, #1561 - improve borg info --help, explain size infos, #1532 - add release signing key / security contact to README, #1560 - add contribution guidelines for developers - development.rst: add sphinx_rtd_theme to the sphinx install command - adjust border color in borg.css - add debug-info usage help file - internals.rst: fix typos - setup.py: fix build_usage to always process all commands - added docs explaining multiple --restrict-to-path flags, #1602 - add more specific warning about write-access debug commands, #1587 - clarify FAQ regarding backup of virtual machines, #1672 - tests: - work around fuse xattr test issue with recent fakeroot - simplify repo/hashindex tests - travis: test fuse-enabled borg, use trusty to have a recent FUSE - re-enable fuse tests for RemoteArchiver (no deadlocks any more) - clean env for pytest based tests, #1714 - fuse_mount contextmanager: accept any options Version 1.0.7 (2016-08-19) -------------------------- Security fixes: - borg serve: fix security issue with remote repository access, #1428 If you used e.g. --restrict-to-path /path/client1/ (with or without trailing slash does not make a difference), it acted like a path prefix match using /path/client1 (note the missing trailing slash) - the code then also allowed working in e.g. /path/client13 or /path/client1000. As this could accidentally lead to major security/privacy issues depending on the pathes you use, the behaviour was changed to be a strict directory match. That means --restrict-to-path /path/client1 (with or without trailing slash does not make a difference) now uses /path/client1/ internally (note the trailing slash here!) for matching and allows precisely that path AND any path below it. So, /path/client1 is allowed, /path/client1/repo1 is allowed, but not /path/client13 or /path/client1000. If you willingly used the undocumented (dangerous) previous behaviour, you may need to rearrange your --restrict-to-path pathes now. We are sorry if that causes work for you, but we did not want a potentially dangerous behaviour in the software (not even using a for-backwards-compat option). Bug fixes: - fixed repeated LockTimeout exceptions when borg serve tried to write into a already write-locked repo (e.g. by a borg mount), #502 part b) This was solved by the fix for #1220 in 1.0.7rc1 already. - fix cosmetics + file leftover for "not a valid borg repository", #1490 - Cache: release lock if cache is invalid, #1501 - borg extract --strip-components: fix leak of preloaded chunk contents - Repository, when a InvalidRepository exception happens: - fix spurious, empty lock.roster - fix repo not closed cleanly New features: - implement borg debug-info, fixes #1122 (just calls already existing code via cli, same output as below tracebacks) Other changes: - skip the O_NOATIME test on GNU Hurd, fixes #1315 (this is a very minor issue and the GNU Hurd project knows the bug) - document using a clean repo to test / build the release Version 1.0.7rc2 (2016-08-13) ----------------------------- Bug fixes: - do not write objects to repository that are bigger than the allowed size, borg will reject reading them, #1451. Important: if you created archives with many millions of files or directories, please verify if you can open them successfully, e.g. try a "borg list REPO::ARCHIVE". - lz4 compression: dynamically enlarge the (de)compression buffer, the static buffer was not big enough for archives with extremely many items, #1453 - larger item metadata stream chunks, raise archive item limit by 8x, #1452 - fix untracked segments made by moved DELETEs, #1442 Impact: Previously (metadata) segments could become untracked when deleting data, these would never be cleaned up. - extended attributes (xattrs) related fixes: - fixed a race condition in xattrs querying that led to the entire file not being backed up (while logging the error, exit code = 1), #1469 - fixed a race condition in xattrs querying that led to a crash, #1462 - raise OSError including the error message derived from errno, deal with path being a integer FD Other changes: - print active env var override by default, #1467 - xattr module: refactor code, deduplicate, clean up - repository: split object size check into too small and too big - add a transaction_id assertion, so borg init on a broken (inconsistent) filesystem does not look like a coding error in borg, but points to the real problem. - explain confusing TypeError caused by compat support for old servers, #1456 - add forgotten usage help file from build_usage - refactor/unify buffer code into helpers.Buffer class, add tests - docs: - document archive limitation, #1452 - improve prune examples Version 1.0.7rc1 (2016-08-05) ----------------------------- Bug fixes: - fix repo lock deadlocks (related to lock upgrade), #1220 - catch unpacker exceptions, resync, #1351 - fix borg break-lock ignoring BORG_REPO env var, #1324 - files cache performance fixes (fixes unneccessary re-reading/chunking/ hashing of unmodified files for some use cases): - fix unintended file cache eviction, #1430 - implement BORG_FILES_CACHE_TTL, update FAQ, raise default TTL from 10 to 20, #1338 - FUSE: - cache partially read data chunks (performance), #965, #966 - always create a root dir, #1125 - use an OrderedDict for helptext, making the build reproducible, #1346 - RemoteRepository init: always call close on exceptions, #1370 (cosmetic) - ignore stdout/stderr broken pipe errors (cosmetic), #1116 New features: - better borg versions management support (useful esp. for borg servers wanting to offer multiple borg versions and for clients wanting to choose a specific server borg version), #1392: - add BORG_VERSION environment variable before executing "borg serve" via ssh - add new placeholder {borgversion} - substitute placeholders in --remote-path - borg init --append-only option (makes using the more secure append-only mode more convenient. when used remotely, this requires 1.0.7+ also on the borg server), #1291. Other changes: - Vagrantfile: - darwin64: upgrade to FUSE for macOS 3.4.1 (aka osxfuse), #1378 - xenial64: use user "ubuntu", not "vagrant" (as usual), #1331 - tests: - fix fuse tests on OS X, #1433 - docs: - FAQ: add backup using stable filesystem names recommendation - FAQ about glibc compatibility added, #491, glibc-check improved - FAQ: 'A' unchanged file; remove ambiguous entry age sentence. - OS X: install pkg-config to build with FUSE support, fixes #1400 - add notes about shell/sudo pitfalls with env. vars, #1380 - added platform feature matrix - implement borg debug-dump-repo-objs Version 1.0.6 (2016-07-12) -------------------------- Bug fixes: - Linux: handle multiple LD_PRELOAD entries correctly, #1314, #1111 - Fix crash with unclear message if the libc is not found, #1314, #1111 Other changes: - tests: - Fixed O_NOATIME tests for Solaris and GNU Hurd, #1315 - Fixed sparse file tests for (file) systems not supporting it, #1310 - docs: - Fixed syntax highlighting, #1313 - misc docs: added data processing overview picture Version 1.0.6rc1 (2016-07-10) ----------------------------- New features: - borg check --repair: heal damaged files if missing chunks re-appear (e.g. if the previously missing chunk was added again in a later backup archive), #148. (*) Also improved logging. Bug fixes: - sync_dir: silence fsync() failing with EINVAL, #1287 Some network filesystems (like smbfs) don't support this and we use this in repository code. - borg mount (FUSE): - fix directories being shadowed when contained paths were also specified, #1295 - raise I/O Error (EIO) on damaged files (unless -o allow_damaged_files is used), #1302. (*) - borg extract: warn if a damaged file is extracted, #1299. (*) - Added some missing return code checks (ChunkIndex._add, hashindex_resize). - borg check: fix/optimize initial hash table size, avoids resize of the table. Other changes: - tests: - add more FUSE tests, #1284 - deduplicate fuse (u)mount code - fix borg binary test issues, #862 - docs: - changelog: added release dates to older borg releases - fix some sphinx (docs generator) warnings, #881 Notes: (*) Some features depend on information (chunks_healthy list) added to item metadata when a file with missing chunks was "repaired" using all-zero replacement chunks. The chunks_healthy list is generated since borg 1.0.4, thus borg can't recognize such "repaired" (but content-damaged) files if the repair was done with an older borg version. Version 1.0.5 (2016-07-07) -------------------------- Bug fixes: - borg mount: fix FUSE crash in xattr code on Linux introduced in 1.0.4, #1282 Other changes: - backport some FAQ entries from master branch - add release helper scripts - Vagrantfile: - centos6: no FUSE, don't build binary - add xz for redhat-like dists Version 1.0.4 (2016-07-07) -------------------------- New features: - borg serve --append-only, #1168 This was included because it was a simple change (append-only functionality was already present via repository config file) and makes better security now practically usable. - BORG_REMOTE_PATH environment variable, #1258 This was included because it was a simple change (--remote-path cli option was already present) and makes borg much easier to use if you need it. - Repository: cleanup incomplete transaction on "no space left" condition. In many cases, this can avoid a 100% full repo filesystem (which is very problematic as borg always needs free space - even to delete archives). Bug fixes: - Fix wrong handling and reporting of OSErrors in borg create, #1138. This was a serious issue: in the context of "borg create", errors like repository I/O errors (e.g. disk I/O errors, ssh repo connection errors) were handled badly and did not lead to a crash (which would be good for this case, because the repo transaction would be incomplete and trigger a transaction rollback to clean up). Now, error handling for source files is cleanly separated from every other error handling, so only problematic input files are logged and skipped. - Implement fail-safe error handling for borg extract. Note that this isn't nearly as critical as the borg create error handling bug, since nothing is written to the repo. So this was "merely" misleading error reporting. - Add missing error handler in directory attr restore loop. - repo: make sure write data hits disk before the commit tag (#1236) and also sync the containing directory. - FUSE: getxattr fail must use errno.ENOATTR, #1126 (fixes Mac OS X Finder malfunction: "zero bytes" file length, access denied) - borg check --repair: do not lose information about the good/original chunks. If we do not lose the original chunk IDs list when "repairing" a file (replacing missing chunks with all-zero chunks), we have a chance to "heal" the file back into its original state later, in case the chunks re-appear (e.g. in a fresh backup). Healing is not implemented yet, see #148. - fixes for --read-special mode: - ignore known files cache, #1241 - fake regular file mode, #1214 - improve symlinks handling, #1215 - remove passphrase from subprocess environment, #1105 - Ignore empty index file (will trigger index rebuild), #1195 - add missing placeholder support for --prefix, #1027 - improve exception handling for placeholder replacement - catch and format exceptions in arg parsing - helpers: fix "undefined name 'e'" in exception handler - better error handling for missing repo manifest, #1043 - borg delete: - make it possible to delete a repo without manifest - borg delete --forced allows to delete corrupted archives, #1139 - borg check: - make borg check work for empty repo - fix resync and msgpacked item qualifier, #1135 - rebuild_manifest: fix crash if 'name' or 'time' key were missing. - better validation of item metadata dicts, #1130 - better validation of archive metadata dicts - close the repo on exit - even if rollback did not work, #1197. This is rather cosmetic, it avoids repo closing in the destructor. - tests: - fix sparse file test, #1170 - flake8: ignore new F405, #1185 - catch "invalid argument" on cygwin, #257 - fix sparseness assertion in test prep, #1264 Other changes: - make borg build/work on OpenSSL 1.0 and 1.1, #1187 - docs / help: - fix / clarify prune help, #1143 - fix "patterns" help formatting - add missing docs / help about placeholders - resources: rename atticmatic to borgmatic - document sshd settings, #545 - more details about checkpoints, add split trick, #1171 - support docs: add freenode web chat link, #1175 - add prune visualization / example, #723 - add note that Fnmatch is default, #1247 - make clear that lzma levels > 6 are a waste of cpu cycles - add a "do not edit" note to auto-generated files, #1250 - update cygwin installation docs - repository interoperability with borg master (1.1dev) branch: - borg check: read item metadata keys from manifest, #1147 - read v2 hints files, #1235 - fix hints file "unknown version" error handling bug - tests: add tests for format_line - llfuse: update version requirement for freebsd - Vagrantfile: - use openbsd 5.9, #716 - do not install llfuse on netbsd (broken) - update OSXfuse to version 3.3.3 - use Python 3.5.2 to build the binaries - glibc compatibility checker: scripts/glibc_check.py - add .eggs to .gitignore Version 1.0.3 (2016-05-20) -------------------------- Bug fixes: - prune: avoid that checkpoints are kept and completed archives are deleted in a prune run), #997 - prune: fix commandline argument validation - some valid command lines were considered invalid (annoying, but harmless), #942 - fix capabilities extraction on Linux (set xattrs last, after chown()), #1069 - repository: fix commit tags being seen in data - when probing key files, do binary reads. avoids crash when non-borg binary files are located in borg's key files directory. - handle SIGTERM and make a clean exit - avoids orphan lock files. - repository cache: don't cache large objects (avoid using lots of temp. disk space), #1063 Other changes: - Vagrantfile: OS X: update osxfuse / install lzma package, #933 - setup.py: add check for platform_darwin.c - setup.py: on freebsd, use a llfuse release that builds ok - docs / help: - update readthedocs URLs, #991 - add missing docs for "borg break-lock", #992 - borg create help: add some words to about the archive name - borg create help: document format tags, #894 Version 1.0.2 (2016-04-16) -------------------------- Bug fixes: - fix malfunction and potential corruption on (nowadays rather rare) big-endian architectures or bi-endian archs in (rare) BE mode. #886, #889 cache resync / index merge was malfunctioning due to this, potentially leading to data loss. borg info had cosmetic issues (displayed wrong values). note: all (widespread) little-endian archs (like x86/x64) or bi-endian archs in (widespread) LE mode (like ARMEL, MIPSEL, ...) were NOT affected. - add overflow and range checks for 1st (special) uint32 of the hashindex values, switch from int32 to uint32. - fix so that refcount will never overflow, but just stick to max. value after a overflow would have occured. - borg delete: fix --cache-only for broken caches, #874 Makes --cache-only idempotent: it won't fail if the cache is already deleted. - fixed borg create --one-file-system erroneously traversing into other filesystems (if starting fs device number was 0), #873 - workround a bug in Linux fadvise FADV_DONTNEED, #907 Other changes: - better test coverage for hashindex, incl. overflow testing, checking correct computations so endianness issues would be discovered. - reproducible doc for ProgressIndicator*, make the build reproducible. - use latest llfuse for vagrant machines - docs: - use /path/to/repo in examples, fixes #901 - fix confusing usage of "repo" as archive name (use "arch") Version 1.0.1 (2016-04-08) -------------------------- New features: Usually there are no new features in a bugfix release, but these were added due to their high impact on security/safety/speed or because they are fixes also: - append-only mode for repositories, #809, #36 (see docs) - borg create: add --ignore-inode option to make borg detect unmodified files even if your filesystem does not have stable inode numbers (like sshfs and possibly CIFS). - add options --warning, --error, --critical for missing log levels, #826. it's not recommended to suppress warnings or errors, but the user may decide this on his own. note: --warning is not given to borg serve so a <= 1.0.0 borg will still work as server (it is not needed as it is the default). do not use --error or --critical when using a <= 1.0.0 borg server. Bug fixes: - fix silently skipping EIO, #748 - add context manager for Repository (avoid orphan repository locks), #285 - do not sleep for >60s while waiting for lock, #773 - unpack file stats before passing to FUSE - fix build on illumos - don't try to backup doors or event ports (Solaris and derivates) - remove useless/misleading libc version display, #738 - test suite: reset exit code of persistent archiver, #844 - RemoteRepository: clean up pipe if remote open() fails - Remote: don't print tracebacks for Error exceptions handled downstream, #792 - if BORG_PASSPHRASE is present but wrong, don't prompt for password, but fail instead, #791 - ArchiveChecker: move "orphaned objects check skipped" to INFO log level, #826 - fix capitalization, add ellipses, change log level to debug for 2 messages, #798 Other changes: - update llfuse requirement, llfuse 1.0 works - update OS / dist packages on build machines, #717 - prefer showing --info over -v in usage help, #859 - docs: - fix cygwin requirements (gcc-g++) - document how to debug / file filesystem issues, #664 - fix reproducible build of api docs - RTD theme: CSS !important overwrite, #727 - Document logo font. Recreate logo png. Remove GIMP logo file. Version 1.0.0 (2016-03-05) -------------------------- The major release number change (0.x -> 1.x) indicates bigger incompatible changes, please read the compatibility notes, adapt / test your scripts and check your backup logs. Compatibility notes: - drop support for python 3.2 and 3.3, require 3.4 or 3.5, #221 #65 #490 note: we provide binaries that include python 3.5.1 and everything else needed. they are an option in case you are stuck with < 3.4 otherwise. - change encryption to be on by default (using "repokey" mode) - moved keyfile keys from ~/.borg/keys to ~/.config/borg/keys, you can either move them manually or run "borg upgrade " - remove support for --encryption=passphrase, use borg migrate-to-repokey to switch to repokey mode, #97 - remove deprecated --compression , use --compression zlib, instead in case of 0, you could also use --compression none - remove deprecated --hourly/daily/weekly/monthly/yearly use --keep-hourly/daily/weekly/monthly/yearly instead - remove deprecated --do-not-cross-mountpoints, use --one-file-system instead - disambiguate -p option, #563: - -p now is same as --progress - -P now is same as --prefix - remove deprecated "borg verify", use "borg extract --dry-run" instead - cleanup environment variable semantics, #355 the environment variables used to be "yes sayers" when set, this was conceptually generalized to "automatic answerers" and they just give their value as answer (as if you typed in that value when being asked). See the "usage" / "Environment Variables" section of the docs for details. - change the builtin default for --chunker-params, create 2MiB chunks, #343 --chunker-params new default: 19,23,21,4095 - old default: 10,23,16,4095 one of the biggest issues with borg < 1.0 (and also attic) was that it had a default target chunk size of 64kiB, thus it created a lot of chunks and thus also a huge chunk management overhead (high RAM and disk usage). please note that the new default won't change the chunks that you already have in your repository. the new big chunks do not deduplicate with the old small chunks, so expect your repo to grow at least by the size of every changed file and in the worst case (e.g. if your files cache was lost / is not used) by the size of every file (minus any compression you might use). in case you want to immediately see a much lower resource usage (RAM / disk) for chunks management, it might be better to start with a new repo than continuing in the existing repo (with an existing repo, you'ld have to wait until all archives with small chunks got pruned to see a lower resource usage). if you used the old --chunker-params default value (or if you did not use --chunker-params option at all) and you'ld like to continue using small chunks (and you accept the huge resource usage that comes with that), just explicitly use borg create --chunker-params=10,23,16,4095. - archive timestamps: the 'time' timestamp now refers to archive creation start time (was: end time), the new 'time_end' timestamp refers to archive creation end time. This might affect prune if your backups take rather long. if you give a timestamp via cli this is stored into 'time', therefore it now needs to mean archive creation start time. New features: - implement password roundtrip, #695 Bug fixes: - remote end does not need cache nor keys directories, do not create them, #701 - added retry counter for passwords, #703 Other changes: - fix compiler warnings, #697 - docs: - update README.rst to new changelog location in docs/changes.rst - add Teemu to AUTHORS - changes.rst: fix old chunker params, #698 - FAQ: how to limit bandwidth Version 1.0.0rc2 (2016-02-28) ----------------------------- New features: - format options for location: user, pid, fqdn, hostname, now, utcnow, user - borg list --list-format - borg prune -v --list enables the keep/prune list output, #658 Bug fixes: - fix _open_rb noatime handling, #657 - add a simple archivename validator, #680 - borg create --stats: show timestamps in localtime, use same labels/formatting as borg info, #651 - llfuse compatibility fixes (now compatible with: 0.40, 0.41, 0.42) Other changes: - it is now possible to use "pip install borgbackup[fuse]" to automatically install the llfuse dependency using the correct version requirement for it. you still need to care about having installed the FUSE / build related OS package first, though, so that building llfuse can succeed. - Vagrant: drop Ubuntu Precise (12.04) - does not have Python >= 3.4 - Vagrant: use pyinstaller v3.1.1 to build binaries - docs: - borg upgrade: add to docs that only LOCAL repos are supported - borg upgrade also handles borg 0.xx -> 1.0 - use pip extras or requirements file to install llfuse - fix order in release process - updated usage docs and other minor / cosmetic fixes - verified borg examples in docs, #644 - freebsd dependency installation and fuse configuration, #649 - add example how to restore a raw device, #671 - add a hint about the dev headers needed when installing from source - add examples for delete (and handle delete after list, before prune), #656 - update example for borg create -v --stats (use iso datetime format), #663 - added example to BORG_RSH docs - "connection closed by remote": add FAQ entry and point to issue #636 Version 1.0.0rc1 (2016-02-07) ----------------------------- New features: - borg migrate-to-repokey ("passphrase" -> "repokey" encryption key mode) - implement --short for borg list REPO, #611 - implement --list for borg extract (consistency with borg create) - borg serve: overwrite client's --restrict-to-path with ssh forced command's option value (but keep everything else from the client commandline), #544 - use $XDG_CONFIG_HOME/keys for keyfile keys (~/.config/borg/keys), #515 - "borg upgrade" moves the keyfile keys to the new location - display both archive creation start and end time in "borg info", #627 Bug fixes: - normalize trailing slashes for the repository path, #606 - Cache: fix exception handling in __init__, release lock, #610 Other changes: - suppress unneeded exception context (PEP 409), simpler tracebacks - removed special code needed to deal with imperfections / incompatibilities / missing stuff in py 3.2/3.3, simplify code that can be done simpler in 3.4 - removed some version requirements that were kept on old versions because newer did not support py 3.2 any more - use some py 3.4+ stdlib code instead of own/openssl/pypi code: - use os.urandom instead of own cython openssl RAND_bytes wrapper, #493 - use hashlib.pbkdf2_hmac from py stdlib instead of own openssl wrapper - use hmac.compare_digest instead of == operator (constant time comparison) - use stat.filemode instead of homegrown code - use "mock" library from stdlib, #145 - remove borg.support (with non-broken argparse copy), it is ok in 3.4+, #358 - Vagrant: copy CHANGES.rst as symlink, #592 - cosmetic code cleanups, add flake8 to tox/travis, #4 - docs / help: - make "borg -h" output prettier, #591 - slightly rephrase prune help - add missing example for --list option of borg create - quote exclude line that includes an asterisk to prevent shell expansion - fix dead link to license - delete Ubuntu Vivid, it is not supported anymore (EOL) - OS X binary does not work for older OS X releases, #629 - borg serve's special support for forced/original ssh commands, #544 - misc. updates and fixes Version 0.30.0 (2016-01-23) --------------------------- Compatibility notes: - you may need to use -v (or --info) more often to actually see output emitted at INFO log level (because it is suppressed at the default WARNING log level). See the "general" section in the usage docs. - for borg create, you need --list (additionally to -v) to see the long file list (was needed so you can have e.g. --stats alone without the long list) - see below about BORG_DELETE_I_KNOW_WHAT_I_AM_DOING (was: BORG_CHECK_I_KNOW_WHAT_I_AM_DOING) Bug fixes: - fix crash when using borg create --dry-run --keep-tag-files, #570 - make sure teardown with cleanup happens for Cache and RepositoryCache, avoiding leftover locks and TEMP dir contents, #285 (partially), #548 - fix locking KeyError, partial fix for #502 - log stats consistently, #526 - add abbreviated weekday to timestamp format, fixes #496 - strip whitespace when loading exclusions from file - unset LD_LIBRARY_PATH before invoking ssh, fixes strange OpenSSL library version warning when using the borg binary, #514 - add some error handling/fallback for C library loading, #494 - added BORG_DELETE_I_KNOW_WHAT_I_AM_DOING for check in "borg delete", #503 - remove unused "repair" rpc method name New features: - borg create: implement exclusions using regular expression patterns. - borg create: implement inclusions using patterns. - borg extract: support patterns, #361 - support different styles for patterns: - fnmatch (`fm:` prefix, default when omitted), like borg <= 0.29. - shell (`sh:` prefix) with `*` not matching directory separators and `**/` matching 0..n directories - path prefix (`pp:` prefix, for unifying borg create pp1 pp2 into the patterns system), semantics like in borg <= 0.29 - regular expression (`re:`), new! - --progress option for borg upgrade (#291) and borg delete - update progress indication more often (e.g. for borg create within big files or for borg check repo), #500 - finer chunker granularity for items metadata stream, #547, #487 - borg create --list now used (additionally to -v) to enable the verbose file list output - display borg version below tracebacks, #532 Other changes: - hashtable size (and thus: RAM and disk consumption) follows a growth policy: grows fast while small, grows slower when getting bigger, #527 - Vagrantfile: use pyinstaller 3.1 to build binaries, freebsd sqlite3 fix, fixes #569 - no separate binaries for centos6 any more because the generic linux binaries also work on centos6 (or in general: on systems with a slightly older glibc than debian7 - dev environment: require virtualenv<14.0 so we get a py32 compatible pip - docs: - add space-saving chunks.archive.d trick to FAQ - important: clarify -v and log levels in usage -> general, please read! - sphinx configuration: create a simple man page from usage docs - add a repo server setup example - disable unneeded SSH features in authorized_keys examples for security. - borg prune only knows "--keep-within" and not "--within" - add gource video to resources docs, #507 - add netbsd install instructions - authors: make it more clear what refers to borg and what to attic - document standalone binary requirements, #499 - rephrase the mailing list section - development docs: run build_api and build_usage before tagging release - internals docs: hash table max. load factor is 0.75 now - markup, typo, grammar, phrasing, clarifications and other fixes. - add gcc gcc-c++ to redhat/fedora/corora install docs, fixes #583 Version 0.29.0 (2015-12-13) --------------------------- Compatibility notes: - when upgrading to 0.29.0 you need to upgrade client as well as server installations due to the locking and commandline interface changes otherwise you'll get an error msg about a RPC protocol mismatch or a wrong commandline option. if you run a server that needs to support both old and new clients, it is suggested that you have a "borg-0.28.2" and a "borg-0.29.0" command. clients then can choose via e.g. "borg --remote-path=borg-0.29.0 ...". - the default waiting time for a lock changed from infinity to 1 second for a better interactive user experience. if the repo you want to access is currently locked, borg will now terminate after 1s with an error message. if you have scripts that shall wait for the lock for a longer time, use --lock-wait N (with N being the maximum wait time in seconds). Bug fixes: - hash table tuning (better chosen hashtable load factor 0.75 and prime initial size of 1031 gave ~1000x speedup in some scenarios) - avoid creation of an orphan lock for one case, #285 - --keep-tag-files: fix file mode and multiple tag files in one directory, #432 - fixes for "borg upgrade" (attic repo converter), #466 - remove --progress isatty magic (and also --no-progress option) again, #476 - borg init: display proper repo URL - fix format of umask in help pages, #463 New features: - implement --lock-wait, support timeout for UpgradableLock, #210 - implement borg break-lock command, #157 - include system info below traceback, #324 - sane remote logging, remote stderr, #461: - remote log output: intercept it and log it via local logging system, with "Remote: " prefixed to message. log remote tracebacks. - remote stderr: output it to local stderr with "Remote: " prefixed. - add --debug and --info (same as --verbose) to set the log level of the builtin logging configuration (which otherwise defaults to warning), #426 note: there are few messages emitted at DEBUG level currently. - optionally configure logging via env var BORG_LOGGING_CONF - add --filter option for status characters: e.g. to show only the added or modified files (and also errors), use "borg create -v --filter=AME ...". - more progress indicators, #394 - use ISO-8601 date and time format, #375 - "borg check --prefix" to restrict archive checking to that name prefix, #206 Other changes: - hashindex_add C implementation (speed up cache re-sync for new archives) - increase FUSE read_size to 1024 (speed up metadata operations) - check/delete/prune --save-space: free unused segments quickly, #239 - increase rpc protocol version to 2 (see also Compatibility notes), #458 - silence borg by default (via default log level WARNING) - get rid of C compiler warnings, #391 - upgrade OS X FUSE to 3.0.9 on the OS X binary build system - use python 3.5.1 to build binaries - docs: - new mailing list borgbackup@python.org, #468 - readthedocs: color and logo improvements - load coverage icons over SSL (avoids mixed content) - more precise binary installation steps - update release procedure docs about OS X FUSE - FAQ entry about unexpected 'A' status for unchanged file(s), #403 - add docs about 'E' file status - add "borg upgrade" docs, #464 - add developer docs about output and logging - clarify encryption, add note about client-side encryption - add resources section, with videos, talks, presentations, #149 - Borg moved to Arch Linux [community] - fix wrong installation instructions for archlinux Version 0.28.2 (2015-11-15) --------------------------- New features: - borg create --exclude-if-present TAGFILE - exclude directories that have the given file from the backup. You can additionally give --keep-tag-files to preserve just the directory roots and the tag-files (but not backup other directory contents), #395, attic #128, attic #142 Other changes: - do not create docs sources at build time (just have them in the repo), completely remove have_cython() hack, do not use the "mock" library at build time, #384 - avoid hidden import, make it easier for PyInstaller, easier fix for #218 - docs: - add description of item flags / status output, fixes #402 - explain how to regenerate usage and API files (build_api or build_usage) and when to commit usage files directly into git, #384 - minor install docs improvements Version 0.28.1 (2015-11-08) --------------------------- Bug fixes: - do not try to build api / usage docs for production install, fixes unexpected "mock" build dependency, #384 Other changes: - avoid using msgpack.packb at import time - fix formatting issue in changes.rst - fix build on readthedocs Version 0.28.0 (2015-11-08) --------------------------- Compatibility notes: - changed return codes (exit codes), see docs. in short: old: 0 = ok, 1 = error. now: 0 = ok, 1 = warning, 2 = error New features: - refactor return codes (exit codes), fixes #61 - add --show-rc option enable "terminating with X status, rc N" output, fixes 58, #351 - borg create backups atime and ctime additionally to mtime, fixes #317 - extract: support atime additionally to mtime - FUSE: support ctime and atime additionally to mtime - support borg --version - emit a warning if we have a slow msgpack installed - borg list --prefix=thishostname- REPO, fixes #205 - Debug commands (do not use except if you know what you do: debug-get-obj, debug-put-obj, debug-delete-obj, debug-dump-archive-items. Bug fixes: - setup.py: fix bug related to BORG_LZ4_PREFIX processing - fix "check" for repos that have incomplete chunks, fixes #364 - borg mount: fix unlocking of repository at umount time, fixes #331 - fix reading files without touching their atime, #334 - non-ascii ACL fixes for Linux, FreeBSD and OS X, #277 - fix acl_use_local_uid_gid() and add a test for it, attic #359 - borg upgrade: do not upgrade repositories in place by default, #299 - fix cascading failure with the index conversion code, #269 - borg check: implement 'cmdline' archive metadata value decoding, #311 - fix RobustUnpacker, it missed some metadata keys (new atime and ctime keys were missing, but also bsdflags). add check for unknown metadata keys. - create from stdin: also save atime, ctime (cosmetic) - use default_notty=False for confirmations, fixes #345 - vagrant: fix msgpack installation on centos, fixes #342 - deal with unicode errors for symlinks in same way as for regular files and have a helpful warning message about how to fix wrong locale setup, fixes #382 - add ACL keys the RobustUnpacker must know about Other changes: - improve file size displays, more flexible size formatters - explicitly commit to the units standard, #289 - archiver: add E status (means that an error occurred when processing this (single) item - do binary releases via "github releases", closes #214 - create: use -x and --one-file-system (was: --do-not-cross-mountpoints), #296 - a lot of changes related to using "logging" module and screen output, #233 - show progress display if on a tty, output more progress information, #303 - factor out status output so it is consistent, fix surrogates removal, maybe fixes #309 - move away from RawConfigParser to ConfigParser - archive checker: better error logging, give chunk_id and sequence numbers (can be used together with borg debug-dump-archive-items). - do not mention the deprecated passphrase mode - emit a deprecation warning for --compression N (giving a just a number) - misc .coverragerc fixes (and coverage measurement improvements), fixes #319 - refactor confirmation code, reduce code duplication, add tests - prettier error messages, fixes #307, #57 - tests: - add a test to find disk-full issues, #327 - travis: also run tests on Python 3.5 - travis: use tox -r so it rebuilds the tox environments - test the generated pyinstaller-based binary by archiver unit tests, #215 - vagrant: tests: announce whether fakeroot is used or not - vagrant: add vagrant user to fuse group for debianoid systems also - vagrant: llfuse install on darwin needs pkgconfig installed - vagrant: use pyinstaller from develop branch, fixes #336 - benchmarks: test create, extract, list, delete, info, check, help, fixes #146 - benchmarks: test with both the binary and the python code - archiver tests: test with both the binary and the python code, fixes #215 - make basic test more robust - docs: - moved docs to borgbackup.readthedocs.org, #155 - a lot of fixes and improvements, use mobile-friendly RTD standard theme - use zlib,6 compression in some examples, fixes #275 - add missing rename usage to docs, closes #279 - include the help offered by borg help in the usage docs, fixes #293 - include a list of major changes compared to attic into README, fixes #224 - add OS X install instructions, #197 - more details about the release process, #260 - fix linux glibc requirement (binaries built on debian7 now) - build: move usage and API generation to setup.py - update docs about return codes, #61 - remove api docs (too much breakage on rtd) - borgbackup install + basics presentation (asciinema) - describe the current style guide in documentation - add section about debug commands - warn about not running out of space - add example for rename - improve chunker params docs, fixes #362 - minor development docs update Version 0.27.0 (2015-10-07) --------------------------- New features: - "borg upgrade" command - attic -> borg one time converter / migration, #21 - temporary hack to avoid using lots of disk space for chunks.archive.d, #235: To use it: rm -rf chunks.archive.d ; touch chunks.archive.d - respect XDG_CACHE_HOME, attic #181 - add support for arbitrary SSH commands, attic #99 - borg delete --cache-only REPO (only delete cache, not REPO), attic #123 Bug fixes: - use Debian 7 (wheezy) to build pyinstaller borgbackup binaries, fixes slow down observed when running the Centos6-built binary on Ubuntu, #222 - do not crash on empty lock.roster, fixes #232 - fix multiple issues with the cache config version check, #234 - fix segment entry header size check, attic #352 plus other error handling improvements / code deduplication there. - always give segment and offset in repo IntegrityErrors Other changes: - stop producing binary wheels, remove docs about it, #147 - docs: - add warning about prune - generate usage include files only as needed - development docs: add Vagrant section - update / improve / reformat FAQ - hint to single-file pyinstaller binaries from README Version 0.26.1 (2015-09-28) --------------------------- This is a minor update, just docs and new pyinstaller binaries. - docs update about python and binary requirements - better docs for --read-special, fix #220 - re-built the binaries, fix #218 and #213 (glibc version issue) - update web site about single-file pyinstaller binaries Note: if you did a python-based installation, there is no need to upgrade. Version 0.26.0 (2015-09-19) --------------------------- New features: - Faster cache sync (do all in one pass, remove tar/compression stuff), #163 - BORG_REPO env var to specify the default repo, #168 - read special files as if they were regular files, #79 - implement borg create --dry-run, attic issue #267 - Normalize paths before pattern matching on OS X, #143 - support OpenBSD and NetBSD (except xattrs/ACLs) - support / run tests on Python 3.5 Bug fixes: - borg mount repo: use absolute path, attic #200, attic #137 - chunker: use off_t to get 64bit on 32bit platform, #178 - initialize chunker fd to -1, so it's not equal to STDIN_FILENO (0) - fix reaction to "no" answer at delete repo prompt, #182 - setup.py: detect lz4.h header file location - to support python < 3.2.4, add less buggy argparse lib from 3.2.6 (#194) - fix for obtaining ``char *`` from temporary Python value (old code causes a compile error on Mint 17.2) - llfuse 0.41 install troubles on some platforms, require < 0.41 (UnicodeDecodeError exception due to non-ascii llfuse setup.py) - cython code: add some int types to get rid of unspecific python add / subtract operations (avoid ``undefined symbol FPE_``... error on some platforms) - fix verbose mode display of stdin backup - extract: warn if a include pattern never matched, fixes #209, implement counters for Include/ExcludePatterns - archive names with slashes are invalid, attic issue #180 - chunker: add a check whether the POSIX_FADV_DONTNEED constant is defined - fixes building on OpenBSD. Other changes: - detect inconsistency / corruption / hash collision, #170 - replace versioneer with setuptools_scm, #106 - docs: - pkg-config is needed for llfuse installation - be more clear about pruning, attic issue #132 - unit tests: - xattr: ignore security.selinux attribute showing up - ext3 seems to need a bit more space for a sparse file - do not test lzma level 9 compression (avoid MemoryError) - work around strange mtime granularity issue on netbsd, fixes #204 - ignore st_rdev if file is not a block/char device, fixes #203 - stay away from the setgid and sticky mode bits - use Vagrant to do easy cross-platform testing (#196), currently: - Debian 7 "wheezy" 32bit, Debian 8 "jessie" 64bit - Ubuntu 12.04 32bit, Ubuntu 14.04 64bit - Centos 7 64bit - FreeBSD 10.2 64bit - OpenBSD 5.7 64bit - NetBSD 6.1.5 64bit - Darwin (OS X Yosemite) Version 0.25.0 (2015-08-29) --------------------------- Compatibility notes: - lz4 compression library (liblz4) is a new requirement (#156) - the new compression code is very compatible: as long as you stay with zlib compression, older borg releases will still be able to read data from a repo/archive made with the new code (note: this is not the case for the default "none" compression, use "zlib,0" if you want a "no compression" mode that can be read by older borg). Also the new code is able to read repos and archives made with older borg versions (for all zlib levels 0..9). Deprecations: - --compression N (with N being a number, as in 0.24) is deprecated. We keep the --compression 0..9 for now to not break scripts, but it is deprecated and will be removed later, so better fix your scripts now: --compression 0 (as in 0.24) is the same as --compression zlib,0 (now). BUT: if you do not want compression, you rather want --compression none (which is the default). --compression 1 (in 0.24) is the same as --compression zlib,1 (now) --compression 9 (in 0.24) is the same as --compression zlib,9 (now) New features: - create --compression none (default, means: do not compress, just pass through data "as is". this is more efficient than zlib level 0 as used in borg 0.24) - create --compression lz4 (super-fast, but not very high compression) - create --compression zlib,N (slower, higher compression, default for N is 6) - create --compression lzma,N (slowest, highest compression, default N is 6) - honor the nodump flag (UF_NODUMP) and do not backup such items - list --short just outputs a simple list of the files/directories in an archive Bug fixes: - fixed --chunker-params parameter order confusion / malfunction, fixes #154 - close fds of segments we delete (during compaction) - close files which fell out the lrucache - fadvise DONTNEED now is only called for the byte range actually read, not for the whole file, fixes #158. - fix issue with negative "all archives" size, fixes #165 - restore_xattrs: ignore if setxattr fails with EACCES, fixes #162 Other changes: - remove fakeroot requirement for tests, tests run faster without fakeroot (test setup does not fail any more without fakeroot, so you can run with or without fakeroot), fixes #151 and #91. - more tests for archiver - recover_segment(): don't assume we have an fd for segment - lrucache refactoring / cleanup, add dispose function, py.test tests - generalize hashindex code for any key length (less hardcoding) - lock roster: catch file not found in remove() method and ignore it - travis CI: use requirements file - improved docs: - replace hack for llfuse with proper solution (install libfuse-dev) - update docs about compression - update development docs about fakeroot - internals: add some words about lock files / locking system - support: mention BountySource and for what it can be used - theme: use a lighter green - add pypi, wheel, dist package based install docs - split install docs into system-specific preparations and generic instructions Version 0.24.0 (2015-08-09) --------------------------- Incompatible changes (compared to 0.23): - borg now always issues --umask NNN option when invoking another borg via ssh on the repository server. By that, it's making sure it uses the same umask for remote repos as for local ones. Because of this, you must upgrade both server and client(s) to 0.24. - the default umask is 077 now (if you do not specify via --umask) which might be a different one as you used previously. The default umask avoids that you accidentally give access permissions for group and/or others to files created by borg (e.g. the repository). Deprecations: - "--encryption passphrase" mode is deprecated, see #85 and #97. See the new "--encryption repokey" mode for a replacement. New features: - borg create --chunker-params ... to configure the chunker, fixes #16 (attic #302, attic #300, and somehow also #41). This can be used to reduce memory usage caused by chunk management overhead, so borg does not create a huge chunks index/repo index and eats all your RAM if you back up lots of data in huge files (like VM disk images). See docs/misc/create_chunker-params.txt for more information. - borg info now reports chunk counts in the chunk index. - borg create --compression 0..9 to select zlib compression level, fixes #66 (attic #295). - borg init --encryption repokey (to store the encryption key into the repo), fixes #85 - improve at-end error logging, always log exceptions and set exit_code=1 - LoggedIO: better error checks / exceptions / exception handling - implement --remote-path to allow non-default-path borg locations, #125 - implement --umask M and use 077 as default umask for better security, #117 - borg check: give a named single archive to it, fixes #139 - cache sync: show progress indication - cache sync: reimplement the chunk index merging in C Bug fixes: - fix segfault that happened for unreadable files (chunker: n needs to be a signed size_t), #116 - fix the repair mode, #144 - repo delete: add destroy to allowed rpc methods, fixes issue #114 - more compatible repository locking code (based on mkdir), maybe fixes #92 (attic #317, attic #201). - better Exception msg if no Borg is installed on the remote repo server, #56 - create a RepositoryCache implementation that can cope with >2GiB, fixes attic #326. - fix Traceback when running check --repair, attic #232 - clarify help text, fixes #73. - add help string for --no-files-cache, fixes #140 Other changes: - improved docs: - added docs/misc directory for misc. writeups that won't be included "as is" into the html docs. - document environment variables and return codes (attic #324, attic #52) - web site: add related projects, fix web site url, IRC #borgbackup - Fedora/Fedora-based install instructions added to docs - Cygwin-based install instructions added to docs - updated AUTHORS - add FAQ entries about redundancy / integrity - clarify that borg extract uses the cwd as extraction target - update internals doc about chunker params, memory usage and compression - added docs about development - add some words about resource usage in general - document how to backup a raw disk - add note about how to run borg from virtual env - add solutions for (ll)fuse installation problems - document what borg check does, fixes #138 - reorganize borgbackup.github.io sidebar, prev/next at top - deduplicate and refactor the docs / README.rst - use borg-tmp as prefix for temporary files / directories - short prune options without "keep-" are deprecated, do not suggest them - improved tox configuration - remove usage of unittest.mock, always use mock from pypi - use entrypoints instead of scripts, for better use of the wheel format and modern installs - add requirements.d/development.txt and modify tox.ini - use travis-ci for testing based on Linux and (new) OS X - use coverage.py, pytest-cov and codecov.io for test coverage support I forgot to list some stuff already implemented in 0.23.0, here they are: New features: - efficient archive list from manifest, meaning a big speedup for slow repo connections and "list ", "delete ", "prune" (attic #242, attic #167) - big speedup for chunks cache sync (esp. for slow repo connections), fixes #18 - hashindex: improve error messages Other changes: - explicitly specify binary mode to open binary files - some easy micro optimizations Version 0.23.0 (2015-06-11) --------------------------- Incompatible changes (compared to attic, fork related): - changed sw name and cli command to "borg", updated docs - package name (and name in urls) uses "borgbackup" to have less collisions - changed repo / cache internal magic strings from ATTIC* to BORG*, changed cache location to .cache/borg/ - this means that it currently won't accept attic repos (see issue #21 about improving that) Bug fixes: - avoid defect python-msgpack releases, fixes attic #171, fixes attic #185 - fix traceback when trying to do unsupported passphrase change, fixes attic #189 - datetime does not like the year 10.000, fixes attic #139 - fix "info" all archives stats, fixes attic #183 - fix parsing with missing microseconds, fixes attic #282 - fix misleading hint the fuse ImportError handler gave, fixes attic #237 - check unpacked data from RPC for tuple type and correct length, fixes attic #127 - fix Repository._active_txn state when lock upgrade fails - give specific path to xattr.is_enabled(), disable symlink setattr call that always fails - fix test setup for 32bit platforms, partial fix for attic #196 - upgraded versioneer, PEP440 compliance, fixes attic #257 New features: - less memory usage: add global option --no-cache-files - check --last N (only check the last N archives) - check: sort archives in reverse time order - rename repo::oldname newname (rename repository) - create -v output more informative - create --progress (backup progress indicator) - create --timestamp (utc string or reference file/dir) - create: if "-" is given as path, read binary from stdin - extract: if --stdout is given, write all extracted binary data to stdout - extract --sparse (simple sparse file support) - extra debug information for 'fread failed' - delete (deletes whole repo + local cache) - FUSE: reflect deduplication in allocated blocks - only allow whitelisted RPC calls in server mode - normalize source/exclude paths before matching - use posix_fadvise to not spoil the OS cache, fixes attic #252 - toplevel error handler: show tracebacks for better error analysis - sigusr1 / sigint handler to print current file infos - attic PR #286 - RPCError: include the exception args we get from remote Other changes: - source: misc. cleanups, pep8, style - docs and faq improvements, fixes, updates - cleanup crypto.pyx, make it easier to adapt to other AES modes - do os.fsync like recommended in the python docs - source: Let chunker optionally work with os-level file descriptor. - source: Linux: remove duplicate os.fsencode calls - source: refactor _open_rb code a bit, so it is more consistent / regular - source: refactor indicator (status) and item processing - source: use py.test for better testing, flake8 for code style checks - source: fix tox >=2.0 compatibility (test runner) - pypi package: add python version classifiers, add FreeBSD to platforms Attic Changelog --------------- Here you can see the full list of changes between each Attic release until Borg forked from Attic: Version 0.17 ~~~~~~~~~~~~ (bugfix release, released on X) - Fix hashindex ARM memory alignment issue (#309) - Improve hashindex error messages (#298) Version 0.16 ~~~~~~~~~~~~ (bugfix release, released on May 16, 2015) - Fix typo preventing the security confirmation prompt from working (#303) - Improve handling of systems with improperly configured file system encoding (#289) - Fix "All archives" output for attic info. (#183) - More user friendly error message when repository key file is not found (#236) - Fix parsing of iso 8601 timestamps with zero microseconds (#282) Version 0.15 ~~~~~~~~~~~~ (bugfix release, released on Apr 15, 2015) - xattr: Be less strict about unknown/unsupported platforms (#239) - Reduce repository listing memory usage (#163). - Fix BrokenPipeError for remote repositories (#233) - Fix incorrect behavior with two character directory names (#265, #268) - Require approval before accessing relocated/moved repository (#271) - Require approval before accessing previously unknown unencrypted repositories (#271) - Fix issue with hash index files larger than 2GB. - Fix Python 3.2 compatibility issue with noatime open() (#164) - Include missing pyx files in dist files (#168) Version 0.14 ~~~~~~~~~~~~ (feature release, released on Dec 17, 2014) - Added support for stripping leading path segments (#95) "attic extract --strip-segments X" - Add workaround for old Linux systems without acl_extended_file_no_follow (#96) - Add MacPorts' path to the default openssl search path (#101) - HashIndex improvements, eliminates unnecessary IO on low memory systems. - Fix "Number of files" output for attic info. (#124) - limit create file permissions so files aren't read while restoring - Fix issue with empty xattr values (#106) Version 0.13 ~~~~~~~~~~~~ (feature release, released on Jun 29, 2014) - Fix sporadic "Resource temporarily unavailable" when using remote repositories - Reduce file cache memory usage (#90) - Faster AES encryption (utilizing AES-NI when available) - Experimental Linux, OS X and FreeBSD ACL support (#66) - Added support for backup and restore of BSDFlags (OSX, FreeBSD) (#56) - Fix bug where xattrs on symlinks were not correctly restored - Added cachedir support. CACHEDIR.TAG compatible cache directories can now be excluded using ``--exclude-caches`` (#74) - Fix crash on extreme mtime timestamps (year 2400+) (#81) - Fix Python 3.2 specific lockf issue (EDEADLK) Version 0.12 ~~~~~~~~~~~~ (feature release, released on April 7, 2014) - Python 3.4 support (#62) - Various documentation improvements a new style - ``attic mount`` now supports mounting an entire repository not only individual archives (#59) - Added option to restrict remote repository access to specific path(s): ``attic serve --restrict-to-path X`` (#51) - Include "all archives" size information in "--stats" output. (#54) - Added ``--stats`` option to ``attic delete`` and ``attic prune`` - Fixed bug where ``attic prune`` used UTC instead of the local time zone when determining which archives to keep. - Switch to SI units (Power of 1000 instead 1024) when printing file sizes Version 0.11 ~~~~~~~~~~~~ (feature release, released on March 7, 2014) - New "check" command for repository consistency checking (#24) - Documentation improvements - Fix exception during "attic create" with repeated files (#39) - New "--exclude-from" option for attic create/extract/verify. - Improved archive metadata deduplication. - "attic verify" has been deprecated. Use "attic extract --dry-run" instead. - "attic prune --hourly|daily|..." has been deprecated. Use "attic prune --keep-hourly|daily|..." instead. - Ignore xattr errors during "extract" if not supported by the filesystem. (#46) Version 0.10 ~~~~~~~~~~~~ (bugfix release, released on Jan 30, 2014) - Fix deadlock when extracting 0 sized files from remote repositories - "--exclude" wildcard patterns are now properly applied to the full path not just the file name part (#5). - Make source code endianness agnostic (#1) Version 0.9 ~~~~~~~~~~~ (feature release, released on Jan 23, 2014) - Remote repository speed and reliability improvements. - Fix sorting of segment names to ignore NFS left over files. (#17) - Fix incorrect display of time (#13) - Improved error handling / reporting. (#12) - Use fcntl() instead of flock() when locking repository/cache. (#15) - Let ssh figure out port/user if not specified so we don't override .ssh/config (#9) - Improved libcrypto path detection (#23). Version 0.8.1 ~~~~~~~~~~~~~ (bugfix release, released on Oct 4, 2013) - Fix segmentation fault issue. Version 0.8 ~~~~~~~~~~~ (feature release, released on Oct 3, 2013) - Fix xattr issue when backing up sshfs filesystems (#4) - Fix issue with excessive index file size (#6) - Support access of read only repositories. - New syntax to enable repository encryption: attic init --encryption="none|passphrase|keyfile". - Detect and abort if repository is older than the cache. Version 0.7 ~~~~~~~~~~~ (feature release, released on Aug 5, 2013) - Ported to FreeBSD - Improved documentation - Experimental: Archives mountable as fuse filesystems. - The "user." prefix is no longer stripped from xattrs on Linux Version 0.6.1 ~~~~~~~~~~~~~ (bugfix release, released on July 19, 2013) - Fixed an issue where mtime was not always correctly restored. Version 0.6 ~~~~~~~~~~~ First public release on July 9, 2013 borgbackup-1.0.13/docs/conf.py0000644000175000017500000002033413431513133016117 0ustar useruser00000000000000# -*- coding: utf-8 -*- # # documentation build configuration file, created by # sphinx-quickstart on Sat Sep 10 18:18:25 2011. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. import sys, os sys.path.insert(0, os.path.abspath('..')) from borg import __version__ as sw_version on_rtd = os.environ.get('READTHEDOCS', None) == 'True' # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = 'Borg - Deduplicating Archiver' copyright = '2010-2014 Jonas Borgström, 2015-2019 The Borg Collective (see AUTHORS file)' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = sw_version.split('-')[0] # The full version, including alpha/beta/rc tags. release = version suppress_warnings = ['image.nonlocal_uri'] # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. today_fmt = '%Y-%m-%d' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. #html_theme = '' if not on_rtd: # only import and set the theme if we're building docs locally import sphinx_rtd_theme html_theme = 'sphinx_rtd_theme' html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] html_style = 'css/borg.css' else: html_context = { 'css_files': [ 'https://media.readthedocs.org/css/sphinx_rtd_theme.css', 'https://media.readthedocs.org/css/readthedocs-doc-embed.css', '_static/css/borg.css', ], } # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = ['_themes'] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. html_logo = '_static/logo.png' # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. html_favicon = '_static/favicon.ico' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['borg_theme'] html_extra_path = ['../borg/paperkey.html'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. html_last_updated_fmt = '%Y-%m-%d' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. html_sidebars = { 'index': ['sidebarlogo.html', 'sidebarusefullinks.html', 'searchbox.html'], '**': ['sidebarlogo.html', 'relations.html', 'searchbox.html', 'localtoc.html', 'sidebarusefullinks.html'] } # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. html_use_index = False # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. html_show_sourcelink = False # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. html_show_sphinx = False # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. html_show_copyright = False # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'borgdoc' # -- Options for LaTeX output -------------------------------------------------- # The paper size ('letter' or 'a4'). #latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). #latex_font_size = '12pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('book', 'Borg.tex', 'Borg Documentation', 'The Borg Collective', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. latex_logo = '_static/logo.png' # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. latex_show_urls = 'footnote' # Additional stuff for the LaTeX preamble. #latex_preamble = '' # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('usage', 'borg', 'BorgBackup is a deduplicating backup program with optional compression and authenticated encryption.', ['The Borg Collective (see AUTHORS file)'], 1), ] extensions = ['sphinx.ext.extlinks', 'sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.viewcode'] extlinks = { 'issue': ('https://github.com/borgbackup/borg/issues/%s', '#'), 'targz_url': ('https://pypi.python.org/packages/source/b/borgbackup/%%s-%s.tar.gz' % version, None), } borgbackup-1.0.13/docs/deployment.rst0000644000175000017500000001537013431513133017536 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: none .. _deployment: Deployment ========== This chapter will give an example how to setup a borg repository server for multiple clients. Machines -------- There are multiple machines used in this chapter and will further be named by their respective fully qualified domain name (fqdn). * The backup server: `backup01.srv.local` * The clients: - John Doe's desktop: `johndoe.clnt.local` - Webserver 01: `web01.srv.local` - Application server 01: `app01.srv.local` User and group -------------- The repository server needs to have only one UNIX user for all the clients. Recommended user and group with additional settings: * User: `backup` * Group: `backup` * Shell: `/bin/bash` (or other capable to run the `borg serve` command) * Home: `/home/backup` Most clients shall initiate a backup from the root user to catch all users, groups and permissions (e.g. when backing up `/home`). Folders ------- The following folder tree layout is suggested on the repository server: * User home directory, /home/backup * Repositories path (storage pool): /home/backup/repos * Clients restricted paths (`/home/backup/repos/`): - johndoe.clnt.local: `/home/backup/repos/johndoe.clnt.local` - web01.srv.local: `/home/backup/repos/web01.srv.local` - app01.srv.local: `/home/backup/repos/app01.srv.local` Restrictions ------------ Borg is instructed to restrict clients into their own paths: ``borg serve --restrict-to-path /home/backup/repos/`` The client will be able to access any file or subdirectory inside of ``/home/backup/repos/`` but no other directories. You can allow a client to access several separate directories by passing multiple `--restrict-to-path` flags, for instance: ``borg serve --restrict-to-path /home/backup/repos/ --restrict-to-path /home/backup/repos/``, which could make sense if multiple machines belong to one person which should then have access to all the backups of their machines. There is only one ssh key per client allowed. Keys are added for ``johndoe.clnt.local``, ``web01.srv.local`` and ``app01.srv.local``. But they will access the backup under only one UNIX user account as: ``backup@backup01.srv.local``. Every key in ``$HOME/.ssh/authorized_keys`` has a forced command and restrictions applied as shown below: :: command="cd /home/backup/repos/; borg serve --restrict-to-path /home/backup/repos/", no-port-forwarding,no-X11-forwarding,no-pty, no-agent-forwarding,no-user-rc .. note:: The text shown above needs to be written on a single line! The options which are added to the key will perform the following: 1. Change working directory 2. Run ``borg serve`` restricted to the client base path 3. Restrict ssh and do not allow stuff which imposes a security risk Due to the ``cd`` command we use, the server automatically changes the current working directory. Then client doesn't need to have knowledge of the absolute or relative remote repository path and can directly access the repositories at ``@:``. .. note:: The setup above ignores all client given commandline parameters which are normally appended to the `borg serve` command. Client ------ The client needs to initialize the `pictures` repository like this: borg init backup@backup01.srv.local:pictures Or with the full path (should actually never be used, as only for demonstrational purposes). The server should automatically change the current working directory to the `` folder. borg init backup@backup01.srv.local:/home/backup/repos/johndoe.clnt.local/pictures When `johndoe.clnt.local` tries to access a not restricted path the following error is raised. John Doe tries to backup into the Web 01 path: borg init backup@backup01.srv.local:/home/backup/repos/web01.srv.local/pictures :: ~~~ SNIP ~~~ Remote: borg.remote.PathNotAllowed: /home/backup/repos/web01.srv.local/pictures ~~~ SNIP ~~~ Repository path not allowed Ansible ------- Ansible takes care of all the system-specific commands to add the user, create the folder. Even when the configuration is changed the repository server configuration is satisfied and reproducible. Automate setting up an repository server with the user, group, folders and permissions a Ansible playbook could be used. Keep in mind the playbook uses the Arch Linux `pacman `_ package manager to install and keep borg up-to-date. :: - hosts: backup01.srv.local vars: user: backup group: backup home: /home/backup pool: "{{ home }}/repos" auth_users: - host: johndoe.clnt.local key: "{{ lookup('file', '/path/to/keys/johndoe.clnt.local.pub') }}" - host: web01.clnt.local key: "{{ lookup('file', '/path/to/keys/web01.clnt.local.pub') }}" - host: app01.clnt.local key: "{{ lookup('file', '/path/to/keys/app01.clnt.local.pub') }}" tasks: - pacman: name=borg state=latest update_cache=yes - group: name="{{ group }}" state=present - user: name="{{ user }}" shell=/bin/bash home="{{ home }}" createhome=yes group="{{ group }}" groups= state=present - file: path="{{ home }}" owner="{{ user }}" group="{{ group }}" mode=0700 state=directory - file: path="{{ home }}/.ssh" owner="{{ user }}" group="{{ group }}" mode=0700 state=directory - file: path="{{ pool }}" owner="{{ user }}" group="{{ group }}" mode=0700 state=directory - authorized_key: user="{{ user }}" key="{{ item.key }}" key_options='command="cd {{ pool }}/{{ item.host }};borg serve --restrict-to-path {{ pool }}/{{ item.host }}",no-port-forwarding,no-X11-forwarding,no-pty,no-agent-forwarding,no-user-rc' with_items: "{{ auth_users }}" - file: path="{{ home }}/.ssh/authorized_keys" owner="{{ user }}" group="{{ group }}" mode=0600 state=file - file: path="{{ pool }}/{{ item.host }}" owner="{{ user }}" group="{{ group }}" mode=0700 state=directory with_items: "{{ auth_users }}" Enhancements ------------ As this chapter only describes a simple and effective setup it could be further enhanced when supporting (a limited set) of client supplied commands. A wrapper for starting `borg serve` could be written. Or borg itself could be enhanced to autodetect it runs under SSH by checking the `SSH_ORIGINAL_COMMAND` environment variable. This is left open for future improvements. When extending ssh autodetection in borg no external wrapper script is necessary and no other interpreter or application has to be deployed. See also -------- * `SSH Daemon manpage `_ * `Ansible `_ borgbackup-1.0.13/docs/development.rst0000644000175000017500000002352213431513133017676 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: bash .. _development: Development =========== This chapter will get you started with |project_name| development. |project_name| is written in Python (with a little bit of Cython and C for the performance critical parts). Contributions ------------- ... are welcome! Some guidance for contributors: - discuss about changes on github issue tracker, IRC or mailing list - make your PRs on the ``master`` branch (see `Branching Model`_ for details) - do clean changesets: - focus on some topic, resist changing anything else. - do not do style changes mixed with functional changes. - try to avoid refactorings mixed with functional changes. - if you need to fix something after commit/push: - if there are ongoing reviews: do a fixup commit you can merge into the bad commit later. - if there are no ongoing reviews or you did not push the bad commit yet: edit the commit to include your fix or merge the fixup commit before pushing. - have a nice, clear, typo-free commit comment - if you fixed an issue, refer to it in your commit comment - follow the style guide (see below) - if you write new code, please add tests and docs for it - run the tests, fix anything that comes up - make a pull request on github - wait for review by other developers Branching model --------------- Borg development happens on the ``master`` branch and uses GitHub pull requests (if you don't have GitHub or don't want to use it you can send smaller patches via the borgbackup :ref:`mailing_list` to the maintainers). Stable releases are maintained on maintenance branches named x.y-maint, eg. the maintenance branch of the 1.0.x series is 1.0-maint. Most PRs should be made against the ``master`` branch. Only if an issue affects **only** a particular maintenance branch a PR should be made against it directly. While discussing / reviewing a PR it will be decided whether the change should be applied to maintenance branch(es). Each maintenance branch has a corresponding *backport/x.y-maint* label, which will then be applied. Changes that are typically considered for backporting: - Data loss, corruption and inaccessibility fixes - Security fixes - Forward-compatibility improvements - Documentation corrections .. rubric:: Maintainer part From time to time a maintainer will backport the changes for a maintenance branch, typically before a release or if enough changes were collected: 1. Notify others that you're doing this to avoid duplicate work. 2. Branch a backporting branch off the maintenance branch. 3. Cherry pick and backport the changes from each labelled PR, remove the label for each PR you've backported. 4. Make a PR of the backporting branch against the maintenance branch for backport review. Mention the backported PRs in this PR, eg: Includes changes from #2055 #2057 #2381 This way GitHub will automatically show in these PRs where they were backported. .. rubric:: Historic model Previously (until release 1.0.10) Borg used a `"merge upwards" `_ model where most minor changes and fixes where committed to a maintenance branch (eg. 1.0-maint), and the maintenance branch(es) were regularly merged back into the main development branch. This became more and more troublesome due to merges growing more conflict-heavy and error-prone. Code and issues --------------- Code is stored on Github, in the `Borgbackup organization `_. `Issues `_ and `pull requests `_ should be sent there as well. See also the :ref:`support` section for more details. Style guide ----------- We generally follow `pep8 `_, with 120 columns instead of 79. We do *not* use form-feed (``^L``) characters to separate sections either. Compliance is tested automatically when you run the tests. Continuous Integration ---------------------- All pull requests go through Travis-CI_, which runs the tests on Linux and Mac OS X as well as the flake8 style checker. Additional Unix-like platforms are tested on Golem_. .. _Golem: https://golem.enkore.de/view/Borg/ .. _Travis-CI: https://travis-ci.org/borgbackup/borg Output and Logging ------------------ When writing logger calls, always use correct log level (debug only for debugging, info for informative messages, warning for warnings, error for errors, critical for critical errors/states). When directly talking to the user (e.g. Y/N questions), do not use logging, but directly output to stderr (not: stdout, it could be connected to a pipe). To control the amount and kinds of messages output to stderr or emitted at info level, use flags like ``--stats`` or ``--list``. Building a development environment ---------------------------------- First, just install borg into a virtual env as described before. To install some additional packages needed for running the tests, activate your virtual env and run:: pip install -r requirements.d/development.txt Running the tests ----------------- The tests are in the borg/testsuite package. To run all the tests, you need to have fakeroot installed. If you do not have fakeroot, you still will be able to run most tests, just leave away the `fakeroot -u` from the given command lines. To run the test suite use the following command:: fakeroot -u tox # run all tests Some more advanced examples:: # verify a changed tox.ini (run this after any change to tox.ini): fakeroot -u tox --recreate fakeroot -u tox -e py34 # run all tests, but only on python 3.4 fakeroot -u tox borg.testsuite.locking # only run 1 test module fakeroot -u tox borg.testsuite.locking -- -k '"not Timer"' # exclude some tests fakeroot -u tox borg.testsuite -- -v # verbose py.test Important notes: - When using ``--`` to give options to py.test, you MUST also give ``borg.testsuite[.module]``. Regenerate usage files ---------------------- Usage documentation is currently committed directly to git, although those files are generated automatically from the source tree. When a command is added, a commandline flag changed, added or removed, the usage docs need to be rebuilt as well:: ./setup.py build_usage However, we prefer to do this as part of our :ref:`releasing` preparations, so it is generally not necessary to update these when submitting patches that change something about the command line. Building the docs with Sphinx ----------------------------- The documentation (in reStructuredText format, .rst) is in docs/. To build the html version of it, you need to have sphinx installed:: pip3 install sphinx sphinx_rtd_theme # important: this will install sphinx with Python 3 Now run:: cd docs/ make html Then point a web browser at docs/_build/html/index.html. The website is updated automatically through Github web hooks on the main repository. Using Vagrant ------------- We use Vagrant for the automated creation of testing environments and borgbackup standalone binaries for various platforms. For better security, there is no automatic sync in the VM to host direction. The plugin `vagrant-scp` is useful to copy stuff from the VMs to the host. Usage:: # To create and provision the VM: vagrant up OS # To create an ssh session to the VM: vagrant ssh OS command # To shut down the VM: vagrant halt OS # To shut down and destroy the VM: vagrant destroy OS # To copy files from the VM (in this case, the generated binary): vagrant scp OS:/vagrant/borg/borg.exe . Creating standalone binaries ---------------------------- Make sure you have everything built and installed (including llfuse and fuse). When using the Vagrant VMs, pyinstaller will already be installed. With virtual env activated:: pip install pyinstaller # or git checkout master pyinstaller -F -n borg-PLATFORM borg/__main__.py for file in dist/borg-*; do gpg --armor --detach-sign $file; done If you encounter issues, see also our `Vagrantfile` for details. .. note:: Standalone binaries built with pyinstaller are supposed to work on same OS, same architecture (x86 32bit, amd64 64bit) without external dependencies. Creating a new release ---------------------- Checklist: - make sure all issues for this milestone are closed or moved to the next milestone - check if there are any pending fixes for security issues - find and fix any low hanging fruit left on the issue tracker - check that Travis CI is happy - update ``CHANGES.rst``, based on ``git log $PREVIOUS_RELEASE..`` - check version number of upcoming release in ``CHANGES.rst`` - verify that ``MANIFEST.in`` and ``setup.py`` are complete - ``python setup.py build_usage`` and commit - tag the release:: git tag -s -m "tagged/signed release X.Y.Z" X.Y.Z - create a clean repo and use it for the following steps:: git clone borg borg-clean This makes sure no uncommitted files get into the release archive. It also will find if you forgot to commit something that is needed. It also makes sure the vagrant machines only get committed files and do a fresh start based on that. - run tox and/or binary builds on all supported platforms via vagrant, check for test failures - create sdist, sign it, upload release to (test) PyPi:: scripts/sdist-sign X.Y.Z scripts/upload-pypi X.Y.Z test scripts/upload-pypi X.Y.Z - put binaries into dist/borg-OSNAME and sign them: scripts/sign-binaries 201912312359 - close release milestone on Github - announce on: - Mailing list - Twitter - IRC channel (change ``/topic``) - create a Github release, include: * standalone binaries (see above for how to create them) + for OS X, document the OS X Fuse version in the README of the binaries. OS X FUSE uses a kernel extension that needs to be compatible with the code contained in the binary. * a link to ``CHANGES.rst`` borgbackup-1.0.13/docs/faq.rst0000644000175000017500000006740313431513133016131 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: none .. _faq: Frequently asked questions ========================== Can I backup VM disk images? ---------------------------- Yes, the `deduplication`_ technique used by |project_name| makes sure only the modified parts of the file are stored. Also, we have optional simple sparse file support for extract. If you use non-snapshotting backup tools like Borg to back up virtual machines, then the VMs should be turned off for the duration of the backup. Backing up live VMs can (and will) result in corrupted or inconsistent backup contents: a VM image is just a regular file to Borg with the same issues as regular files when it comes to concurrent reading and writing from the same file. For backing up live VMs use file system snapshots on the VM host, which establishes crash-consistency for the VM images. This means that with most file systems (that are journaling) the FS will always be fine in the backup (but may need a journal replay to become accessible). Usually this does not mean that file *contents* on the VM are consistent, since file contents are normally not journaled. Notable exceptions are ext4 in data=journal mode, ZFS and btrfs (unless nodatacow is used). Applications designed with crash-consistency in mind (most relational databases like PostgreSQL, SQLite etc. but also for example Borg repositories) should always be able to recover to a consistent state from a backup created with crash-consistent snapshots (even on ext4 with data=writeback or XFS). Hypervisor snapshots capturing most of the VM's state can also be used for backups and can be a better alternative to pure file system based snapshots of the VM's disk, since no state is lost. Depending on the application this can be the easiest and most reliable way to create application-consistent backups. Other applications may require a lot of work to reach application-consistency: It's a broad and complex issue that cannot be explained in entirety here. Borg doesn't intend to address these issues due to their huge complexity and platform/software dependency. Combining Borg with the mechanisms provided by the platform (snapshots, hypervisor features) will be the best approach to start tackling them. Can I backup from multiple servers into a single repository? ------------------------------------------------------------ Yes, but in order for the deduplication used by |project_name| to work, it needs to keep a local cache containing checksums of all file chunks already stored in the repository. This cache is stored in ``~/.cache/borg/``. If |project_name| detects that a repository has been modified since the local cache was updated it will need to rebuild the cache. This rebuild can be quite time consuming. So, yes it's possible. But it will be most efficient if a single repository is only modified from one place. Also keep in mind that |project_name| will keep an exclusive lock on the repository while creating or deleting archives, which may make *simultaneous* backups fail. Can I copy or synchronize my repo to another location? ------------------------------------------------------ Yes, you could just copy all the files. Make sure you do that while no backup is running. So what you get here is this: - client machine ---borg create---> repo1 - repo1 ---copy---> repo2 There is no special borg command to do the copying, just use cp or rsync if you want to do that. But think about whether that is really what you want. If something goes wrong in repo1, you will have the same issue in repo2 after the copy. If you want to have 2 independent backups, it is better to do it like this: - client machine ---borg create---> repo1 - client machine ---borg create---> repo2 Which file types, attributes, etc. are *not* preserved? ------------------------------------------------------- * UNIX domain sockets (because it does not make sense - they are meaningless without the running process that created them and the process needs to recreate them in any case). So, don't panic if your backup misses a UDS! * The precise on-disk (or rather: not-on-disk) representation of the holes in a sparse file. Archive creation has no special support for sparse files, holes are backed up as (deduplicated and compressed) runs of zero bytes. Archive extraction has optional support to extract all-zero chunks as holes in a sparse file. * filesystem specific attributes, like ext4 immutable bit, see :issue:`618`. * For hardlinked symlinks, the hardlinking can not be archived (and thus, the hardlinking will not be done at extraction time). The symlinks will be archived and extracted as non-hardlinked symlinks, see :issue:`2379`. Are there other known limitations? ---------------------------------- - A single archive can only reference a limited volume of file/dir metadata, usually corresponding to tens or hundreds of millions of files/dirs. When trying to go beyond that limit, you will get a fatal IntegrityError exception telling that the (archive) object is too big. An easy workaround is to create multiple archives with less items each. See also the :ref:`archive_limitation` and :issue:`1452`. Why is my backup bigger than with attic? Why doesn't |project_name| do compression by default? ---------------------------------------------------------------------------------------------- Attic was rather unflexible when it comes to compression, it always compressed using zlib level 6 (no way to switch compression off or adjust the level or algorithm). |project_name| offers a lot of different compression algorithms and levels. Which of them is the best for you pretty much depends on your use case, your data, your hardware -- so you need to do an informed decision about whether you want to use compression, which algorithm and which level you want to use. This is why compression defaults to none. How can I specify the encryption passphrase programmatically? ------------------------------------------------------------- The encryption passphrase or a command to retrieve the passphrase can be specified programmatically using the `BORG_PASSPHRASE` or `BORG_PASSCOMMAND` environment variables. This is convenient when setting up automated encrypted backups. Another option is to use key file based encryption with a blank passphrase. See :ref:`encrypted_repos` for more details. .. _password_env: .. note:: Be careful how you set the environment; using the ``env`` command, a ``system()`` call or using inline shell scripts (e.g. ``BORG_PASSPHRASE=hunter12 borg ...``) might expose the credentials in the process list directly and they will be readable to all users on a system. Using ``export`` in a shell script file should be safe, however, as the environment of a process is `accessible only to that user `_. When backing up to remote encrypted repos, is encryption done locally? ---------------------------------------------------------------------- Yes, file and directory metadata and data is locally encrypted, before leaving the local machine. We do not mean the transport layer encryption by that, but the data/metadata itself. Transport layer encryption (e.g. when ssh is used as a transport) applies additionally. When backing up to remote servers, do I have to trust the remote server? ------------------------------------------------------------------------ Yes and No. No, as far as data confidentiality is concerned - if you use encryption, all your files/dirs data and metadata are stored in their encrypted form into the repository. Yes, as an attacker with access to the remote server could delete (or otherwise make unavailable) all your backups. How can I protect against a hacked backup client? ------------------------------------------------- Assume you backup your backup client machine C to the backup server S and C gets hacked. In a simple push setup, the attacker could then use borg on C to delete all backups residing on S. These are your options to protect against that: - Do not allow to permanently delete data from the repo, see :ref:`append_only_mode`. - Use a pull-mode setup using ``ssh -R``, see :issue:`900`. - Mount C's filesystem on another machine and then create a backup of it. - Do not give C filesystem-level access to S. How can I protect against a hacked backup server? ------------------------------------------------- Just in case you got the impression that pull-mode backups are way more safe than push-mode, you also need to consider the case that your backup server S gets hacked. In case S has access to a lot of clients C, that might bring you into even bigger trouble than a hacked backup client in the previous FAQ entry. These are your options to protect against that: - Use the standard push-mode setup (see also previous FAQ entry). - Mount (the repo part of) S's filesystem on C. - Do not give S file-system level access to C. - Have your backup server at a well protected place (maybe not reachable from the internet), configure it safely, apply security updates, monitor it, ... How can I protect against theft, sabotage, lightning, fire, ...? ---------------------------------------------------------------- In general: if your only backup medium is nearby the backupped machine and always connected, you can easily get into trouble: they likely share the same fate if something goes really wrong. Thus: - have multiple backup media - have media disconnected from network, power, computer - have media at another place - have a relatively recent backup on your media How do I report security issue with |project_name|? --------------------------------------------------- Send a private email to the :ref:`security-contact` if you think you have discovered a security issue. Please disclose security issues responsibly. Why do I get "connection closed by remote" after a while? --------------------------------------------------------- When doing a backup to a remote server (using a ssh: repo URL), it sometimes stops after a while (some minutes, hours, ... - not immediately) with "connection closed by remote" error message. Why? That's a good question and we are trying to find a good answer in :issue:`636`. Why am I seeing idle borg serve processes on the repo server? ------------------------------------------------------------- Maybe the ssh connection between client and server broke down and that was not yet noticed on the server. Try these settings: :: # /etc/ssh/sshd_config on borg repo server - kill connection to client # after ClientAliveCountMax * ClientAliveInterval seconds with no response ClientAliveInterval 20 ClientAliveCountMax 3 If you have multiple borg create ... ; borg create ... commands in a already serialized way in a single script, you need to give them --lock-wait N (with N being a bit more than the time the server needs to terminate broken down connections and release the lock). The borg cache eats way too much disk space, what can I do? ----------------------------------------------------------- There is a temporary (but maybe long lived) hack to avoid using lots of disk space for chunks.archive.d (see :issue:`235` for details): :: # this assumes you are working with the same user as the backup. # you can get the REPOID from the "config" file inside the repository. cd ~/.cache/borg/ rm -rf chunks.archive.d ; touch chunks.archive.d This deletes all the cached archive chunk indexes and replaces the directory that kept them with a file, so borg won't be able to store anything "in" there in future. This has some pros and cons, though: - much less disk space needs for ~/.cache/borg. - chunk cache resyncs will be slower as it will have to transfer chunk usage metadata for all archives from the repository (which might be slow if your repo connection is slow) and it will also have to build the hashtables from that data. chunk cache resyncs happen e.g. if your repo was written to by another machine (if you share same backup repo between multiple machines) or if your local chunks cache was lost somehow. The long term plan to improve this is called "borgception", see :issue:`474`. If a backup stops mid-way, does the already-backed-up data stay there? ---------------------------------------------------------------------- Yes, |project_name| supports resuming backups. During a backup a special checkpoint archive named ``.checkpoint`` is saved every checkpoint interval (the default value for this is 5 minutes) containing all the data backed-up until that point. Checkpoints only happen between files (so they don't help for interruptions happening while a very large file is being processed). This checkpoint archive is a valid archive (all files in it are valid and complete), but it is only a partial backup (not all files that you wanted to backup are contained in it). Having it in the repo until a successful, full backup is completed is useful because it references all the transmitted chunks up to the checkpoint. This means that in case of an interruption, you only need to retransfer the data since the last checkpoint. If a backup was interrupted, you do not need to do any special considerations, just invoke ``borg create`` as you always do. You may use the same archive name as in previous attempt or a different one (e.g. if you always include the current datetime), it does not matter. |project_name| always does full single-pass backups, so it will start again from the beginning - but it will be much faster, because some of the data was already stored into the repo (and is still referenced by the checkpoint archive), so it does not need to get transmitted and stored again. Once your backup has finished successfully, you can delete all ``.checkpoint`` archives. How can I backup huge file(s) over a unstable connection? --------------------------------------------------------- You can use this "split trick" as a workaround for the in-between-files-only checkpoints (see above), huge files and a instable connection to the repository: Split the huge file(s) into parts of manageable size (e.g. 100MB) and create a temporary archive of them. Borg will create checkpoints now more frequently than if you try to backup the files in their original form (e.g. 100GB). After that, you can remove the parts again and backup the huge file(s) in their original form. This will now work a lot faster as a lot of content chunks are already in the repository. After you have successfully backed up the huge original file(s), you can remove the temporary archive you made from the parts. We realize that this is just a better-than-nothing workaround, see :issue:`1198` for a potential solution. Please note that this workaround only helps you for backup, not for restore. Can I backup my root partition (/) with Borg? --------------------------------------------- Backing up your entire root partition works just fine, but remember to exclude directories that make no sense to backup, such as /dev, /proc, /sys, /tmp and /run. If it crashes with a UnicodeError, what can I do? ------------------------------------------------- Check if your encoding is set correctly. For most POSIX-like systems, try:: export LANG=en_US.UTF-8 # or similar, important is correct charset I can't extract non-ascii filenames by giving them on the commandline!? ----------------------------------------------------------------------- This might be due to different ways to represent some characters in unicode or due to other non-ascii encoding issues. If you run into that, try this: - avoid the non-ascii characters on the commandline by e.g. extracting the parent directory (or even everything) - mount the repo using FUSE and use some file manager Can |project_name| add redundancy to the backup data to deal with hardware malfunction? --------------------------------------------------------------------------------------- No, it can't. While that at first sounds like a good idea to defend against some defect HDD sectors or SSD flash blocks, dealing with this in a reliable way needs a lot of low-level storage layout information and control which we do not have (and also can't get, even if we wanted). So, if you need that, consider RAID or a filesystem that offers redundant storage or just make backups to different locations / different hardware. See also :issue:`225`. Can |project_name| verify data integrity of a backup archive? ------------------------------------------------------------- Yes, if you want to detect accidental data damage (like bit rot), use the ``check`` operation. It will notice corruption using CRCs and hashes. If you want to be able to detect malicious tampering also, use an encrypted repo. It will then be able to check using CRCs and HMACs. .. _a_status_oddity: I am seeing 'A' (added) status for a unchanged file!? ----------------------------------------------------- The files cache is used to determine whether |project_name| already "knows" / has backed up a file and if so, to skip the file from chunking. It does intentionally *not* contain files that have a modification time (mtime) same as the newest mtime in the created archive. So, if you see an 'A' status for unchanged file(s), they are likely the files with the most recent mtime in that archive. This is expected: it is to avoid data loss with files that are backed up from a snapshot and that are immediately changed after the snapshot (but within mtime granularity time, so the mtime would not change). Without the code that removes these files from the files cache, the change that happened right after the snapshot would not be contained in the next backup as |project_name| would think the file is unchanged. This does not affect deduplication, the file will be chunked, but as the chunks will often be the same and already stored in the repo (except in the above mentioned rare condition), it will just re-use them as usual and not store new data chunks. If you want to avoid unnecessary chunking, just create or touch a small or empty file in your backup source file set (so that one has the latest mtime, not your 50GB VM disk image) and, if you do snapshots, do the snapshot after that. Since only the files cache is used in the display of files status, those files are reported as being added when, really, chunks are already used. .. _always_chunking: It always chunks all my files, even unchanged ones! --------------------------------------------------- |project_name| maintains a files cache where it remembers the mtime, size and inode of files. When |project_name| does a new backup and starts processing a file, it first looks whether the file has changed (compared to the values stored in the files cache). If the values are the same, the file is assumed unchanged and thus its contents won't get chunked (again). |project_name| can't keep an infinite history of files of course, thus entries in the files cache have a "maximum time to live" which is set via the environment variable BORG_FILES_CACHE_TTL (and defaults to 20). Every time you do a backup (on the same machine, using the same user), the cache entries' ttl values of files that were not "seen" are incremented by 1 and if they reach BORG_FILES_CACHE_TTL, the entry is removed from the cache. So, for example, if you do daily backups of 26 different data sets A, B, C, ..., Z on one machine (using the default TTL), the files from A will be already forgotten when you repeat the same backups on the next day and it will be slow because it would chunk all the files each time. If you set BORG_FILES_CACHE_TTL to at least 26 (or maybe even a small multiple of that), it would be much faster. Another possible reason is that files don't always have the same path, for example if you mount a filesystem without stable mount points for each backup or if you are running the backup from a filesystem snapshot whose name is not stable. If the directory where you mount a filesystem is different every time, |project_name| assume they are different files. Is there a way to limit bandwidth with |project_name|? ------------------------------------------------------ There is no command line option to limit bandwidth with |project_name|, but bandwidth limiting can be accomplished with pipeviewer_: Create a wrapper script: /usr/local/bin/pv-wrapper :: #!/bin/sh ## -q, --quiet do not output any transfer information at all ## -L, --rate-limit RATE limit transfer to RATE bytes per second RATE=307200 pv -q -L $RATE | "$@" Add BORG_RSH environment variable to use pipeviewer wrapper script with ssh. :: export BORG_RSH='/usr/local/bin/pv-wrapper ssh' Now |project_name| will be bandwidth limited. Nice thing about pv is that you can change rate-limit on the fly: :: pv -R $(pidof pv) -L 102400 .. _pipeviewer: http://www.ivarch.com/programs/pv.shtml I am having troubles with some network/FUSE/special filesystem, why? -------------------------------------------------------------------- |project_name| is doing nothing special in the filesystem, it only uses very common and compatible operations (even the locking is just "mkdir"). So, if you are encountering issues like slowness, corruption or malfunction when using a specific filesystem, please try if you can reproduce the issues with a local (non-network) and proven filesystem (like ext4 on Linux). If you can't reproduce the issue then, you maybe have found an issue within the filesystem code you used (not with |project_name|). For this case, it is recommended that you talk to the developers / support of the network fs and maybe open an issue in their issue tracker. Do not file an issue in the |project_name| issue tracker. If you can reproduce the issue with the proven filesystem, please file an issue in the |project_name| issue tracker about that. Why does running 'borg check --repair' warn about data loss? ------------------------------------------------------------ Repair usually works for recovering data in a corrupted archive. However, it's impossible to predict all modes of corruption. In some very rare instances, such as malfunctioning storage hardware, additional repo corruption may occur. If you can't afford to lose the repo, it's strongly recommended that you perform repair on a copy of the repo. In other words, the warning is there to emphasize that |project_name|: - Will perform automated routines that modify your backup repository - Might not actually fix the problem you are experiencing - Might, in very rare cases, further corrupt your repository In the case of malfunctioning hardware, such as a drive or USB hub corrupting data when read or written, it's best to diagnose and fix the cause of the initial corruption before attempting to repair the repo. If the corruption is caused by a one time event such as a power outage, running `borg check --repair` will fix most problems. Why isn't there more progress / ETA information displayed? ---------------------------------------------------------- Some borg runs take quite a bit, so it would be nice to see a progress display, maybe even including a ETA (expected time of "arrival" [here rather "completion"]). For some functionality, this can be done: if the total amount of work is more or less known, we can display progress. So check if there is a ``--progress`` option. But sometimes, the total amount is unknown (e.g. for ``borg create`` we just do a single pass over the filesystem, so we do not know the total file count or data volume before reaching the end). Adding another pass just to determine that would take additional time and could be incorrect, if the filesystem is changing. Even if the fs does not change and we knew count and size of all files, we still could not compute the ``borg create`` ETA as we do not know the amount of changed chunks, how the bandwidth of source and destination or system performance might fluctuate. You see, trying to display ETA would be futile. The borg developers prefer to rather not implement progress / ETA display than doing futile attempts. See also: https://xkcd.com/612/ Why am I getting 'Operation not permitted' errors when backing up on sshfs? --------------------------------------------------------------------------- By default, ``sshfs`` is not entirely POSIX-compliant when renaming files due to a technicality in the SFTP protocol. Fortunately, it also provides a workaround_ to make it behave correctly:: sshfs -o workaround=rename user@host:dir /mnt/dir .. _workaround: https://unix.stackexchange.com/a/123236 Requirements for the borg single-file binary, esp. (g)libc? ----------------------------------------------------------- We try to build the binary on old, but still supported systems - to keep the minimum requirement for the (g)libc low. The (g)libc can't be bundled into the binary as it needs to fit your kernel and OS, but Python and all other required libraries will be bundled into the binary. If your system fulfills the minimum (g)libc requirement (see the README that is released with the binary), there should be no problem. If you are slightly below the required version, maybe just try. Due to the dynamic loading (or not loading) of some shared libraries, it might still work depending on what libraries are actually loaded and used. In the borg git repository, there is scripts/glibc_check.py that can determine (based on the symbols' versions they want to link to) whether a set of given (Linux) binaries works with a given glibc version. Why was Borg forked from Attic? ------------------------------- Borg was created in May 2015 in response to the difficulty of getting new code or larger changes incorporated into Attic and establishing a bigger developer community / more open development. More details can be found in `ticket 217 `_ that led to the fork. Borg intends to be: * simple: * as simple as possible, but no simpler * do the right thing by default, but offer options * open: * welcome feature requests * accept pull requests of good quality and coding style * give feedback on PRs that can't be accepted "as is" * discuss openly, don't work in the dark * changing: * Borg is not compatible with Attic * do not break compatibility accidentally, without a good reason or without warning. allow compatibility breaking for other cases. * if major version number changes, it may have incompatible changes What are the differences between Attic and Borg? ------------------------------------------------ Borg is a fork of `Attic`_ and maintained by "`The Borg collective`_". .. _Attic: https://github.com/jborg/attic .. _The Borg collective: https://borgbackup.readthedocs.org/en/latest/authors.html Here's a (incomplete) list of some major changes: * more open, faster paced development (see `issue #1 `_) * lots of attic issues fixed (see `issue #5 `_) * less chunk management overhead (less memory and disk usage for chunks index) * faster remote cache resync (useful when backing up multiple machines into same repo) * compression: no, lz4, zlib or lzma compression, adjustable compression levels * repokey replaces problematic passphrase mode (you can't change the passphrase nor the pbkdf2 iteration count in "passphrase" mode) * simple sparse file support, great for virtual machine disk files * can read special files (e.g. block devices) or from stdin, write to stdout * mkdir-based locking is more compatible than attic's posix locking * uses fadvise to not spoil / blow up the fs cache * better error messages / exception handling * better logging, screen output, progress indication * tested on misc. Linux systems, 32 and 64bit, FreeBSD, OpenBSD, NetBSD, Mac OS X Please read the :ref:`changelog` (or ``docs/changes.rst`` in the source distribution) for more information. Borg is not compatible with original attic (but there is a one-way conversion). borgbackup-1.0.13/docs/global.rst.inc0000644000175000017500000000302213431513133017355 0ustar useruser00000000000000.. highlight:: bash .. |project_name| replace:: Borg .. |package_dirname| replace:: borgbackup-|version| .. |package_filename| replace:: |package_dirname|.tar.gz .. |package_url| replace:: https://pypi.python.org/packages/source/b/borgbackup/|package_filename| .. |git_url| replace:: https://github.com/borgbackup/borg.git .. _github: https://github.com/borgbackup/borg .. _issue tracker: https://github.com/borgbackup/borg/issues .. _deduplication: https://en.wikipedia.org/wiki/Data_deduplication .. _AES: https://en.wikipedia.org/wiki/Advanced_Encryption_Standard .. _HMAC-SHA256: https://en.wikipedia.org/wiki/HMAC .. _SHA256: https://en.wikipedia.org/wiki/SHA-256 .. _PBKDF2: https://en.wikipedia.org/wiki/PBKDF2 .. _ACL: https://en.wikipedia.org/wiki/Access_control_list .. _libacl: https://savannah.nongnu.org/projects/acl/ .. _libattr: https://savannah.nongnu.org/projects/attr/ .. _liblz4: https://github.com/Cyan4973/lz4 .. _OpenSSL: https://www.openssl.org/ .. _`Python 3`: https://www.python.org/ .. _Buzhash: https://en.wikipedia.org/wiki/Buzhash .. _msgpack: https://msgpack.org/ .. _`msgpack-python`: https://pypi.python.org/pypi/msgpack-python/ .. _llfuse: https://pypi.python.org/pypi/llfuse/ .. _homebrew: http://brew.sh/ .. _userspace filesystems: https://en.wikipedia.org/wiki/Filesystem_in_Userspace .. _Cython: http://cython.org/ .. _virtualenv: https://pypi.python.org/pypi/virtualenv/ .. _mailing list discussion about internals: http://librelist.com/browser/attic/2014/5/6/questions-and-suggestions-about-inner-working-of-attic> borgbackup-1.0.13/docs/index.rst0000644000175000017500000000051313431513133016456 0ustar useruser00000000000000.. include:: global.rst.inc Borg Documentation ================== .. include:: ../README.rst .. when you add an element here, do not forget to add it to book.rst .. toctree:: :maxdepth: 2 installation quickstart usage deployment faq support resources changes internals development authors borgbackup-1.0.13/docs/installation.rst0000644000175000017500000003711213431513133020055 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: bash .. _installation: Installation ============ There are different ways to install |project_name|: - :ref:`distribution-package` - easy and fast if a package is available from your distribution. - :ref:`pyinstaller-binary` - easy and fast, we provide a ready-to-use binary file that comes bundled with all dependencies. - :ref:`source-install`, either: - :ref:`pip-installation` - installing a source package with pip needs more installation steps and requires all dependencies with development headers and a compiler. - :ref:`git-installation` - for developers and power users who want to have the latest code or use revision control (each release is tagged). .. _installation-requirements: Pre-Installation Considerations ------------------------------- (G)LIBC requirements -------------------- Borg uses some filesytem functions from Python's `os` standard library module with `follow_symlinks=False`. These are implemented since quite a while with the non-symlink-following (g)libc functions like e.g. `lstat` or `lutimes` (not: `stat` or `utimes`). Some stoneage systems (like RHEL/CentOS 5) and also Python interpreter binaries compiled to be able to run on such systems (like Python installed via Anaconda) might miss these functions and Borg won't be able to work correctly. This issue will be detected early and Borg will abort with a fatal error. For the Borg binaries, there are additional (g)libc requirements, see below. .. _distribution-package: Distribution Package -------------------- Some distributions might offer a ready-to-use ``borgbackup`` package which can be installed with the package manager. As |project_name| is still a young project, such a package might be not available for your system yet. ============ ============================================= ======= Distribution Source Command ============ ============================================= ======= Arch Linux `[community]`_ ``pacman -S borg`` Debian `Debian packages`_ ``apt install borgbackup`` Gentoo `ebuild`_ ``emerge borgbackup`` GNU Guix `GNU Guix`_ ``guix package --install borg`` Fedora/RHEL `Fedora official repository`_ ``dnf install borgbackup`` FreeBSD `FreeBSD ports`_ ``cd /usr/ports/archivers/py-borgbackup && make install clean`` macOS `Brew cask`_ ``brew cask install borgbackup`` Mageia `cauldron`_ ``urpmi borgbackup`` NetBSD `pkgsrc`_ ``pkg_add py-borgbackup`` NixOS `.nix file`_ N/A OpenBSD `OpenBSD ports`_ ``pkg_add borgbackup`` OpenIndiana `OpenIndiana hipster repository`_ ``pkg install borg`` openSUSE `openSUSE official repository`_ ``zypper in borgbackup`` Raspbian `Raspbian testing`_ ``apt install borgbackup`` Ubuntu `Ubuntu packages`_, `Ubuntu PPA`_ ``apt install borgbackup`` ============ ============================================= ======= .. _[community]: https://www.archlinux.org/packages/?name=borg .. _Debian packages: https://packages.debian.org/search?keywords=borgbackup&searchon=names&exact=1&suite=all§ion=all .. _Fedora official repository: https://apps.fedoraproject.org/packages/borgbackup .. _FreeBSD ports: http://www.freshports.org/archivers/py-borgbackup/ .. _ebuild: https://packages.gentoo.org/packages/app-backup/borgbackup .. _GNU Guix: https://www.gnu.org/software/guix/package-list.html#borg .. _pkgsrc: http://pkgsrc.se/sysutils/py-borgbackup .. _cauldron: http://madb.mageia.org/package/show/application/0/release/cauldron/name/borgbackup .. _Xenial 16.04: https://launchpad.net/ubuntu/xenial/+source/borgbackup .. _Wily 15.10 (backport PPA): https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/borgbackup .. _Trusty 14.04 (backport PPA): https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/borgbackup .. _.nix file: https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/backup/borg/default.nix .. _Brew cask: https://caskroom.github.io/ .. _Raspbian testing: http://archive.raspbian.org/raspbian/pool/main/b/borgbackup/ .. _Ubuntu packages: http://packages.ubuntu.com/xenial/borgbackup .. _Ubuntu PPA: https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/borgbackup Please ask package maintainers to build a package or, if you can package / submit it yourself, please help us with that! See :issue:`105` on github to followup on packaging efforts. If a package is available, it might be interesting to check its version and compare that to our latest release and review the :doc:`changes`. .. _pyinstaller-binary: Standalone Binary ----------------- .. note:: Releases are signed with an OpenPGP key, see :ref:`security-contact` for more instructions. |project_name| binaries (generated with `pyinstaller`_) are available on the releases_ page for the following platforms: * **Linux**: glibc >= 2.13 (ok for most supported Linux releases). Maybe older glibc versions also work, if they are compatible to 2.13. * **Mac OS X**: 10.10 (does not work with older OS X releases) * **FreeBSD**: 10.2 (unknown whether it works for older releases) To install such a binary, just drop it into a directory in your ``PATH``, make borg readable and executable for its users and then you can run ``borg``:: sudo cp borg-linux64 /usr/local/bin/borg sudo chown root:root /usr/local/bin/borg sudo chmod 755 /usr/local/bin/borg Note that the binary uses /tmp to unpack |project_name| with all dependencies. It will fail if /tmp has not enough free space or is mounted with the ``noexec`` option. You can change the temporary directory by setting the ``TEMP`` environment variable before running |project_name|. If a new version is released, you will have to manually download it and replace the old version using the same steps as shown above. .. _pyinstaller: http://www.pyinstaller.org .. _releases: https://github.com/borgbackup/borg/releases .. _platforms: Features & platforms -------------------- Besides regular file and directory structures, |project_name| can preserve * Hardlinks (considering all files in the same archive) * Symlinks (stored as symlink, the symlink is not followed) * Special files: * Character and block device files (restored via mknod) * FIFOs ("named pipes") * Special file *contents* can be backed up in ``--read-special`` mode. By default the metadata to create them with mknod(2), mkfifo(2) etc. is stored. * Timestamps in nanosecond precision: mtime, atime, ctime * Permissions: * IDs of owning user and owning group * Names of owning user and owning group (if the IDs can be resolved) * Unix Mode/Permissions (u/g/o permissions, suid, sgid, sticky) On some platforms additional features are supported: .. Yes/No's are grouped by reason/mechanism/reference. +------------------+----------+-----------+------------+ | Platform | ACLs | xattr | Flags | | | [#acls]_ | [#xattr]_ | [#flags]_ | +==================+==========+===========+============+ | Linux x86 | Yes | Yes | No | +------------------+ | | | | Linux PowerPC | | | | +------------------+ | | | | Linux ARM | | | | +------------------+----------+-----------+------------+ | Mac OS X | Yes | Yes | Yes (all) | +------------------+----------+-----------+ | | FreeBSD | Yes | Yes | | +------------------+----------+-----------+ | | OpenBSD | n/a | n/a | | +------------------+----------+-----------+ | | NetBSD | n/a | No [2]_ | | +------------------+----------+-----------+------------+ | Solaris 11 | No [3]_ | n/a | +------------------+ | | | OpenIndiana | | | +------------------+----------+-----------+------------+ | Windows (cygwin) | No [4]_ | No | No | +------------------+----------+-----------+------------+ Some Distributions (e.g. Debian) run additional tests after each release, these are not reflected here. Other Unix-like operating systems may work as well, but have not been tested at all. Note that most of the platform-dependent features also depend on the file system. For example, ntfs-3g on Linux isn't able to convey NTFS ACLs. .. [2] Feature request :issue:`1332` .. [3] Feature request :issue:`1337` .. [4] Cygwin tries to map NTFS ACLs to permissions with varying degress of success. .. [#acls] The native access control list mechanism of the OS. This normally limits access to non-native ACLs. For example, NTFS ACLs aren't completely accessible on Linux with ntfs-3g. .. [#xattr] extended attributes; key-value pairs attached to a file, mainly used by the OS. This includes resource forks on Mac OS X. .. [#flags] aka *BSD flags*. .. _source-install: From Source ----------- Dependencies ~~~~~~~~~~~~ To install |project_name| from a source package (including pip), you have to install the following dependencies first: * `Python 3`_ >= 3.4.0, plus development headers. Even though Python 3 is not the default Python version on most systems, it is usually available as an optional install. * OpenSSL_ >= 1.0.0, plus development headers. * libacl_ (that pulls in libattr_ also), both plus development headers. * liblz4_, plus development headers. * some Python dependencies, pip will automatically install them for you * optionally, the llfuse_ Python package is required if you wish to mount an archive as a FUSE filesystem. See setup.py about the version requirements. If you have troubles finding the right package names, have a look at the distribution specific sections below and also at the Vagrantfile in our repo. In the following, the steps needed to install the dependencies are listed for a selection of platforms. If your distribution is not covered by these instructions, try to use your package manager to install the dependencies. On FreeBSD, you may need to get a recent enough OpenSSL version from FreeBSD ports. After you have installed the dependencies, you can proceed with steps outlined under :ref:`pip-installation`. Debian / Ubuntu +++++++++++++++ Install the dependencies with development headers:: sudo apt-get install python3 python3-dev python3-pip python-virtualenv \ libssl-dev openssl \ libacl1-dev libacl1 \ liblz4-dev liblz4-1 \ build-essential sudo apt-get install libfuse-dev fuse pkg-config # optional, for FUSE support In case you get complaints about permission denied on ``/etc/fuse.conf``: on Ubuntu this means your user is not in the ``fuse`` group. Add yourself to that group, log out and log in again. Fedora / Korora +++++++++++++++ Install the dependencies with development headers:: sudo dnf install python3 python3-devel python3-pip python3-virtualenv sudo dnf install openssl-devel openssl sudo dnf install libacl-devel libacl sudo dnf install lz4-devel sudo dnf install gcc gcc-c++ sudo dnf install fuse-devel fuse pkgconfig # optional, for FUSE support openSUSE Tumbleweed / Leap ++++++++++++++++++++++++++ Install the dependencies automatically using zypper:: sudo zypper source-install --build-deps-only borgbackup Alternatively, you can enumerate all build dependencies in the command line:: sudo zypper install python3 python3-devel \ libacl-devel liblz4-devel openssl-devel \ python3-Cython python3-Sphinx python3-msgpack-python \ python3-pytest python3-setuptools python3-setuptools_scm \ python3-sphinx_rtd_theme python3-llfuse gcc gcc-c++ Mac OS X ++++++++ Assuming you have installed homebrew_, the following steps will install all the dependencies:: brew install python3 lz4 openssl brew install pkg-config # optional, for FUSE support pip3 install virtualenv For FUSE support to mount the backup archives, you need at least version 3.0 of FUSE for OS X, which is available via github_, or via homebrew:: brew cask install osxfuse .. _github: https://github.com/osxfuse/osxfuse/releases/latest FreeBSD ++++++++ Listed below are packages you will need to install |project_name|, its dependencies, and commands to make fuse work for using the mount command. :: pkg install -y python3 openssl liblz4 fusefs-libs pkgconf pkg install -y git python3.4 -m ensurepip # to install pip for Python3 To use the mount command: echo 'fuse_load="YES"' >> /boot/loader.conf echo 'vfs.usermount=1' >> /etc/sysctl.conf kldload fuse sysctl vfs.usermount=1 Windows 10's Linux Subsystem ++++++++++++++++++++++++++++ .. note:: Running under Windows 10's Linux Subsystem is experimental and has not been tested much yet. Just follow the Ubuntu Linux installation steps. You can omit the FUSE stuff, it won't work anyway. Cygwin ++++++ .. note:: Running under Cygwin is experimental and has only been tested with Cygwin (x86-64) v2.5.2. Remote repositories are known broken, local repositories should work. Use the Cygwin installer to install the dependencies:: python3 python3-devel python3-setuptools binutils gcc-g++ libopenssl openssl-devel liblz4_1 liblz4-devel git make openssh You can then install ``pip`` and ``virtualenv``:: easy_install-3.4 pip pip install virtualenv .. _pip-installation: Using pip ~~~~~~~~~ Virtualenv_ can be used to build and install |project_name| without affecting the system Python or requiring root access. Using a virtual environment is optional, but recommended except for the most simple use cases. .. note:: If you install into a virtual environment, you need to **activate** it first (``source borg-env/bin/activate``), before running ``borg``. Alternatively, symlink ``borg-env/bin/borg`` into some directory that is in your ``PATH`` so you can just run ``borg``. This will use ``pip`` to install the latest release from PyPi:: virtualenv --python=python3 borg-env source borg-env/bin/activate # install Borg + Python dependencies into virtualenv pip install borgbackup # or alternatively (if you want FUSE support): pip install borgbackup[fuse] To upgrade |project_name| to a new version later, run the following after activating your virtual environment:: pip install -U borgbackup # or ... borgbackup[fuse] .. _git-installation: Using git ~~~~~~~~~ This uses latest, unreleased development code from git. While we try not to break master, there are no guarantees on anything. :: # get borg from github git clone https://github.com/borgbackup/borg.git virtualenv --python=python3 borg-env source borg-env/bin/activate # always before using! # install borg + dependencies into virtualenv pip install sphinx # optional, to build the docs cd borg pip install -r requirements.d/development.txt pip install -r requirements.d/fuse.txt # optional, for FUSE support pip install -e . # in-place editable mode # optional: run all the tests, on all supported Python versions # requires fakeroot, available through your package manager fakeroot -u tox .. note:: As a developer or power user, you always want to use a virtual environment. borgbackup-1.0.13/docs/internals.rst0000644000175000017500000004504513431513133017357 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: none .. _internals: Internals ========= This page documents the internal data structures and storage mechanisms of |project_name|. It is partly based on `mailing list discussion about internals`_ and also on static code analysis. Repository and Archives ----------------------- |project_name| stores its data in a `Repository`. Each repository can hold multiple `Archives`, which represent individual backups that contain a full archive of the files specified when the backup was performed. Deduplication is performed across multiple backups, both on data and metadata, using `Chunks` created by the chunker using the Buzhash_ algorithm. Each repository has the following file structure: README simple text file telling that this is a |project_name| repository config repository configuration data/ directory where the actual data is stored hints.%d hints for repository compaction index.%d repository index lock.roster and lock.exclusive/* used by the locking system to manage shared and exclusive locks Lock files ---------- |project_name| uses locks to get (exclusive or shared) access to the cache and the repository. The locking system is based on creating a directory `lock.exclusive` (for exclusive locks). Inside the lock directory, there is a file indicating hostname, process id and thread id of the lock holder. There is also a json file `lock.roster` that keeps a directory of all shared and exclusive lockers. If the process can create the `lock.exclusive` directory for a resource, it has the lock for it. If creation fails (because the directory has already been created by some other process), lock acquisition fails. The cache lock is usually in `~/.cache/borg/REPOID/lock.*`. The repository lock is in `repository/lock.*`. In case you run into troubles with the locks, you can use the ``borg break-lock`` command after you first have made sure that no |project_name| process is running on any machine that accesses this resource. Be very careful, the cache or repository might get damaged if multiple processes use it at the same time. Config file ----------- Each repository has a ``config`` file which which is a ``INI``-style file and looks like this:: [repository] version = 1 segments_per_dir = 10000 max_segment_size = 5242880 id = 57d6c1d52ce76a836b532b0e42e677dec6af9fca3673db511279358828a21ed6 This is where the ``repository.id`` is stored. It is a unique identifier for repositories. It will not change if you move the repository around so you can make a local transfer then decide to move the repository to another (even remote) location at a later time. Keys ---- The key to address the key/value store is usually computed like this: key = id = id_hash(unencrypted_data) The id_hash function is: * sha256 (no encryption keys available) * hmac-sha256 (encryption keys available) As the id / key is used for deduplication, id_hash must be a cryptographically strong hash or MAC. Segments and archives --------------------- A |project_name| repository is a filesystem based transactional key/value store. It makes extensive use of msgpack_ to store data and, unless otherwise noted, data is stored in msgpack_ encoded files. Objects referenced by a key are stored inline in files (`segments`) of approx. 5MB size in numbered subdirectories of ``repo/data``. They contain: * header size * crc * size * tag * key * data Segments are built locally, and then uploaded. Those files are strictly append-only and modified only once. Tag is either ``PUT``, ``DELETE``, or ``COMMIT``. A segment file is basically a transaction log where each repository operation is appended to the file. So if an object is written to the repository a ``PUT`` tag is written to the file followed by the object id and data. If an object is deleted a ``DELETE`` tag is appended followed by the object id. A ``COMMIT`` tag is written when a repository transaction is committed. When a repository is opened any ``PUT`` or ``DELETE`` operations not followed by a ``COMMIT`` tag are discarded since they are part of a partial/uncommitted transaction. The manifest ------------ The manifest is an object with an all-zero key that references all the archives. It contains: * version * list of archive infos * timestamp * config Each archive info contains: * name * id * time It is the last object stored, in the last segment, and is replaced each time. The Archive ----------- The archive metadata does not contain the file items directly. Only references to other objects that contain that data. An archive is an object that contains: * version * name * list of chunks containing item metadata (size: count * ~40B) * cmdline * hostname * username * time .. _archive_limitation: Note about archive limitations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The archive is currently stored as a single object in the repository and thus limited in size to MAX_OBJECT_SIZE (20MiB). As one chunk list entry is ~40B, that means we can reference ~500.000 item metadata stream chunks per archive. Each item metadata stream chunk is ~128kiB (see hardcoded ITEMS_CHUNKER_PARAMS). So that means the whole item metadata stream is limited to ~64GiB chunks. If compression is used, the amount of storable metadata is bigger - by the compression factor. If the medium size of an item entry is 100B (small size file, no ACLs/xattrs), that means a limit of ~640 million files/directories per archive. If the medium size of an item entry is 2kB (~100MB size files or more ACLs/xattrs), the limit will be ~32 million files/directories per archive. If one tries to create an archive object bigger than MAX_OBJECT_SIZE, a fatal IntegrityError will be raised. A workaround is to create multiple archives with less items each, see also :issue:`1452`. The Item -------- Each item represents a file, directory or other fs item and is stored as an ``item`` dictionary that contains: * path * list of data chunks (size: count * ~40B) * user * group * uid * gid * mode (item type + permissions) * source (for links) * rdev (for devices) * mtime, atime, ctime in nanoseconds * xattrs * acl * bsdfiles All items are serialized using msgpack and the resulting byte stream is fed into the same chunker algorithm as used for regular file data and turned into deduplicated chunks. The reference to these chunks is then added to the archive metadata. To achieve a finer granularity on this metadata stream, we use different chunker params for this chunker, which result in smaller chunks. A chunk is stored as an object as well, of course. .. _chunker_details: Chunks ------ The |project_name| chunker uses a rolling hash computed by the Buzhash_ algorithm. It triggers (chunks) when the last HASH_MASK_BITS bits of the hash are zero, producing chunks of 2^HASH_MASK_BITS Bytes on average. Buzhash is **only** used for cutting the chunks at places defined by the content, the buzhash value is **not** used as the deduplication criteria (we use a cryptographically strong hash/MAC over the chunk contents for this, the id_hash). ``borg create --chunker-params CHUNK_MIN_EXP,CHUNK_MAX_EXP,HASH_MASK_BITS,HASH_WINDOW_SIZE`` can be used to tune the chunker parameters, the default is: - CHUNK_MIN_EXP = 19 (minimum chunk size = 2^19 B = 512 kiB) - CHUNK_MAX_EXP = 23 (maximum chunk size = 2^23 B = 8 MiB) - HASH_MASK_BITS = 21 (statistical medium chunk size ~= 2^21 B = 2 MiB) - HASH_WINDOW_SIZE = 4095 [B] (`0xFFF`) The buzhash table is altered by XORing it with a seed randomly generated once for the repository, and stored encrypted in the keyfile. This is to prevent chunk size based fingerprinting attacks on your encrypted repo contents (to guess what files you have based on a specific set of chunk sizes). For some more general usage hints see also ``--chunker-params``. Indexes / Caches ---------------- The **files cache** is stored in ``cache/files`` and is used at backup time to quickly determine whether a given file is unchanged and we have all its chunks. The files cache is a key -> value mapping and contains: * key: - full, absolute file path id_hash * value: - file inode number - file size - file mtime_ns - list of file content chunk id hashes - age (0 [newest], 1, 2, 3, ..., BORG_FILES_CACHE_TTL - 1) To determine whether a file has not changed, cached values are looked up via the key in the mapping and compared to the current file attribute values. If the file's size, mtime_ns and inode number is still the same, it is considered to not have changed. In that case, we check that all file content chunks are (still) present in the repository (we check that via the chunks cache). If everything is matching and all chunks are present, the file is not read / chunked / hashed again (but still a file metadata item is written to the archive, made from fresh file metadata read from the filesystem). This is what makes borg so fast when processing unchanged files. If there is a mismatch or a chunk is missing, the file is read / chunked / hashed. Chunks already present in repo won't be transferred to repo again. The inode number is stored and compared to make sure we distinguish between different files, as a single path may not be unique across different archives in different setups. Not all filesystems have stable inode numbers. If that is the case, borg can be told to ignore the inode number in the check via --ignore-inode. The age value is used for cache management. If a file is "seen" in a backup run, its age is reset to 0, otherwise its age is incremented by one. If a file was not seen in BORG_FILES_CACHE_TTL backups, its cache entry is removed. See also: :ref:`always_chunking` and :ref:`a_status_oddity` The files cache is a python dictionary, storing python objects, which generates a lot of overhead. Borg can also work without using the files cache (saves memory if you have a lot of files or not much RAM free), then all files are assumed to have changed. This is usually much slower than with files cache. The **chunks cache** is stored in ``cache/chunks`` and is used to determine whether we already have a specific chunk, to count references to it and also for statistics. The chunks cache is a key -> value mapping and contains: * key: - chunk id_hash * value: - reference count - size - encrypted/compressed size The chunks cache is a hashindex, a hash table implemented in C and tuned for memory efficiency. The **repository index** is stored in ``repo/index.%d`` and is used to determine a chunk's location in the repository. The repo index is a key -> value mapping and contains: * key: - chunk id_hash * value: - segment (that contains the chunk) - offset (where the chunk is located in the segment) The repo index is a hashindex, a hash table implemented in C and tuned for memory efficiency. Hints are stored in a file (``repo/hints.%d``). It contains: * version * list of segments * compact hints and index can be recreated if damaged or lost using ``check --repair``. The chunks cache and the repository index are stored as hash tables, with only one slot per bucket, but that spreads the collisions to the following buckets. As a consequence the hash is just a start position for a linear search, and if the element is not in the table the index is linearly crossed until an empty bucket is found. When the hash table is filled to 75%, its size is grown. When it's emptied to 25%, its size is shrinked. So operations on it have a variable complexity between constant and linear with low factor, and memory overhead varies between 33% and 300%. .. _cache-memory-usage: Indexes / Caches memory usage ----------------------------- Here is the estimated memory usage of |project_name| - it's complicated: chunk_count ~= total_file_size / 2 ^ HASH_MASK_BITS repo_index_usage = chunk_count * 40 chunks_cache_usage = chunk_count * 44 files_cache_usage = total_file_count * 240 + chunk_count * 80 mem_usage ~= repo_index_usage + chunks_cache_usage + files_cache_usage = chunk_count * 164 + total_file_count * 240 Due to the hashtables, the best/usual/worst cases for memory allocation can be estimated like that: mem_allocation = mem_usage / load_factor # l_f = 0.25 .. 0.75 mem_allocation_peak = mem_allocation * (1 + growth_factor) # g_f = 1.1 .. 2 All units are Bytes. It is assuming every chunk is referenced exactly once (if you have a lot of duplicate chunks, you will have less chunks than estimated above). It is also assuming that typical chunk size is 2^HASH_MASK_BITS (if you have a lot of files smaller than this statistical medium chunk size, you will have more chunks than estimated above, because 1 file is at least 1 chunk). If a remote repository is used the repo index will be allocated on the remote side. The chunks cache, files cache and the repo index are all implemented as hash tables. A hash table must have a significant amount of unused entries to be fast - the so-called load factor gives the used/unused elements ratio. When a hash table gets full (load factor getting too high), it needs to be grown (allocate new, bigger hash table, copy all elements over to it, free old hash table) - this will lead to short-time peaks in memory usage each time this happens. Usually does not happen for all hashtables at the same time, though. For small hash tables, we start with a growth factor of 2, which comes down to ~1.1x for big hash tables. E.g. backing up a total count of 1 Mi (IEC binary prefix i.e. 2^20) files with a total size of 1TiB. a) with ``create --chunker-params 10,23,16,4095`` (custom, like borg < 1.0 or attic): mem_usage = 2.8GiB b) with ``create --chunker-params 19,23,21,4095`` (default): mem_usage = 0.31GiB .. note:: There is also the ``--no-files-cache`` option to switch off the files cache. You'll save some memory, but it will need to read / chunk all the files as it can not skip unmodified files then. Encryption ---------- AES_-256 is used in CTR mode (so no need for padding). A 64bit initialization vector is used, a `HMAC-SHA256`_ is computed on the encrypted chunk with a random 64bit nonce and both are stored in the chunk. The header of each chunk is: ``TYPE(1)`` + ``HMAC(32)`` + ``NONCE(8)`` + ``CIPHERTEXT``. Encryption and HMAC use two different keys. In AES CTR mode you can think of the IV as the start value for the counter. The counter itself is incremented by one after each 16 byte block. The IV/counter is not required to be random but it must NEVER be reused. So to accomplish this |project_name| initializes the encryption counter to be higher than any previously used counter value before encrypting new data. To reduce payload size, only 8 bytes of the 16 bytes nonce is saved in the payload, the first 8 bytes are always zeros. This does not affect security but limits the maximum repository capacity to only 295 exabytes (2**64 * 16 bytes). Encryption keys (and other secrets) are kept either in a key file on the client ('keyfile' mode) or in the repository config on the server ('repokey' mode). In both cases, the secrets are generated from random and then encrypted by a key derived from your passphrase (this happens on the client before the key is stored into the keyfile or as repokey). The passphrase is passed through the ``BORG_PASSPHRASE`` environment variable or prompted for interactive usage. Key files --------- When initialized with the ``init -e keyfile`` command, |project_name| needs an associated file in ``$HOME/.config/borg/keys`` to read and write the repository. The format is based on msgpack_, base64 encoding and PBKDF2_ SHA256 hashing, which is then encoded again in a msgpack_. The internal data structure is as follows: version currently always an integer, 1 repository_id the ``id`` field in the ``config`` ``INI`` file of the repository. enc_key the key used to encrypt data with AES (256 bits) enc_hmac_key the key used to HMAC the encrypted data (256 bits) id_key the key used to HMAC the plaintext chunk data to compute the chunk's id chunk_seed the seed for the buzhash chunking table (signed 32 bit integer) Those fields are processed using msgpack_. The utf-8 encoded passphrase is processed with PBKDF2_ (SHA256_, 100000 iterations, random 256 bit salt) to give us a derived key. The derived key is 256 bits long. A `HMAC-SHA256`_ checksum of the above fields is generated with the derived key, then the derived key is also used to encrypt the above pack of fields. Then the result is stored in a another msgpack_ formatted as follows: version currently always an integer, 1 salt random 256 bits salt used to process the passphrase iterations number of iterations used to process the passphrase (currently 100000) algorithm the hashing algorithm used to process the passphrase and do the HMAC checksum (currently the string ``sha256``) hash the HMAC of the encrypted derived key data the derived key, encrypted with AES over a PBKDF2_ SHA256 key described above The resulting msgpack_ is then encoded using base64 and written to the key file, wrapped using the standard ``textwrap`` module with a header. The header is a single line with a MAGIC string, a space and a hexadecimal representation of the repository id. Compression ----------- |project_name| supports the following compression methods: - none (no compression, pass through data 1:1) - lz4 (low compression, but super fast) - zlib (level 0-9, level 0 is no compression [but still adding zlib overhead], level 1 is low, level 9 is high compression) - lzma (level 0-9, level 0 is low, level 9 is high compression). Speed: none > lz4 > zlib > lzma Compression: lzma > zlib > lz4 > none Be careful, higher zlib and especially lzma compression levels might take a lot of resources (CPU and memory). The overall speed of course also depends on the speed of your target storage. If that is slow, using a higher compression level might yield better overall performance. You need to experiment a bit. Maybe just watch your CPU load, if that is relatively low, increase compression until 1 core is 70-100% loaded. Even if your target storage is rather fast, you might see interesting effects: while doing no compression at all (none) is a operation that takes no time, it likely will need to store more data to the storage compared to using lz4. The time needed to transfer and store the additional data might be much more than if you had used lz4 (which is super fast, but still might compress your data about 2:1). This is assuming your data is compressible (if you backup already compressed data, trying to compress them at backup time is usually pointless). Compression is applied after deduplication, thus using different compression methods in one repo does not influence deduplication. See ``borg create --help`` about how to specify the compression level and its default. borgbackup-1.0.13/docs/introduction.rst0000644000175000017500000000035013431513133020067 0ustar useruser00000000000000Introduction ============ .. this shim is here to fix the structure in the PDF rendering. without this stub, the elements in the toctree of index.rst show up a level below the README file included .. include:: ../README.rst borgbackup-1.0.13/docs/misc/0000755000175000017500000000000013431514213015551 5ustar useruser00000000000000borgbackup-1.0.13/docs/misc/asciinema/0000755000175000017500000000000013431514213017502 5ustar useruser00000000000000borgbackup-1.0.13/docs/misc/asciinema/install_and_basics.json0000644000175000017500000020664313431513133024224 0ustar useruser00000000000000{ "version": 1, "width": 80, "height": 24, "duration": 332.0, "command": "/bin/bash", "title": "borgbackup - installation and basic usage", "env": { "TERM": "xterm", "SHELL": "/bin/bash" }, "stdout": [ [ 0.083341, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.349103, "#" ], [ 0.338948, " " ], [ 0.185424, "b" ], [ 0.142971, "o" ], [ 0.091227, "r" ], [ 0.092867, "g" ], [ 0.222552, "b" ], [ 0.114706, "a" ], [ 0.125044, "c" ], [ 0.144755, "k" ], [ 0.241044, "u" ], [ 0.243681, "p" ], [ 0.265888, " " ], [ 0.345247, "-" ], [ 0.251918, " " ], [ 0.233420, "i" ], [ 0.078609, "n" ], [ 0.076809, "s" ], [ 0.070225, "t" ], [ 0.148413, "a" ], [ 0.077403, "l" ], [ 0.139884, "l" ], [ 0.084807, "a" ], [ 0.138823, "t" ], [ 0.068185, "i" ], [ 0.170422, "o" ], [ 0.161091, "n" ], [ 0.169247, " " ], [ 0.110722, "a" ], [ 0.113785, "n" ], [ 0.397895, "d" ], [ 0.305048, " " ], [ 0.211476, "b" ], [ 0.109865, "a" ], [ 0.230634, "s" ], [ 0.277915, "i" ], [ 0.206167, "c" ], [ 0.145265, " " ], [ 0.219619, "u" ], [ 0.139989, "s" ], [ 0.180240, "a" ], [ 0.200391, "g" ], [ 0.116961, "e" ], [ 0.172074, "\r\n" ], [ 0.000449, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.620909, "#" ], [ 0.217833, " " ], [ 0.592920, "I" ], [ 0.166726, " " ], [ 0.161953, "h" ], [ 0.072501, "a" ], [ 0.170951, "v" ], [ 0.154067, "e" ], [ 0.110535, " " ], [ 0.155235, "a" ], [ 0.130825, "l" ], [ 0.111834, "r" ], [ 0.142378, "e" ], [ 0.165867, "a" ], [ 0.062556, "d" ], [ 0.091778, "y" ], [ 0.216280, " " ], [ 0.169501, "d" ], [ 0.198240, "o" ], [ 0.092373, "w" ], [ 0.143405, "n" ], [ 0.207324, "l" ], [ 0.164248, "o" ], [ 0.088481, "a" ], [ 0.129191, "d" ], [ 0.179234, "e" ], [ 0.189248, "d" ], [ 0.145203, " " ], [ 0.221625, "t" ], [ 0.100064, "h" ], [ 0.133349, "e" ], [ 0.066501, " " ], [ 0.187004, "f" ], [ 0.142461, "i" ], [ 0.204723, "l" ], [ 0.068716, "e" ], [ 0.237576, "s" ], [ 0.128085, ":" ], [ 0.242282, "\r\n" ], [ 0.000327, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 1.796834, "l" ], [ 0.092545, "s" ], [ 0.210322, " " ], [ 0.189710, "-" ], [ 0.215532, "l" ], [ 0.852863, "\r\n" ], [ 0.002104, "total 10620\r\n" ], [ 0.000040, "-rw-rw-r-- 1 tw tw 10869049 Oct 24 22:11 borg-linux64" ], [ 0.000007, "\r\n" ], [ 0.000019, "-rw-rw-r-- 1 tw tw 819 Oct 24 22:11 borg-linux64.asc\r\n" ], [ 0.000431, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.513172, "#" ], [ 0.284059, " " ], [ 0.330931, "b" ], [ 0.118806, "i" ], [ 0.100553, "n" ], [ 0.259930, "a" ], [ 0.106715, "r" ], [ 0.276545, "y" ], [ 0.126132, " " ], [ 0.379724, "+" ], [ 0.199249, " " ], [ 0.295913, "G" ], [ 0.108970, "P" ], [ 0.080480, "G" ], [ 0.349293, " " ], [ 0.236785, "s" ], [ 0.105197, "i" ], [ 0.289951, "g" ], [ 0.351385, "n" ], [ 0.282003, "a" ], [ 0.206591, "t" ], [ 0.163963, "u" ], [ 0.082416, "r" ], [ 0.125432, "e" ], [ 0.369988, "\r\n" ], [ 0.000341, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.889617, "#" ], [ 0.226974, " " ], [ 0.218497, "l" ], [ 0.134545, "e" ], [ 0.103159, "t" ], [ 0.711682, "'" ], [ 0.185463, "s" ], [ 0.162130, " " ], [ 0.166049, "v" ], [ 0.183069, "e" ], [ 0.099764, "r" ], [ 0.234211, "i" ], [ 0.854328, "f" ], [ 0.203758, "y" ], [ 0.166681, " " ], [ 0.216715, "t" ], [ 0.560064, "h" ], [ 0.151837, "a" ], [ 0.194509, "t" ], [ 0.119665, " " ], [ 0.141089, "t" ], [ 0.096803, "h" ], [ 0.104718, "e" ], [ 0.106761, " " ], [ 0.229401, "b" ], [ 0.213802, "i" ], [ 0.075481, "n" ], [ 0.138720, "a" ], [ 0.062411, "r" ], [ 0.292719, "y" ], [ 0.482737, " " ], [ 0.211595, "i" ], [ 0.110964, "s" ], [ 0.102100, " " ], [ 0.143380, "v" ], [ 0.189214, "a" ], [ 0.099337, "l" ], [ 0.172757, "i" ], [ 0.082456, "d" ], [ 0.177514, ":" ], [ 0.622492, "\r\n" ], [ 0.000313, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 2.000000, "g" ], [ 0.261924, "p" ], [ 0.108570, "g" ], [ 0.247315, " " ], [ 0.277162, "-" ], [ 0.141397, "-" ], [ 0.143255, "v" ], [ 0.162858, "e" ], [ 0.040051, "r" ], [ 0.105941, "i" ], [ 0.144872, "f" ], [ 0.306497, "y" ], [ 0.468271, " " ], [ 2.000000, "b" ], [ 0.119390, "o" ], [ 0.463137, "\u0007" ], [ 0.000095, "rg-linux64" ], [ 0.341519, "." ], [ 0.146977, "asc " ], [ 0.186292, "\r\n" ], [ 0.100648, "gpg: Signature made Wed 07 Oct 2015 02:41:38 PM CEST\r\n" ], [ 0.000011, "gpg: using RSA key 243ACFA951F78E01\r\n" ], [ 0.006906, "gpg: Good signature from \"Thomas Waldmann \u003ctw@waldmann-edv.de\u003e\"" ], [ 0.000033, "\r\ngpg: aka \"Thomas Waldmann \u003ctw-public@gmx.de\u003e\"\r\ngpg: aka \"Thomas Waldmann \u003ctwaldmann@thinkmo.de\u003e\"\r\n" ], [ 0.000018, "gpg: aka \"Thomas Waldmann \u003cthomas.j.waldmann@gmail.com\u003e\"\r\n" ], [ 0.003077, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 2.000000, "#" ], [ 0.241501, " " ], [ 0.186571, "e" ], [ 0.214388, "v" ], [ 0.157101, "e" ], [ 0.042348, "r" ], [ 0.253261, "y" ], [ 0.254356, "t" ], [ 0.094622, "h" ], [ 0.213972, "i" ], [ 0.084853, "n" ], [ 0.084920, "g" ], [ 0.178519, " " ], [ 0.256151, "o" ], [ 0.217918, "k" ], [ 0.153899, "!" ], [ 0.246211, "\r\n" ], [ 0.000330, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 2.000000, "#" ], [ 0.288008, " " ], [ 0.232836, "i" ], [ 0.055326, "n" ], [ 0.142978, "s" ], [ 0.080599, "t" ], [ 0.139018, "a" ], [ 0.111052, "l" ], [ 0.132419, "l" ], [ 0.169037, " " ], [ 0.117036, "t" ], [ 0.092749, "h" ], [ 0.124768, "e" ], [ 0.088888, " " ], [ 0.184118, "b" ], [ 0.182336, "i" ], [ 0.075466, "n" ], [ 0.085516, "a" ], [ 0.060363, "r" ], [ 0.843225, "y" ], [ 0.209758, " " ], [ 0.168892, "a" ], [ 0.151126, "s" ], [ 0.127487, " " ], [ 0.300923, "\"" ], [ 0.217060, "b" ], [ 0.221579, "o" ], [ 0.047775, "r" ], [ 0.107202, "g" ], [ 0.438939, "\"" ], [ 0.253153, ":" ], [ 0.617823, "\r\n" ], [ 0.000326, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.816740, "c" ], [ 0.168734, "p" ], [ 0.230846, " " ], [ 0.299588, "b" ], [ 0.121082, "o" ], [ 0.214148, "\u0007" ], [ 0.000011, "rg-linux64" ], [ 0.331736, " " ], [ 0.812264, "~" ], [ 0.518926, "/" ], [ 0.233797, "b" ], [ 0.214141, "i" ], [ 0.098062, "n" ], [ 0.607725, "/" ], [ 0.566434, "b" ], [ 0.145886, "o" ], [ 0.113081, "r" ], [ 0.068870, "g" ], [ 0.851794, "\r\n" ], [ 0.012632, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 2.000000, "#" ], [ 0.269926, " " ], [ 0.208575, "m" ], [ 0.135192, "a" ], [ 0.119543, "k" ], [ 0.080873, "e" ], [ 0.156871, " " ], [ 0.197124, "i" ], [ 0.078784, "t" ], [ 0.142373, " " ], [ 0.189080, "e" ], [ 0.232597, "x" ], [ 0.170105, "e" ], [ 0.132039, "c" ], [ 0.230568, "u" ], [ 0.086573, "t" ], [ 0.255047, "a" ], [ 0.231478, "b" ], [ 0.283723, "l" ], [ 0.112987, "e" ], [ 0.518611, ":" ], [ 0.459423, "\r\n" ], [ 0.000822, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.353739, "c" ], [ 0.114161, "h" ], [ 0.268562, "m" ], [ 0.179085, "o" ], [ 0.145360, "d" ], [ 0.075599, " " ], [ 0.773964, "+" ], [ 0.113699, "x" ], [ 0.187579, " " ], [ 0.381391, "~" ], [ 0.512520, "/" ], [ 0.231090, "b" ], [ 0.197636, "i" ], [ 0.101238, "n" ], [ 0.341295, "/" ], [ 0.306047, "b" ], [ 0.106898, "o" ], [ 0.233773, "rg " ], [ 0.519336, "\r\n" ], [ 0.001408, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 2.000000, "#" ], [ 0.247104, " " ], [ 0.218717, "i" ], [ 0.067769, "n" ], [ 0.139583, "s" ], [ 0.092034, "t" ], [ 0.152729, "a" ], [ 0.083844, "l" ], [ 0.145806, "l" ], [ 0.120879, "a" ], [ 0.164967, "t" ], [ 0.065308, "i" ], [ 0.816983, "o" ], [ 0.231669, "n" ], [ 0.185168, " " ], [ 0.125214, "d" ], [ 0.112630, "o" ], [ 0.068650, "n" ], [ 0.108386, "e" ], [ 0.563031, "!" ], [ 2.000000, "\r\n" ], [ 0.000365, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.347093, "#" ], [ 0.262764, " " ], [ 0.191568, "l" ], [ 0.086614, "e" ], [ 0.110365, "t" ], [ 0.707057, "'" ], [ 0.220060, "s" ], [ 0.181690, " " ], [ 0.128039, "c" ], [ 0.176264, "r" ], [ 0.171208, "e" ], [ 0.199371, "a" ], [ 0.161622, "t" ], [ 0.145989, "e" ], [ 0.187920, " " ], [ 0.734653, "a" ], [ 0.185812, " " ], [ 0.270851, "r" ], [ 0.120000, "e" ], [ 0.161097, "p" ], [ 0.179813, "o" ], [ 0.170557, "s" ], [ 0.145457, "i" ], [ 0.165200, "t" ], [ 0.135578, "o" ], [ 0.130363, "r" ], [ 0.461631, "y" ], [ 0.303047, ":" ], [ 0.955198, "\r\n" ], [ 0.000300, "\u001b]0;tw@tux: ~/borg/demo/download\u0007tw@tux:~/borg/demo/download$ " ], [ 0.301237, "c" ], [ 0.084267, "d" ], [ 0.155241, " " ], [ 0.813751, "." ], [ 0.157147, "." ], [ 0.573720, "\r\n" ], [ 0.000508, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.225463, "b" ], [ 0.274841, "o" ], [ 0.125292, "r" ], [ 0.083313, "g" ], [ 0.088596, " " ], [ 0.231502, "i" ], [ 0.062726, "n" ], [ 0.144877, "i" ], [ 0.112508, "t" ], [ 0.313489, " " ], [ 0.298944, "r" ], [ 0.216556, "e" ], [ 0.180484, "p" ], [ 0.204141, "o" ], [ 0.682782, "\r\n" ], [ 0.352828, "Initializing repository at \"repo\"\r\n" ], [ 0.001407, "Encryption NOT enabled.\r\nUse the \"--encryption=repokey|keyfile|passphrase\" to enable encryption." ], [ 0.000009, "\r\n" ], [ 0.008492, "Synchronizing chunks cache..." ], [ 0.000009, "\r\n" ], [ 0.000030, "Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0." ], [ 0.000004, "\r\n" ], [ 0.000024, "Done." ], [ 0.000004, "\r\n" ], [ 0.027827, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.988184, "#" ], [ 0.248844, " " ], [ 0.199486, "l" ], [ 0.104455, "e" ], [ 0.127960, "t" ], [ 0.484976, "'" ], [ 0.186103, "s" ], [ 0.151763, " " ], [ 0.177456, "c" ], [ 0.178972, "r" ], [ 0.183533, "e" ], [ 0.192725, "a" ], [ 0.146352, "t" ], [ 0.156199, "e" ], [ 0.232699, " " ], [ 0.513490, "o" ], [ 0.229828, "u" ], [ 0.104744, "r" ], [ 0.115068, " " ], [ 0.201439, "f" ], [ 0.333315, "i" ], [ 0.209070, "r" ], [ 0.259194, "s" ], [ 0.076346, "t" ], [ 0.125673, " " ], [ 0.198575, "b" ], [ 0.089009, "a" ], [ 0.238307, "c" ], [ 0.105568, "k" ], [ 0.254971, "u" ], [ 0.318094, "p" ], [ 0.690770, ":" ], [ 0.580155, "\r\n" ], [ 0.000308, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.603046, "b" ], [ 0.104492, "o" ], [ 0.148182, "r" ], [ 0.087024, "g" ], [ 0.176897, " " ], [ 0.183168, "c" ], [ 0.185325, "r" ], [ 0.183347, "e" ], [ 0.182868, "a" ], [ 0.170600, "t" ], [ 0.137005, "e" ], [ 0.164357, " " ], [ 0.427028, "-" ], [ 0.147791, "-" ], [ 0.440101, "s" ], [ 0.177193, "t" ], [ 0.203817, "a" ], [ 0.217150, "t" ], [ 0.229771, "s" ], [ 0.191220, " " ], [ 0.269939, "-" ], [ 0.145163, "-" ], [ 0.450053, "p" ], [ 0.165194, "r" ], [ 0.044264, "o" ], [ 0.204568, "g" ], [ 0.104759, "r" ], [ 0.213137, "e" ], [ 0.216596, "s" ], [ 0.163238, "s" ], [ 0.241084, " " ], [ 0.300727, "-" ], [ 0.149156, "-" ], [ 0.259608, "c" ], [ 0.120930, "o" ], [ 0.098838, "m" ], [ 0.234615, "p" ], [ 0.084600, "r" ], [ 0.166072, "e" ], [ 0.185576, "s" ], [ 0.159984, "s" ], [ 0.122793, "i" ], [ 0.180423, "o" ], [ 0.196311, "n" ], [ 0.181682, " " ], [ 0.242129, "l" ], [ 0.842020, "z" ], [ 0.707941, "4" ], [ 0.180354, " " ], [ 0.419080, "r" ], [ 0.189076, "e" ], [ 0.172527, "p" ], [ 0.154922, "o" ], [ 0.728059, ":" ], [ 0.147089, ":" ], [ 0.396117, "b" ], [ 0.090233, "a" ], [ 0.199537, "c" ], [ 0.084686, "k" ], [ 0.278049, "u \r" ], [ 0.268438, "p" ], [ 0.491592, "1" ], [ 0.508588, " " ], [ 0.174143, "d" ], [ 0.175430, "a" ], [ 0.166841, "t" ], [ 0.127029, "a" ], [ 0.380593, "\r\n" ], [ 0.557518, " 2.68 MB O 1.25 MB C 1.25 MB D data/linux-4.1.8/Doc...ia/v4l/func-read.xml\r" ], [ 0.200102, " 5.37 MB O 2.46 MB C 2.46 MB D data/linux-4.1.8/Documentation/backlight \r" ], [ 0.200342, " 6.99 MB O 3.36 MB C 3.36 MB D data/linux-4.1.8/Doc...rm_big_little_dt.txt\r" ], [ 0.200137, " 7.83 MB O 3.87 MB C 3.87 MB D data/linux-4.1.8/Doc...s/mtd/atmel-nand.txt\r" ], [ 0.200271, " 8.77 MB O 4.41 MB C 4.41 MB D data/linux-4.1.8/Doc...ngs/soc/fsl/qman.txt\r" ], [ 0.200577, " 9.99 MB O 5.12 MB C 5.12 MB D data/linux-4.1.8/Doc...ching/cachefiles.txt\r" ], [ 0.200033, " 12.11 MB O 6.34 MB C 6.33 MB D data/linux-4.1.8/Doc...infiniband/ipoib.txt\r" ], [ 0.200272, " 15.27 MB O 8.08 MB C 8.08 MB D data/linux-4.1.8/Doc.../networking/team.txt\r" ], [ 0.200072, " 18.22 MB O 9.72 MB C 9.71 MB D data/linux-4.1.8/Doc...tation/sysctl/vm.txt\r" ], [ 0.202107, " 21.05 MB O 11.22 MB C 11.21 MB D data/linux-4.1.8/MAINTAINERS \r" ], [ 0.200251, " 23.04 MB O 12.20 MB C 12.20 MB D data/linux-4.1.8/arc...de/uapi/asm/unistd.h\r" ], [ 0.200450, " 25.45 MB O 13.17 MB C 13.17 MB D data/linux-4.1.8/arc.../boot/dts/imx23.dtsi\r" ], [ 0.200093, " 27.65 MB O 14.01 MB C 14.00 MB D data/linux-4.1.8/arc...omap3-overo-tobi.dts\r" ], [ 0.200314, " 30.26 MB O 14.89 MB C 14.89 MB D data/linux-4.1.8/arc...ot/dts/tps65910.dtsi\r" ], [ 0.200003, " 31.90 MB O 15.63 MB C 15.63 MB D data/linux-4.1.8/arc...include/asm/probes.h\r" ], [ 0.200493, " 34.66 MB O 16.84 MB C 16.83 MB D data/linux-4.1.8/arc...i/include/mach/psc.h\r" ], [ 0.200675, " 36.62 MB O 17.70 MB C 17.70 MB D data/linux-4.1.8/arc...mach-ixp4xx/common.c\r" ], [ 0.200307, " 38.40 MB O 18.54 MB C 18.53 MB D data/linux-4.1.8/arch/arm/mach-omap2/cm.h \r" ], [ 0.200254, " 41.29 MB O 19.63 MB C 19.63 MB D data/linux-4.1.8/arch/arm/mach-pxa/idp.c \r" ], [ 0.219493, " 43.57 MB O 20.67 MB C 20.66 MB D data/linux-4.1.8/arc...bile/clock-r8a7778.c\r" ], [ 0.200451, " 45.55 MB O 21.59 MB C 21.59 MB D data/linux-4.1.8/arc...m/plat-samsung/adc.c\r" ], [ 0.200370, " 47.50 MB O 22.51 MB C 22.51 MB D data/linux-4.1.8/arch/arm64/lib/memmove.S \r" ], [ 0.200686, " 49.21 MB O 23.33 MB C 23.32 MB D data/linux-4.1.8/arc...ckfin/kernel/trace.c\r" ], [ 0.200393, " 53.22 MB O 24.51 MB C 24.50 MB D data/linux-4.1.8/arch/c6x/include/asm/soc.h\r" ], [ 0.200371, " 56.19 MB O 25.50 MB C 25.49 MB D data/linux-4.1.8/arc...op/iop_sw_cpu_defs.h\r" ], [ 0.200450, " 57.84 MB O 26.17 MB C 26.14 MB D data/linux-4.1.8/arc...include/asm/vm_mmu.h\r" ], [ 0.200573, " 60.21 MB O 27.27 MB C 27.25 MB D data/linux-4.1.8/arch/ia64/kernel/time.c \r" ], [ 0.200222, " 62.31 MB O 28.18 MB C 28.15 MB D data/linux-4.1.8/arc.../coldfire/sltimers.c\r" ], [ 0.200756, " 67.09 MB O 29.98 MB C 29.90 MB D data/linux-4.1.8/arc...8k/include/asm/tlb.h\r" ], [ 0.200716, " 68.75 MB O 30.80 MB C 30.72 MB D data/linux-4.1.8/arc...ude/uapi/asm/fcntl.h\r" ], [ 0.200734, " 70.69 MB O 31.67 MB C 31.59 MB D data/linux-4.1.8/arc...figs/malta_defconfig\r" ], [ 0.200198, " 72.12 MB O 32.34 MB C 32.26 MB D data/linux-4.1.8/arc...de/asm/mc146818rtc.h\r" ], [ 0.200446, " 76.01 MB O 33.45 MB C 33.37 MB D data/linux-4.1.8/arch/mips/jazz/jazzdma.c \r" ], [ 0.200111, " 78.19 MB O 34.46 MB C 34.38 MB D data/linux-4.1.8/arc...tlogic/common/time.c\r" ], [ 0.200191, " 79.84 MB O 35.30 MB C 35.21 MB D data/linux-4.1.8/arc...de/uapi/asm/msgbuf.h\r" ], [ 0.200421, " 81.35 MB O 36.07 MB C 35.99 MB D data/linux-4.1.8/arc...sc/include/asm/rtc.h\r" ], [ 0.200090, " 83.49 MB O 37.03 MB C 36.95 MB D data/linux-4.1.8/arc...fsl/qoriq-dma-1.dtsi\r" ], [ 0.200331, " 85.13 MB O 37.80 MB C 37.72 MB D data/linux-4.1.8/arc...pc836x_rdk_defconfig\r" ], [ 0.200114, " 87.04 MB O 38.71 MB C 38.63 MB D data/linux-4.1.8/arc...ude/uapi/asm/nvram.h\r" ], [ 0.200280, " 90.24 MB O 40.19 MB C 40.11 MB D data/linux-4.1.8/arc...pc/math-emu/mtfsfi.c\r" ], [ 0.216796, " 92.68 MB O 41.41 MB C 41.33 MB D data/linux-4.1.8/arc...rms/powermac/nvram.c\r" ], [ 0.200198, " 95.32 MB O 42.60 MB C 42.52 MB D data/linux-4.1.8/arc...nclude/asm/pgtable.h\r" ], [ 0.200304, " 97.31 MB O 43.50 MB C 43.42 MB D data/linux-4.1.8/arc...mach-dreamcast/irq.c\r" ], [ 0.200328, " 99.46 MB O 44.41 MB C 44.33 MB D data/linux-4.1.8/arc...artner-jet-setup.txt\r" ], [ 0.200102, "101.28 MB O 45.25 MB C 45.16 MB D data/linux-4.1.8/arc...rc/include/asm/ecc.h\r" ], [ 0.200253, "103.53 MB O 46.27 MB C 46.19 MB D data/linux-4.1.8/arc.../kernel/una_asm_64.S\r" ], [ 0.200503, "105.76 MB O 47.32 MB C 47.23 MB D data/linux-4.1.8/arch/tile/kernel/reboot.c \r" ], [ 0.201177, "107.64 MB O 48.27 MB C 48.18 MB D data/linux-4.1.8/arc...t/compressed/eboot.c\r" ], [ 0.200192, "109.82 MB O 49.22 MB C 49.13 MB D data/linux-4.1.8/arc...clude/asm/spinlock.h\r" ], [ 0.200851, "112.71 MB O 50.56 MB C 50.48 MB D data/linux-4.1.8/arch/x86/kernel/ptrace.c \r" ], [ 0.200195, "115.71 MB O 51.96 MB C 51.87 MB D data/linux-4.1.8/arc...s/platform_emc1403.c\r" ], [ 0.200306, "117.28 MB O 52.79 MB C 52.70 MB D data/linux-4.1.8/arc...nclude/variant/tie.h\r" ], [ 0.204475, "122.68 MB O 55.35 MB C 55.26 MB D data/linux-4.1.8/fir...x-e1-6.2.9.0.fw.ihex\r" ], [ 0.199974, "127.39 MB O 58.15 MB C 57.97 MB D data/linux-4.1.8/fs/afs/fsclient.c \r" ], [ 0.201254, "132.58 MB O 60.42 MB C 60.24 MB D data/linux-4.1.8/fs/cifs/cifssmb.c \r" ], [ 0.216710, "136.76 MB O 62.28 MB C 62.10 MB D data/linux-4.1.8/fs/ext4/inline.c \r" ], [ 0.200891, "140.78 MB O 64.15 MB C 63.97 MB D data/linux-4.1.8/fs/jbd2/commit.c \r" ], [ 0.199883, "144.88 MB O 65.98 MB C 65.80 MB D data/linux-4.1.8/fs/nfs/objlayout \r" ], [ 0.201488, "150.31 MB O 67.96 MB C 67.78 MB D data/linux-4.1.8/fs/...fy/dnotify/dnotify.c\r" ], [ 0.205472, "154.72 MB O 69.97 MB C 69.79 MB D data/linux-4.1.8/fs/quota/dquot.c \r" ], [ 0.200493, "159.06 MB O 71.91 MB C 71.73 MB D data/linux-4.1.8/fs/...xfs/xfs_inode_fork.h\r" ], [ 0.200000, "161.54 MB O 73.09 MB C 72.91 MB D data/linux-4.1.8/inc.../crypto/public_key.h\r" ], [ 0.205041, "164.32 MB O 74.28 MB C 74.09 MB D data/linux-4.1.8/inc...inux/cgroup_subsys.h\r" ], [ 0.200371, "166.33 MB O 75.23 MB C 75.05 MB D data/linux-4.1.8/include/linux/if_team.h \r" ], [ 0.200340, "168.82 MB O 76.24 MB C 76.06 MB D data/linux-4.1.8/inc.../mfd/pcf50633/gpio.h\r" ], [ 0.200162, "171.65 MB O 77.36 MB C 77.17 MB D data/linux-4.1.8/include/linux/phy.h \r" ], [ 0.200385, "172.84 MB O 77.97 MB C 77.79 MB D data/linux-4.1.8/include/linux/scc.h \r" ], [ 0.200918, "174.87 MB O 78.94 MB C 78.76 MB D data/linux-4.1.8/include/linux/wait.h \r" ], [ 0.200117, "177.06 MB O 80.01 MB C 79.83 MB D data/linux-4.1.8/inc...er/nfnetlink_queue.h\r" ], [ 0.200254, "179.53 MB O 81.13 MB C 80.95 MB D data/linux-4.1.8/inc...e/events/intel-sst.h\r" ], [ 0.200176, "181.40 MB O 82.05 MB C 81.86 MB D data/linux-4.1.8/include/uapi/linux/mpls.h \r" ], [ 0.200438, "183.11 MB O 82.88 MB C 82.70 MB D data/linux-4.1.8/inc...api/scsi/fc/fc_els.h\r" ], [ 0.200226, "186.12 MB O 84.31 MB C 84.12 MB D data/linux-4.1.8/kernel/jump_label.c \r" ], [ 0.200138, "190.76 MB O 86.46 MB C 86.28 MB D data/linux-4.1.8/lib/Kconfig.debug \r" ], [ 0.200958, "194.21 MB O 87.82 MB C 87.64 MB D data/linux-4.1.8/mm/memblock.c \r" ], [ 0.200544, "198.19 MB O 89.69 MB C 89.51 MB D data/linux-4.1.8/net/bluetooth/ecc.c \r" ], [ 0.200232, "202.28 MB O 91.52 MB C 91.34 MB D data/linux-4.1.8/net/hsr/hsr_slave.c \r" ], [ 0.200153, "206.23 MB O 93.40 MB C 93.22 MB D data/linux-4.1.8/net/ipx/af_ipx.c \r" ], [ 0.200526, "210.30 MB O 95.08 MB C 94.89 MB D data/linux-4.1.8/net...ter/ipvs/ip_vs_ftp.c\r" ], [ 0.200433, "213.29 MB O 96.37 MB C 96.19 MB D data/linux-4.1.8/net/phonet/af_phonet.c \r" ], [ 0.200669, "217.21 MB O 98.21 MB C 98.03 MB D data/linux-4.1.8/net.../svc_rdma_recvfrom.c\r" ], [ 0.200014, "220.20 MB O 99.53 MB C 99.35 MB D data/linux-4.1.8/scr...e/free/iounmap.cocci\r" ], [ 0.200446, "222.94 MB O 100.82 MB C 100.64 MB D data/linux-4.1.8/security/selinux/Makefile \r" ], [ 0.214711, "226.41 MB O 102.23 MB C 102.05 MB D data/linux-4.1.8/sou...seq/seq_midi_event.c\r" ], [ 0.202631, "228.96 MB O 103.31 MB C 103.13 MB D data/linux-4.1.8/sound/mips/ad1843.c \r" ], [ 0.200095, "232.28 MB O 104.65 MB C 104.47 MB D data/linux-4.1.8/sound/pci/ctxfi/Makefile \r" ], [ 0.200726, "236.33 MB O 106.24 MB C 106.06 MB D data/linux-4.1.8/sound/pci/nm256/Makefile \r" ], [ 0.199902, "239.73 MB O 107.58 MB C 107.40 MB D data/linux-4.1.8/sou.../codecs/cs4271-spi.c\r" ], [ 0.200592, "244.29 MB O 109.08 MB C 108.90 MB D data/linux-4.1.8/sound/soc/codecs/wm8940.c \r" ], [ 0.200357, "247.98 MB O 110.35 MB C 110.17 MB D data/linux-4.1.8/sou...oc/omap/omap-mcpdm.c\r" ], [ 0.200901, "250.64 MB O 111.50 MB C 111.32 MB D data/linux-4.1.8/sound/usb/mixer_scarlett.c\r" ], [ 0.200535, "252.14 MB O 112.20 MB C 112.01 MB D data/linux-4.1.8/tools/perf/builtin-kvm.c \r" ], [ 0.200239, "254.11 MB O 113.07 MB C 112.88 MB D data/linux-4.1.8/tools/perf/util/record.c \r" ], [ 0.200233, "255.70 MB O 113.82 MB C 113.64 MB D data/linux-4.1.8/too...re/bin/configinit.sh\r" ], [ 0.395702, " \r------------------------------------------------------------------------------\r\nArchive name: backup1\r\nArchive fingerprint: b3104802be9faa610f281619c69e4d3e672df2ce97528a35d83f15080d02ed86\r\nStart time: Sat Oct 24 22:27:24 2015\r\nEnd time: Sat Oct 24 22:27:43 2015\r\nDuration: 19.32 seconds\r\nNumber of files: 31557\r\n\r\n Original size Compressed size Deduplicated size\r\nThis archive: 257.06 MB 114.44 MB 114.26 MB\r\nAll archives: 257.06 MB 114.44 MB 114.26 MB\r\n\r\n Unique chunks Total chunks\r\nChunk index: 33731 34030\r\n------------------------------------------------------------------------------\r\n" ], [ 0.046138, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 1.000000, "#" ], [ 0.564664, " " ], [ 0.313339, "c" ], [ 0.492152, "h" ], [ 0.479518, "a" ], [ 0.536708, "n" ], [ 0.134006, "g" ], [ 0.147326, "e" ], [ 0.068957, " " ], [ 0.179678, "s" ], [ 0.096249, "o" ], [ 0.081003, "m" ], [ 0.124342, "e" ], [ 0.117830, " " ], [ 0.138019, "d" ], [ 0.137898, "a" ], [ 0.199628, "t" ], [ 0.104935, "a" ], [ 0.150868, " " ], [ 0.144877, "s" ], [ 0.126816, "l" ], [ 0.178466, "i" ], [ 0.113395, "g" ], [ 0.101022, "h" ], [ 0.102395, "t" ], [ 0.311498, "l" ], [ 0.366608, "y" ], [ 0.657991, ":" ], [ 0.423140, "\r\n" ], [ 0.000708, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 2.000000, "e" ], [ 0.000021, "c" ], [ 0.000024, "h" ], [ 0.000029, "o" ], [ 0.000018, " " ], [ 0.000025, "\"" ], [ 0.000025, "s" ], [ 0.000026, "o" ], [ 0.000070, "me " ], [ 0.000022, "m" ], [ 0.000028, "o" ], [ 0.000027, "r" ], [ 0.000026, "e" ], [ 0.000029, " " ], [ 0.000028, "d" ], [ 0.000028, "a" ], [ 0.000028, "t" ], [ 0.000026, "a" ], [ 0.000033, "\"" ], [ 0.000028, " " ], [ 0.000059, "\u003e" ], [ 0.000045, " " ], [ 0.000020, "d" ], [ 0.000040, "a" ], [ 0.000035, "t" ], [ 0.000039, "a" ], [ 0.000034, "/" ], [ 0.000034, "o" ], [ 0.000034, "n" ], [ 0.000037, "e" ], [ 0.000036, "_" ], [ 0.000037, "f" ], [ 0.000037, "i" ], [ 0.000717, "le_more\r\n" ], [ 0.000181, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 2.000000, "#" ], [ 0.266289, " " ], [ 0.194686, "n" ], [ 0.157296, "o" ], [ 0.084026, "w" ], [ 0.092729, " " ], [ 0.148154, "c" ], [ 0.169136, "r" ], [ 0.214327, "e" ], [ 0.180678, "a" ], [ 0.161652, "t" ], [ 0.128260, "e" ], [ 0.158131, " " ], [ 0.118838, "a" ], [ 0.120885, " " ], [ 0.797511, "s" ], [ 0.200585, "e" ], [ 0.171811, "c" ], [ 0.106721, "o" ], [ 0.153298, "n" ], [ 0.052244, "d" ], [ 0.149675, " " ], [ 0.183517, "b" ], [ 0.076768, "a" ], [ 0.189428, "c" ], [ 0.088431, "k" ], [ 0.229617, "u" ], [ 0.272021, "p" ], [ 0.965855, ":" ], [ 0.674517, "\r\n" ], [ 0.000322, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.946131, "b" ], [ 0.111159, "o" ], [ 0.094622, "r" ], [ 0.085288, "g" ], [ 0.165429, " " ], [ 0.936087, "c" ], [ 0.192608, "r" ], [ 0.187511, "e" ], [ 0.173135, "a" ], [ 0.179441, "t" ], [ 0.125923, "e" ], [ 0.164920, " " ], [ 0.737259, "-" ], [ 0.185417, "-" ], [ 0.233405, "s" ], [ 0.152945, "t" ], [ 0.181548, "a" ], [ 0.330237, "t" ], [ 0.735524, "s" ], [ 0.179019, " " ], [ 0.245324, "-" ], [ 0.142362, "-" ], [ 0.233989, "p" ], [ 0.153782, "r" ], [ 0.064431, "o" ], [ 0.104827, "g" ], [ 0.090533, "r" ], [ 0.168129, "e" ], [ 0.206325, "s" ], [ 0.157551, "s" ], [ 0.383630, " " ], [ 0.759364, "r" ], [ 0.199262, "e" ], [ 0.139781, "p" ], [ 0.151367, "o" ], [ 0.720350, ":" ], [ 0.144801, ":" ], [ 0.532566, "b" ], [ 0.226514, "a" ], [ 0.209449, "c" ], [ 0.142062, "k" ], [ 0.300090, "u" ], [ 0.262794, "p" ], [ 0.218785, "2" ], [ 0.249599, " " ], [ 0.187125, "d" ], [ 0.157741, "a" ], [ 0.175739, "t" ], [ 0.139896, "a" ], [ 0.795560, "\r\n" ], [ 0.571808, " 6.50 MB O 3.09 MB C 0 B D data/linux-4.1.8/Doc...ngs/arm/armadeus.txt\r" ], [ 0.200103, " 11.82 MB O 6.17 MB C 0 B D data/linux-4.1.8/Documentation/hwmon/w83795\r" ], [ 0.200121, " 27.38 MB O 13.89 MB C 0 B D data/linux-4.1.8/arc...ot/dts/nspire-cx.dts\r" ], [ 0.200110, " 39.92 MB O 19.04 MB C 0 B D data/linux-4.1.8/arc...omap2/opp2430_data.c\r" ], [ 0.200088, " 52.28 MB O 24.23 MB C 0 B D data/linux-4.1.8/arc...fin/mach-bf561/smp.c\r" ], [ 0.200078, " 67.02 MB O 29.94 MB C 0 B D data/linux-4.1.8/arc...8k/include/asm/pci.h\r" ], [ 0.200116, " 78.29 MB O 34.52 MB C 0 B D data/linux-4.1.8/arc...etlogic/xlr/wakeup.c\r" ], [ 0.200081, " 90.07 MB O 40.11 MB C 0 B D data/linux-4.1.8/arc...eature-fixups-test.S\r" ], [ 0.200092, "101.15 MB O 45.19 MB C 0 B D data/linux-4.1.8/arc...rc/crypto/sha1_asm.S\r" ], [ 0.200078, "115.05 MB O 51.63 MB C 0 B D data/linux-4.1.8/arc...6/mm/kasan_init_64.c\r" ], [ 0.200062, "147.39 MB O 66.98 MB C 0 B D data/linux-4.1.8/fs/nls/nls_cp864.c \r" ], [ 0.200117, "169.16 MB O 76.38 MB C 0 B D data/linux-4.1.8/inc.../mfd/twl4030-audio.h\r" ], [ 0.200074, "181.43 MB O 82.06 MB C 0 B D data/linux-4.1.8/include/uapi/linux/mtio.h \r" ], [ 0.200131, "209.10 MB O 94.58 MB C 0 B D data/linux-4.1.8/net/mac80211/scan.c \r" ], [ 0.200079, "234.87 MB O 105.68 MB C 0 B D data/linux-4.1.8/sou...i/hda/patch_si3054.c\r" ], [ 0.200110, "255.66 MB O 113.80 MB C 0 B D data/linux-4.1.8/too...ves/asm/asm-compat.h\r" ], [ 0.201350, " \r------------------------------------------------------------------------------\r\nArchive name: backup2\r\nArchive fingerprint: 5737afe8ad2cda7667973b7f2e1d83f097ef3117b5753a38ba7664b616fbdc5a\r\nStart time: Sat Oct 24 22:28:24 2015\r\nEnd time: Sat Oct 24 22:28:27 2015\r\nDuration: 3.41 seconds\r\nNumber of files: 31557\r\n" ], [ 0.001858, "\r\n Original size Compressed size Deduplicated size\r\nThis archive: 257.06 MB 114.47 MB 45.19 kB\r\nAll archives: 514.12 MB 228.92 MB 114.31 MB\r\n\r\n Unique chunks Total chunks\r\nChunk index: 33733 68060\r\n------------------------------------------------------------------------------\r\n" ], [ 0.033369, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 1.013315, "#" ], [ 0.482095, " " ], [ 0.319571, "w" ], [ 0.140740, "o" ], [ 0.090380, "w" ], [ 0.304400, "," ], [ 0.137310, " " ], [ 0.662280, "t" ], [ 0.162678, "h" ], [ 0.114083, "a" ], [ 0.077660, "t" ], [ 0.120839, " " ], [ 0.207626, "w" ], [ 0.195480, "a" ], [ 0.060188, "s" ], [ 0.149129, " " ], [ 0.094522, "a" ], [ 0.098801, " " ], [ 0.266235, "l" ], [ 0.184774, "o" ], [ 0.255040, "t" ], [ 0.170498, " " ], [ 0.201599, "f" ], [ 0.189774, "a" ], [ 0.229140, "s" ], [ 0.275243, "t" ], [ 0.177347, "e" ], [ 0.090806, "r" ], [ 0.204494, "!" ], [ 0.479851, "\r\n" ], [ 0.000316, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.734961, "#" ], [ 0.300000, " " ], [ 0.300000, "n" ], [ 0.199297, "o" ], [ 0.148047, "t" ], [ 0.071101, "i" ], [ 0.185554, "c" ], [ 0.395933, "e" ], [ 0.180285, " " ], [ 0.199321, "t" ], [ 0.094767, "h" ], [ 0.166966, "a" ], [ 0.102814, "t" ], [ 0.415016, " " ], [ 0.286089, "\"" ], [ 0.795323, "D" ], [ 0.180152, "e" ], [ 0.311214, "d" ], [ 0.214812, "u" ], [ 0.251616, "p" ], [ 0.203533, "l" ], [ 0.187084, "i" ], [ 0.124066, "c" ], [ 0.158062, "a" ], [ 0.260540, "t" ], [ 0.136405, "e" ], [ 0.278039, "d" ], [ 0.323148, " " ], [ 0.172337, "s" ], [ 0.074541, "i" ], [ 0.269245, "z" ], [ 0.123599, "e" ], [ 0.533647, "\"" ], [ 0.234738, " " ], [ 0.150720, "f" ], [ 0.144329, "o" ], [ 0.086533, "r" ], [ 0.159717, " " ], [ 0.274291, "\"" ], [ 0.471163, "T" ], [ 0.162135, "h" ], [ 0.233501, "i" ], [ 0.134923, "s" ], [ 0.190779, " " ], [ 0.307322, "a" ], [ 0.153882, "r" ], [ 0.246471, "c" ], [ 0.110018, "h" ], [ 0.259798, "i" ], [ 0.132853, "v" ], [ 0.171373, "e" ], [ 0.560405, "\"" ], [ 0.609162, "!" ], [ 0.559020, "\r\n" ], [ 0.000296, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.941369, "#" ], [ 0.254237, " " ], [ 0.176998, "b" ], [ 0.124943, "o" ], [ 0.097140, "r" ], [ 0.057513, "g" ], [ 0.232990, " " ], [ 0.549539, "r" ], [ 0.112992, "e" ], [ 0.240733, "c" ], [ 0.164146, "o" ], [ 0.209755, "g" ], [ 0.145638, "n" ], [ 0.151826, "i" ], [ 0.471625, "z" ], [ 0.759625, "e" ], [ 0.229566, "d" ], [ 0.254596, " " ], [ 0.209452, "t" ], [ 0.088606, "h" ], [ 0.155981, "a" ], [ 0.086797, "t" ], [ 0.098574, " " ], [ 0.243290, "m" ], [ 0.120288, "o" ], [ 0.092890, "s" ], [ 0.058823, "t" ], [ 0.125344, " " ], [ 0.211464, "f" ], [ 0.086483, "i" ], [ 0.213685, "l" ], [ 0.096764, "e" ], [ 0.176075, "s" ], [ 0.122962, " " ], [ 0.174342, "d" ], [ 0.103474, "i" ], [ 0.089744, "d" ], [ 0.181539, " " ], [ 0.461771, "n" ], [ 0.219395, "o" ], [ 0.095042, "t" ], [ 0.119662, " " ], [ 0.156060, "c" ], [ 0.116988, "h" ], [ 0.118775, "a" ], [ 0.126173, "n" ], [ 0.118518, "g" ], [ 0.109977, "e" ], [ 0.167095, " " ], [ 0.208137, "a" ], [ 0.155464, "n" ], [ 0.074939, "d" ], [ 0.616534, "\r\n" ], [ 0.000405, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.836535, "#" ], [ 0.248630, " " ], [ 0.211525, "d" ], [ 0.171252, "e" ], [ 0.244098, "d" ], [ 0.121718, "u" ], [ 0.219002, "p" ], [ 0.197839, "l" ], [ 0.161081, "i" ], [ 0.112763, "c" ], [ 0.154565, "a" ], [ 0.230427, "t" ], [ 0.180004, "e" ], [ 0.182279, "d" ], [ 0.201281, " " ], [ 0.202485, "t" ], [ 0.078397, "h" ], [ 0.178577, "e" ], [ 0.150264, "m" ], [ 0.482274, "." ], [ 0.300000, "\r\n" ], [ 0.000265, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.287124, "#" ], [ 0.314208, " " ], [ 0.219731, "n" ], [ 0.176210, "o" ], [ 0.108529, "w" ], [ 0.224056, "," ], [ 0.210976, " " ], [ 0.190508, "l" ], [ 0.098452, "e" ], [ 0.101431, "t" ], [ 0.855722, "'" ], [ 0.220403, "s" ], [ 0.229447, " " ], [ 0.134839, "e" ], [ 0.241915, "x" ], [ 0.217004, "t" ], [ 0.183774, "r" ], [ 0.231721, "a" ], [ 0.221361, "c" ], [ 0.436221, "t" ], [ 0.097256, " " ], [ 0.163933, "a" ], [ 0.099964, " " ], [ 0.216806, "b" ], [ 0.086493, "a" ], [ 0.211732, "c" ], [ 0.139016, "k" ], [ 0.379423, "u" ], [ 0.250049, "p" ], [ 0.717916, ":" ], [ 0.307136, "\r\n" ], [ 0.000301, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.967533, "m" ], [ 0.103707, "v" ], [ 0.407957, " " ], [ 0.177312, "d" ], [ 0.166158, "a" ], [ 0.242593, "t" ], [ 0.090471, "a" ], [ 0.699594, " " ], [ 0.273219, "d" ], [ 0.170371, "a" ], [ 0.169331, "t" ], [ 0.126739, "a" ], [ 0.288488, "." ], [ 0.305856, "o" ], [ 0.135252, "r" ], [ 0.152717, "i" ], [ 0.090343, "g" ], [ 0.312536, "\r\n" ], [ 0.002579, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.944876, "b" ], [ 0.149049, "o" ], [ 0.114834, "r" ], [ 0.074682, "g" ], [ 0.129000, " " ], [ 0.129618, "e" ], [ 0.261479, "x" ], [ 0.203937, "t" ], [ 0.196213, "r" ], [ 0.193561, "a" ], [ 0.215314, "c" ], [ 0.236817, "t" ], [ 0.188232, " " ], [ 0.177286, "r" ], [ 0.200598, "e" ], [ 0.105866, "p" ], [ 0.173864, "o" ], [ 0.388954, ":" ], [ 0.144865, ":" ], [ 0.347420, "b" ], [ 0.105814, "a" ], [ 0.198728, "c" ], [ 0.096349, "k" ], [ 0.261559, "u" ], [ 0.241998, "p" ], [ 0.240033, "2" ], [ 0.981903, "\r\n" ], [ 2.000000, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.937291, "#" ], [ 0.994897, " " ], [ 0.151752, "c" ], [ 0.096956, "h" ], [ 0.337975, "e" ], [ 0.207037, "c" ], [ 0.177028, "k" ], [ 0.740370, " " ], [ 0.330206, "i" ], [ 0.177976, "f" ], [ 0.218757, " " ], [ 0.329345, "o" ], [ 0.098735, "r" ], [ 0.098576, "i" ], [ 0.103157, "g" ], [ 0.107275, "i" ], [ 0.117332, "n" ], [ 0.194072, "a" ], [ 0.211456, "l" ], [ 0.197712, " " ], [ 0.189172, "d" ], [ 0.163930, "a" ], [ 0.188334, "t" ], [ 0.165129, "a" ], [ 0.220652, " " ], [ 0.224411, "a" ], [ 0.136137, "n" ], [ 0.155260, "d" ], [ 0.074238, " " ], [ 0.104154, "r" ], [ 0.690499, "e" ], [ 0.193678, "s" ], [ 0.165163, "t" ], [ 0.165594, "o" ], [ 0.111779, "r" ], [ 0.135625, "e" ], [ 0.202851, "d" ], [ 0.096040, " " ], [ 0.165090, "d" ], [ 0.155594, "a" ], [ 0.220606, "t" ], [ 0.163143, "a" ], [ 0.174099, " " ], [ 0.209780, "d" ], [ 0.166062, "i" ], [ 0.084688, "f" ], [ 0.140851, "f" ], [ 0.204458, "e" ], [ 0.088661, "r" ], [ 0.334162, "s" ], [ 0.904233, ":" ], [ 0.590489, "\r\n" ], [ 0.000283, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.503183, "d" ], [ 0.082614, "i" ], [ 0.216272, "f" ], [ 0.123813, "f" ], [ 0.183603, " " ], [ 0.302144, "-" ], [ 0.150946, "r" ], [ 0.152436, " " ], [ 2.000000, "d" ], [ 0.196047, "a" ], [ 0.206372, "t" ], [ 0.146051, "a" ], [ 0.326306, "." ], [ 0.363408, "o" ], [ 0.269988, "rig/" ], [ 0.776581, " " ], [ 0.137720, "d" ], [ 0.156080, "a" ], [ 0.242275, "\u0007" ], [ 0.000020, "ta" ], [ 0.872887, "/" ], [ 0.273993, "\r\n" ], [ 2.000000, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.488581, "#" ], [ 0.234021, " " ], [ 0.380938, "n" ], [ 0.240685, "o" ], [ 0.204436, " " ], [ 0.390794, "o" ], [ 0.225563, "u" ], [ 0.167295, "t" ], [ 0.140625, "p" ], [ 0.183668, "u" ], [ 0.106161, "t" ], [ 0.132063, " " ], [ 0.204757, "m" ], [ 0.082693, "e" ], [ 0.216428, "a" ], [ 0.121584, "n" ], [ 0.127398, "s" ], [ 0.264644, " " ], [ 0.201524, "i" ], [ 0.110738, "t" ], [ 0.120653, " " ], [ 0.311187, "d" ], [ 0.119826, "o" ], [ 0.082654, "e" ], [ 0.182518, "s" ], [ 0.096372, " " ], [ 0.192821, "n" ], [ 0.193829, "o" ], [ 0.065739, "t" ], [ 0.678808, "." ], [ 0.246797, " " ], [ 0.520369, "f" ], [ 0.058288, "i" ], [ 0.064783, "n" ], [ 0.104851, "e" ], [ 0.292910, "." ], [ 0.174086, " " ], [ 0.226556, ":" ], [ 0.249808, ")" ], [ 2.000000, "\r\n" ], [ 0.000261, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.477759, "#" ], [ 0.223699, " " ], [ 0.237979, "l" ], [ 0.152769, "i" ], [ 0.135150, "s" ], [ 0.068576, "t" ], [ 0.100516, "i" ], [ 0.078648, "n" ], [ 0.099435, "g" ], [ 0.157388, " " ], [ 0.115327, "t" ], [ 0.133738, "h" ], [ 0.135662, "e" ], [ 0.100677, " " ], [ 0.180392, "r" ], [ 0.190922, "e" ], [ 0.093920, "p" ], [ 0.173588, "o" ], [ 0.193023, " " ], [ 0.206907, "c" ], [ 0.106376, "o" ], [ 0.175291, "n" ], [ 0.080726, "t" ], [ 0.179258, "e" ], [ 0.101491, "n" ], [ 0.096807, "t" ], [ 0.211455, "s" ], [ 0.508210, ":" ], [ 0.373837, "\r\n" ], [ 0.000249, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.559782, "b" ], [ 0.116587, "o" ], [ 0.139513, "r" ], [ 0.072751, "g" ], [ 0.103968, " " ], [ 0.984928, "l" ], [ 0.173603, "i" ], [ 0.112444, "s" ], [ 0.066704, "t" ], [ 0.114771, " " ], [ 0.263745, "r" ], [ 0.113121, "e" ], [ 0.126283, "p" ], [ 0.187453, "o" ], [ 0.409044, "\r\n" ], [ 0.360675, "backup1 Sat Oct 24 22:27:43 2015" ], [ 0.000011, "\r\n" ], [ 0.000006, "backup2 Sat Oct 24 22:28:27 2015" ], [ 0.000005, "\r\n" ], [ 0.027766, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.637813, "#" ], [ 0.257629, " " ], [ 0.231710, "l" ], [ 0.213387, "i" ], [ 0.132149, "s" ], [ 0.244957, "t" ], [ 0.180264, "i" ], [ 0.082882, "n" ], [ 0.142810, "g" ], [ 0.134815, " " ], [ 0.167455, "s" ], [ 0.114155, "o" ], [ 0.106847, "m" ], [ 0.070629, "e" ], [ 0.507340, " " ], [ 0.234237, "b" ], [ 0.070181, "a" ], [ 0.220534, "c" ], [ 0.092316, "k" ], [ 0.257003, "u" ], [ 0.233598, "p" ], [ 0.201484, " " ], [ 0.124810, "a" ], [ 0.084732, "r" ], [ 0.249719, "c" ], [ 0.119605, "h" ], [ 0.203875, "i" ], [ 0.076269, "v" ], [ 0.174299, "e" ], [ 0.109711, " " ], [ 0.238294, "c" ], [ 0.102351, "o" ], [ 0.155761, "n" ], [ 0.060278, "t" ], [ 0.179564, "e" ], [ 0.112342, "n" ], [ 0.078100, "t" ], [ 0.190203, "s" ], [ 0.865560, " " ], [ 0.297799, "(" ], [ 0.225741, "s" ], [ 0.080329, "h" ], [ 0.233668, "o" ], [ 0.127773, "r" ], [ 0.190065, "t" ], [ 0.187679, "e" ], [ 0.147219, "n" ], [ 0.064472, "e" ], [ 0.188512, "d" ], [ 0.459222, ")" ], [ 0.723165, ":" ], [ 0.645995, "\r\n" ], [ 0.000096, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.446688, "b" ], [ 0.145841, "o" ], [ 0.105605, "r" ], [ 0.088953, "g" ], [ 0.120803, " " ], [ 0.227780, "l" ], [ 0.175052, "i" ], [ 0.106579, "s" ], [ 0.058441, "t" ], [ 0.093196, " " ], [ 0.172940, "r" ], [ 0.134731, "e" ], [ 0.119062, "p" ], [ 0.183075, "o" ], [ 0.388321, ":" ], [ 0.140589, ":" ], [ 0.324109, "b" ], [ 0.058606, "a" ], [ 0.205450, "c" ], [ 0.105362, "k" ], [ 0.235009, "u" ], [ 0.243485, "p" ], [ 0.485432, "2" ], [ 0.148177, " " ], [ 0.632383, "|" ], [ 0.389914, " " ], [ 0.174128, "t" ], [ 0.201473, "a" ], [ 0.116517, "i" ], [ 0.225072, "l" ], [ 0.699624, "\r\n" ], [ 2.000000, "-rw-rw-r-- tw tw 5516 Jul 21 19:10 data/linux-4.1.8/virt/kvm/async_pf.c\r\n" ], [ 0.000019, "-rw-rw-r-- tw tw 1120 Jul 21 19:10 data/linux-4.1.8/virt/kvm/async_pf.h\r\n-rw-rw-r-- tw tw 4215 Jul 21 19:10 data/linux-4.1.8/virt/kvm/coalesced_mmio.c\r\n-rw-rw-r-- tw tw 915 Jul 21 19:10 data/linux-4.1.8/virt/kvm/coalesced_mmio.h\r\n-rw-rw-r-- tw tw 22879 Jul 21 19:10 data/linux-4.1.8/virt/kvm/eventfd.c\r\n-rw-rw-r-- tw tw 5563 Jul 21 19:10 data/linux-4.1.8/virt/kvm/irqchip.c\r\n-rw-rw-r-- tw tw 79385 Jul 21 19:10 data/linux-4.1.8/virt/kvm/kvm_main.c\r\n" ], [ 0.000011, "-rw-rw-r-- tw tw 6132 Jul 21 19:10 data/linux-4.1.8/virt/kvm/vfio.c\r\n-rw-rw-r-- tw tw 250 Jul 21 19:10 data/linux-4.1.8/virt/kvm/vfio.h\r\n" ], [ 0.000009, "-rw-rw-r-- tw tw 15 Oct 24 22:28 data/one_file_more\r\n" ], [ 0.000389, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 1.000000, "#" ], [ 0.337351, " " ], [ 0.147170, "e" ], [ 0.235736, "a" ], [ 0.251314, "s" ], [ 0.471185, "y" ], [ 0.277723, "," ], [ 0.204225, " " ], [ 0.182231, "i" ], [ 0.174424, "s" ], [ 0.074677, "n" ], [ 0.786274, "'" ], [ 0.264836, "t" ], [ 0.330352, " " ], [ 0.266876, "i" ], [ 0.112564, "t" ], [ 0.897299, "?" ], [ 0.623501, " " ], [ 0.656625, "t" ], [ 0.115934, "h" ], [ 0.625213, "a" ], [ 0.588409, "t" ], [ 0.160071, " " ], [ 0.830693, "i" ], [ 0.163118, "s" ], [ 0.075663, " " ], [ 0.186138, "a" ], [ 0.109916, "l" ], [ 0.137005, "l" ], [ 0.171009, " " ], [ 0.153348, "y" ], [ 0.132919, "o" ], [ 0.568100, "u" ], [ 0.211350, " " ], [ 0.195450, "n" ], [ 0.257974, "e" ], [ 0.185529, "e" ], [ 0.265130, "d" ], [ 0.129116, " " ], [ 0.169264, "t" ], [ 0.148964, "o" ], [ 0.437043, " " ], [ 0.431197, "k" ], [ 0.219557, "n" ], [ 0.257996, "o" ], [ 0.158826, "w" ], [ 0.406870, " " ], [ 0.659664, "f" ], [ 0.130963, "o" ], [ 0.125395, "r" ], [ 0.613713, " " ], [ 0.646957, "b" ], [ 0.154695, "a" ], [ 0.259741, "s" ], [ 0.156692, "i" ], [ 0.124345, "c" ], [ 0.513209, "\r\n" ], [ 0.000296, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.965828, "#" ], [ 0.232285, " " ], [ 0.266818, "u" ], [ 0.132723, "s" ], [ 0.216208, "a" ], [ 0.206362, "g" ], [ 0.142608, "e" ], [ 2.000000, "." ], [ 0.238868, " " ], [ 0.302986, "i" ], [ 0.196338, "f" ], [ 0.092936, " " ], [ 0.197594, "y" ], [ 0.122297, "o" ], [ 0.175360, "u" ], [ 0.145063, " " ], [ 0.313719, "l" ], [ 0.169678, "i" ], [ 0.185628, "k" ], [ 0.120660, "e" ], [ 0.078389, " " ], [ 0.648628, "#" ], [ 0.337514, "b" ], [ 0.108598, "o" ], [ 0.123792, "r" ], [ 0.136099, "g" ], [ 0.235539, "b" ], [ 0.091671, "a" ], [ 0.208697, "c" ], [ 0.100567, "k" ], [ 0.227477, "u" ], [ 0.236900, "p" ], [ 0.302154, "," ], [ 0.207291, " " ], [ 0.205656, "s" ], [ 0.123737, "p" ], [ 0.142016, "r" ], [ 0.197260, "e" ], [ 0.197471, "a" ], [ 0.104498, "d" ], [ 0.163267, " " ], [ 0.178420, "t" ], [ 0.091669, "h" ], [ 0.107735, "e" ], [ 0.102742, " " ], [ 0.211413, "w" ], [ 0.124959, "o" ], [ 0.105787, "r" ], [ 0.231403, "d" ], [ 0.299061, "!" ], [ 2.000000, "\r\n" ], [ 0.000307, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.304768, "#" ], [ 0.229433, " " ], [ 0.647220, "t" ], [ 0.070692, "h" ], [ 0.349432, "a" ], [ 0.112924, "n" ], [ 0.207031, "k" ], [ 0.567641, "s" ], [ 0.121708, " " ], [ 0.135723, "f" ], [ 0.139102, "o" ], [ 0.060453, "r" ], [ 0.152408, " " ], [ 0.116234, "v" ], [ 0.142885, "i" ], [ 0.106596, "e" ], [ 0.231115, "w" ], [ 0.416046, "i" ], [ 0.086563, "n" ], [ 0.144009, "g" ], [ 0.725139, "!" ], [ 0.299810, "\r\n" ], [ 0.000250, "\u001b]0;tw@tux: ~/borg/demo\u0007tw@tux:~/borg/demo$ " ], [ 0.710767, "exit" ], [ 0.000006, "\r\n" ] ] } borgbackup-1.0.13/docs/misc/asciinema/install_and_basics.txt0000644000175000017500000000225413431513133024062 0ustar useruser00000000000000# borgbackup - installation and basic usage # I have already downloaded the binary release from github: ls -l # binary file + GPG signature # verifying whether the binary is valid: gpg --verify borg-linux64.asc borg-linux64 # install it as "borg": cp borg-linux64 ~/bin/borg # making it executable: chmod +x ~/bin/borg # yay, installation done! let's make backups! # creating a repository: borg init repo # creating our first backup with stuff from "data" directory: borg create --stats --progress --compression lz4 repo::backup1 data # changing the data slightly: echo "some more data" > data/one_file_more # creating another backup: borg create --stats --progress repo::backup2 data # that was much faster! it recognized/deduplicated unchanged files. # see the "Deduplicated size" column for "This archive"! :) # extracting a backup archive: mv data data.orig borg extract repo::backup2 # checking if restored data differs from original data: diff -r data.orig data # no, it doesn't! :) # listing the repo contents: borg list repo # listing the backup2 archive contents (shortened): borg list repo::backup2 | tail # easy, isn't it? # if you like #borgbackup, spread the word! borgbackup-1.0.13/docs/misc/create_chunker-params.txt0000644000175000017500000001037013431513133022556 0ustar useruser00000000000000About borg create --chunker-params ================================== --chunker-params CHUNK_MIN_EXP,CHUNK_MAX_EXP,HASH_MASK_BITS,HASH_WINDOW_SIZE CHUNK_MIN_EXP and CHUNK_MAX_EXP give the exponent N of the 2^N minimum and maximum chunk size. Required: CHUNK_MIN_EXP < CHUNK_MAX_EXP. Defaults: 19 (2^19 == 512KiB) minimum, 23 (2^23 == 8MiB) maximum. HASH_MASK_BITS is the number of least-significant bits of the rolling hash that need to be zero to trigger a chunk cut. Recommended: CHUNK_MIN_EXP + X <= HASH_MASK_BITS <= CHUNK_MAX_EXP - X, X >= 2 (this allows the rolling hash some freedom to make its cut at a place determined by the windows contents rather than the min/max. chunk size). Default: 21 (statistically, chunks will be about 2^21 == 2MiB in size) HASH_WINDOW_SIZE: the size of the window used for the rolling hash computation. Default: 4095B Trying it out ============= I backed up a VM directory to demonstrate how different chunker parameters influence repo size, index size / chunk count, compression, deduplication. repo-sm: ~64kiB chunks (16 bits chunk mask), min chunk size 1kiB (2^10B) (these are attic / borg 0.23 internal defaults) repo-lg: ~1MiB chunks (20 bits chunk mask), min chunk size 64kiB (2^16B) repo-xl: 8MiB chunks (2^23B max chunk size), min chunk size 64kiB (2^16B). The chunk mask bits was set to 31, so it (almost) never triggers. This degrades the rolling hash based dedup to a fixed-offset dedup as the cutting point is now (almost) always the end of the buffer (at 2^23B == 8MiB). The repo index size is an indicator for the RAM needs of Borg. In this special case, the total RAM needs are about 2.1x the repo index size. You see index size of repo-sm is 16x larger than of repo-lg, which corresponds to the ratio of the different target chunk sizes. Note: RAM needs were not a problem in this specific case (37GB data size). But just imagine, you have 37TB of such data and much less than 42GB RAM, then you'ld definitely want the "lg" chunker params so you only need 2.6GB RAM. Or even bigger chunks than shown for "lg" (see "xl"). You also see compression works better for larger chunks, as expected. Duplication works worse for larger chunks, also as expected. small chunks ============ $ borg info /extra/repo-sm::1 Command line: /home/tw/w/borg-env/bin/borg create --chunker-params 10,23,16,4095 /extra/repo-sm::1 /home/tw/win Number of files: 3 Original size Compressed size Deduplicated size This archive: 37.12 GB 14.81 GB 12.18 GB All archives: 37.12 GB 14.81 GB 12.18 GB Unique chunks Total chunks Chunk index: 378374 487316 $ ls -l /extra/repo-sm/index* -rw-rw-r-- 1 tw tw 20971538 Jun 20 23:39 index.2308 $ du -sk /extra/repo-sm 11930840 /extra/repo-sm large chunks ============ $ borg info /extra/repo-lg::1 Command line: /home/tw/w/borg-env/bin/borg create --chunker-params 16,23,20,4095 /extra/repo-lg::1 /home/tw/win Number of files: 3 Original size Compressed size Deduplicated size This archive: 37.10 GB 14.60 GB 13.38 GB All archives: 37.10 GB 14.60 GB 13.38 GB Unique chunks Total chunks Chunk index: 25889 29349 $ ls -l /extra/repo-lg/index* -rw-rw-r-- 1 tw tw 1310738 Jun 20 23:10 index.2264 $ du -sk /extra/repo-lg 13073928 /extra/repo-lg xl chunks ========= (borg-env)tw@tux:~/w/borg$ borg info /extra/repo-xl::1 Command line: /home/tw/w/borg-env/bin/borg create --chunker-params 16,23,31,4095 /extra/repo-xl::1 /home/tw/win Number of files: 3 Original size Compressed size Deduplicated size This archive: 37.10 GB 14.59 GB 14.59 GB All archives: 37.10 GB 14.59 GB 14.59 GB Unique chunks Total chunks Chunk index: 4319 4434 $ ls -l /extra/repo-xl/index* -rw-rw-r-- 1 tw tw 327698 Jun 21 00:52 index.2011 $ du -sk /extra/repo-xl/ 14253464 /extra/repo-xl/ borgbackup-1.0.13/docs/misc/internals-picture.txt0000644000175000017500000000344113431513133021764 0ustar useruser00000000000000BorgBackup from 10.000m ======================= +--------+ +--------+ +--------+ |archive0| |archive1| ... |archiveN| +--------+ +--------+ +--+-----+ | | | | | | | +---+ | | | | | | | +------+-------+ | | | | | /chunk\/chunk\/chunk\... /maybe different chunks lists\ +-----------------------------------------------------------------+ |item list | +-----------------------------------------------------------------+ | +-------------------------------------+--------------+ | | | | | | +-------------+ +-------------+ | |item0 | |item1 | | | - owner | | - owner | | | - size | | - size | ... | - ... | | - ... | | - chunks | | - chunks | +----+--------+ +-----+-------+ | | | +-----+----------------------------+-----------------+ | | | | +-o-----o------------+ | | | | | | /chunk0\/chunk1\ ... /chunkN\ /chunk0\/chunk1\ ... /chunkN'\ +-----------------------------+ +------------------------------+ |file0 | |file0' | +-----------------------------+ +------------------------------+ Thanks to anarcat for drawing the picture! borgbackup-1.0.13/docs/misc/prune-example.txt0000644000175000017500000000713313431513133021100 0ustar useruser00000000000000borg prune visualized ===================== Assume it is 2016-01-01, today's backup has not yet been made and you have created at least one backup on each day in 2015 except on 2015-12-19 (no backup made on that day). This is what borg prune --keep-daily 14 --keep-monthly 6 would keep. Backups kept by the --keep-daily rule are marked by a "d" to the right, backups kept by the --keep-monthly rule are marked by a "m" to the right. Calendar view ------------- 2015 January February March Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su 1 2 3 4 1 1 5 6 7 8 9 10 11 2 3 4 5 6 7 8 2 3 4 5 6 7 8 12 13 14 15 16 17 18 9 10 11 12 13 14 15 9 10 11 12 13 14 15 19 20 21 22 23 24 25 16 17 18 19 20 21 22 16 17 18 19 20 21 22 26 27 28 29 30 31 23 24 25 26 27 28 23 24 25 26 27 28 29 30 31 April May June Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su 1 2 3 4 5 1 2 3 1 2 3 4 5 6 7 6 7 8 9 10 11 12 4 5 6 7 8 9 10 8 9 10 11 12 13 14 13 14 15 16 17 18 19 11 12 13 14 15 16 17 15 16 17 18 19 20 21 20 21 22 23 24 25 26 18 19 20 21 22 23 24 22 23 24 25 26 27 28 27 28 29 30 25 26 27 28 29 30 31 29 30m July August September Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su 1 2 3 4 5 1 2 1 2 3 4 5 6 6 7 8 9 10 11 12 3 4 5 6 7 8 9 7 8 9 10 11 12 13 13 14 15 16 17 18 19 10 11 12 13 14 15 16 14 15 16 17 18 19 20 20 21 22 23 24 25 26 17 18 19 20 21 22 23 21 22 23 24 25 26 27 27 28 29 30 31m 24 25 26 27 28 29 30 28 29 30m 31m October November December Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su 1 2 3 4 1 1 2 3 4 5 6 5 6 7 8 9 10 11 2 3 4 5 6 7 8 7 8 9 10 11 12 13 12 13 14 15 16 17 18 9 10 11 12 13 14 15 14 15 16 17d18d19 20d 19 20 21 22 23 24 25 16 17 18 19 20 21 22 21d22d23d24d25d26d27d 26 27 28 29 30 31m 23 24 25 26 27 28 29 28d29d30d31d 30m List view --------- --keep-daily 14 --keep-monthly 6 ------------------------------------------------- 1. 2015-12-31 (2015-12-31 kept by daily rule) 2. 2015-12-30 1. 2015-11-30 3. 2015-12-29 2. 2015-10-31 4. 2015-12-28 3. 2015-09-30 5. 2015-12-27 4. 2015-08-31 6. 2015-12-26 5. 2015-07-31 7. 2015-12-25 6. 2015-06-30 8. 2015-12-24 9. 2015-12-23 10. 2015-12-22 11. 2015-12-21 12. 2015-12-20 (no backup made on 2015-12-19) 13. 2015-12-18 14. 2015-12-17 Notes ----- 2015-12-31 is kept due to the --keep-daily 14 rule (because it is applied first), not due to the --keep-monthly rule. Because of that, the --keep-monthly 6 rule keeps Nov, Oct, Sep, Aug, Jul and Jun. December is not considered for this rule, because that backup was already kept because of the daily rule. 2015-12-17 is kept to satisfy the --keep-daily 14 rule - because no backup was made on 2015-12-19. If a backup had been made on that day, it would not keep the one from 2015-12-17. We did not include yearly, weekly, hourly, minutely or secondly rules to keep this example simple. They all work in basically the same way. The weekly rule is easy to understand roughly, but hard to understand in all details. If interested, read "ISO 8601:2000 standard week-based year". borgbackup-1.0.13/docs/quickstart.rst0000644000175000017500000002545113431513133017551 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: bash .. _quickstart: Quick Start =========== This chapter will get you started with |project_name|. The first section presents a simple step by step example that uses |project_name| to backup data. The next section continues by showing how backups can be automated. Important note about free space ------------------------------- Before you start creating backups, please make sure that there is *always* a good amount of free space on the filesystem that has your backup repository (and also on ~/.cache). A few GB should suffice for most hard-drive sized repositories. See also :ref:`cache-memory-usage`. If |project_name| runs out of disk space, it tries to free as much space as it can while aborting the current operation safely, which allows to free more space by deleting/pruning archives. This mechanism is not bullet-proof though. If you *really* run out of disk space, it can be hard or impossible to free space, because |project_name| needs free space to operate - even to delete backup archives. There is a ``--save-space`` option for some commands, but even with that |project_name| will need free space to operate. You can use some monitoring process or just include the free space information in your backup log files (you check them regularly anyway, right?). Also helpful: - create a big file as a "space reserve", that you can delete to free space - if you use LVM: use a LV + a filesystem that you can resize later and have some unallocated PEs you can add to the LV. - consider using quotas - use `prune` regularly A step by step example ---------------------- 1. Before a backup can be made a repository has to be initialized:: $ borg init /path/to/repo 2. Backup the ``~/src`` and ``~/Documents`` directories into an archive called *Monday*:: $ borg create /path/to/repo::Monday ~/src ~/Documents 3. The next day create a new archive called *Tuesday*:: $ borg create -v --stats /path/to/repo::Tuesday ~/src ~/Documents This backup will be a lot quicker and a lot smaller since only new never before seen data is stored. The ``--stats`` option causes |project_name| to output statistics about the newly created archive such as the amount of unique data (not shared with other archives):: ------------------------------------------------------------------------------ Archive name: Tuesday Archive fingerprint: bd31004d58f51ea06ff735d2e5ac49376901b21d58035f8fb05dbf866566e3c2 Time (start): Tue, 2016-02-16 18:15:11 Time (end): Tue, 2016-02-16 18:15:11 Duration: 0.19 seconds Number of files: 127 ------------------------------------------------------------------------------ Original size Compressed size Deduplicated size This archive: 4.16 MB 4.17 MB 26.78 kB All archives: 8.33 MB 8.34 MB 4.19 MB Unique chunks Total chunks Chunk index: 132 261 ------------------------------------------------------------------------------ 4. List all archives in the repository:: $ borg list /path/to/repo Monday Mon, 2016-02-15 19:14:44 Tuesday Tue, 2016-02-16 19:15:11 5. List the contents of the *Monday* archive:: $ borg list /path/to/repo::Monday drwxr-xr-x user group 0 Mon, 2016-02-15 18:22:30 home/user/Documents -rw-r--r-- user group 7961 Mon, 2016-02-15 18:22:30 home/user/Documents/Important.doc ... 6. Restore the *Monday* archive by extracting the files relative to the current directory:: $ borg extract /path/to/repo::Monday 7. Recover disk space by manually deleting the *Monday* archive:: $ borg delete /path/to/repo::Monday .. Note:: Borg is quiet by default (it works on WARNING log level). Add the ``-v`` (or ``--verbose`` or ``--info``) option to adjust the log level to INFO and also use options like ``--progress`` or ``--list`` to get progress reporting during command execution. Automating backups ------------------ The following example script backs up ``/home`` and ``/var/www`` to a remote server. The script also uses the :ref:`borg_prune` subcommand to maintain a certain number of old archives:: #!/bin/sh REPOSITORY=username@remoteserver.com:backup # Setting this, so you won't be asked for your repository passphrase: export BORG_PASSPHRASE='XYZl0ngandsecurepa_55_phrasea&&123' # or this to ask an external program to supply the passphrase: export BORG_PASSCOMMAND='pass show backup' # Backup all of /home and /var/www except a few # excluded directories borg create -v --stats \ $REPOSITORY::'{hostname}-{now:%Y-%m-%d}' \ /home \ /var/www \ --exclude '/home/*/.cache' \ --exclude /home/Ben/Music/Justin\ Bieber \ --exclude '*.pyc' # Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly # archives of THIS machine. The '{hostname}-' prefix is very important to # limit prune's operation to this machine's archives and not apply to # other machine's archives also. borg prune -v --list $REPOSITORY --prefix '{hostname}-' \ --keep-daily=7 --keep-weekly=4 --keep-monthly=6 Pitfalls with shell variables and environment variables ------------------------------------------------------- This applies to all environment variables you want borg to see, not just ``BORG_PASSPHRASE``. The short explanation is: always ``export`` your variable, and use single quotes if you're unsure of the details of your shell's expansion behavior. E.g.:: export BORG_PASSPHRASE='complicated & long' This is because ``export`` exposes variables to subprocesses, which borg may be one of. More on ``export`` can be found in the "ENVIRONMENT" section of the bash(1) man page. Beware of how ``sudo`` interacts with environment variables. For example, you may be surprised that the following ``export`` has no effect on your command:: export BORG_PASSPHRASE='complicated & long' sudo ./yourborgwrapper.sh # still prompts for password For more information, refer to the sudo(8) man page and ``env_keep`` in the sudoers(5) man page. .. Tip:: To debug what your borg process is actually seeing, find its PID (``ps aux|grep borg``) and then look into ``/proc//environ``. .. backup_compression: Backup compression ------------------ Default is no compression, but we support different methods with high speed or high compression: If you have a fast repo storage and you want some compression: :: $ borg create --compression lz4 /path/to/repo::arch ~ If you have a less fast repo storage and you want a bit more compression (N=0..9, 0 means no compression, 9 means high compression): :: $ borg create --compression zlib,N /path/to/repo::arch ~ If you have a very slow repo storage and you want high compression (N=0..9, 0 means low compression, 9 means high compression): :: $ borg create --compression lzma,N /path/to/repo::arch ~ You'll need to experiment a bit to find the best compression for your use case. Keep an eye on CPU load and throughput. .. _encrypted_repos: Repository encryption --------------------- Repository encryption can be enabled or disabled at repository creation time (the default is enabled, with `repokey` method):: $ borg init --encryption=none|repokey|keyfile PATH When repository encryption is enabled all data is encrypted using 256-bit AES_ encryption and the integrity and authenticity is verified using `HMAC-SHA256`_. All data is encrypted on the client before being written to the repository. This means that an attacker who manages to compromise the host containing an encrypted archive will not be able to access any of the data, even while the backup is being made. |project_name| supports different methods to store the AES and HMAC keys. ``repokey`` mode The key is stored inside the repository (in its "config" file). Use this mode if you trust in your good passphrase giving you enough protection. The repository server never sees the plaintext key. ``keyfile`` mode The key is stored on your local disk (in ``~/.config/borg/keys/``). Use this mode if you want "passphrase and having-the-key" security. In both modes, the key is stored in encrypted form and can be only decrypted by providing the correct passphrase. For automated backups the passphrase can be specified using the `BORG_PASSPHRASE` environment variable. .. note:: Be careful about how you set that environment, see :ref:`this note about password environments ` for more information. .. warning:: The repository data is totally inaccessible without the key and the key passphrase. Make a backup copy of the key file (``keyfile`` mode) or repo config file (``repokey`` mode) and keep it at a safe place, so you still have the key in case it gets corrupted or lost. Also keep your passphrase at a safe place. You can make backups using :ref:`borg_key_export` subcommand. If you want to print a backup of your key to paper use the ``--paper`` option of this command and print the result, or this print `template`_ if you need a version with QR-Code. A backup inside of the backup that is encrypted with that key/passphrase won't help you with that, of course. .. _template: paperkey.html .. _remote_repos: Remote repositories ------------------- |project_name| can initialize and access repositories on remote hosts if the host is accessible using SSH. This is fastest and easiest when |project_name| is installed on the remote host, in which case the following syntax is used:: $ borg init user@hostname:/path/to/repo Note: please see the usage chapter for a full documentation of repo URLs. Remote operations over SSH can be automated with SSH keys. You can restrict the use of the SSH keypair by prepending a forced command to the SSH public key in the remote server's `authorized_keys` file. This example will start |project_name| in server mode and limit it to a specific filesystem path:: command="borg serve --restrict-to-path /path/to/repo",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-rsa AAAAB3[...] If it is not possible to install |project_name| on the remote host, it is still possible to use the remote host to store a repository by mounting the remote filesystem, for example, using sshfs:: $ sshfs user@hostname:/path/to /path/to $ borg init /path/to/repo $ fusermount -u /path/to You can also use other remote filesystems in a similar way. Just be careful, not all filesystems out there are really stable and working good enough to be acceptable for backup usage. borgbackup-1.0.13/docs/resources.rst0000644000175000017500000000303413431513133017362 0ustar useruser00000000000000.. include:: global.rst.inc .. _resources: Resources ========= This is a collection of additional resources that are somehow related to borgbackup. Videos, Talks, Presentations ---------------------------- Some of them refer to attic, but you can do the same stuff (and more) with borgbackup. - `BorgBackup Installation and Basic Usage `_ (english screencast) - `TW's slides for borgbackup talks / lightning talks `_ (just grab the latest ones) - `Attic / Borg Backup talk from GPN 2015 (media.ccc.de) `_ - `Attic / Borg Backup talk from GPN 2015 (youtube) `_ - `Attic talk from Easterhegg 2015 (media.ccc.de) `_ - `Attic talk from Easterhegg 2015 (youtube) `_ - `Attic Backup: Mount your encrypted backups over ssh (youtube) `_ - `Evolution of Borg (youtube) `_ Software -------- - `BorgWeb - a very simple web UI for BorgBackup `_ - some other stuff found at the `BorgBackup Github organisation `_ - `borgmatic `_ - simple wrapper script for BorgBackup that creates and prunes backups borgbackup-1.0.13/docs/support.rst0000644000175000017500000001012213431513133017060 0ustar useruser00000000000000.. include:: global.rst.inc .. _support: Support ======= Please first read the docs, the existing issue tracker issues and mailing list posts -- a lot of stuff is already documented / explained / discussed / filed there. Issue Tracker ------------- If you've found a bug or have a concrete feature request, please create a new ticket on the project's `issue tracker`_. For more general questions or discussions, IRC or mailing list are preferred. Chat (IRC) ---------- Join us on channel #borgbackup on chat.freenode.net. As usual on IRC, just ask or tell directly and then patiently wait for replies. Stay connected. You could use the following link (after connecting, you can change the random nickname you get by typing "/nick mydesirednickname"): http://webchat.freenode.net/?randomnick=1&channels=%23borgbackup&uio=MTY9dHJ1ZSY5PXRydWUa8 .. _mailing_list: Mailing list ------------ To find out about the mailing list, its topic, how to subscribe, how to unsubscribe and where you can find the archives of the list, see the `mailing list homepage `_. Twitter ------- Follow @borgbackup for announcements. You can also add @borgbackup if you would like to get retweeted for a borg related tweet. Please understand that Twitter is not suitable for longer / more complex discussions - use one of the other channels for that. Bounties and Fundraisers ------------------------ We use `BountySource `_ to allow monetary contributions to the project and the developers, who push it forward. There, you can give general funds to the borgbackup members (the developers will then spend the funds as they deem fit). If you do not have some specific bounty (see below), you can use this as a general way to say "Thank You!" and support the software / project you like. If you want to encourage developers to fix some specific issue or implement some specific feature suggestion, you can post a new bounty or back an existing one (they always refer to an issue in our `issue tracker`_). As a developer, you can become a Bounty Hunter and win bounties (earn money) by contributing to |project_name|, a free and open source software project. We might also use BountySource to fund raise for some bigger goals. .. _security-contact: Security -------- In case you discover a security issue, please use this contact for reporting it privately and please, if possible, use encrypted E-Mail: Thomas Waldmann GPG Key Fingerprint: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393 The public key can be fetched from any GPG keyserver, but be careful: you must use the **full fingerprint** to check that you got the correct key. Verifying signed releases ------------------------- `Releases `_ are signed with the same GPG key and a .asc file is provided for each binary. To verify a signature, the public key needs to be known to GPG. It can be imported into the local keystore from a keyserver with the fingerprint:: gpg --recv-keys "6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393" If GPG successfully imported the key, the output should be (among other things): 'Total number processed: 1'. To verify for example the signature of the borg-linux64 binary:: gpg --verify borg-linux64.asc GPG outputs if it finds a good signature. The output should look similar to this:: gpg: Signature made Sat 30 Dec 2017 01:07:36 PM CET using RSA key ID 51F78E01 gpg: Good signature from "Thomas Waldmann " gpg: aka "Thomas Waldmann " gpg: aka "Thomas Waldmann " gpg: aka "Thomas Waldmann " gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393 Subkey fingerprint: 2F81 AFFB AB04 E11F E8EE 65D4 243A CFA9 51F7 8E01 If you want to make absolutely sure that you have the right key, you need to verify it via another channel and assign a trust-level to it. borgbackup-1.0.13/docs/usage/0000755000175000017500000000000013431514213015722 5ustar useruser00000000000000borgbackup-1.0.13/docs/usage/break-lock.rst.inc0000644000175000017500000000313513431513133021240 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_break-lock: borg break-lock --------------- :: usage: borg break-lock [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [REPOSITORY] Break the repository lock (e.g. in case it was left by a dead borg. positional arguments: REPOSITORY repository for which to break the locks optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") Description ~~~~~~~~~~~ This command breaks the repository and cache locks. Please use carefully and only while no borg process (on any machine) is trying to access the Cache or the Repository. borgbackup-1.0.13/docs/usage/change-passphrase.rst.inc0000644000175000017500000000353213431513133022623 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_change-passphrase: borg change-passphrase ---------------------- :: usage: borg change-passphrase [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [REPOSITORY] Change repository key file passphrase positional arguments: REPOSITORY optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") Description ~~~~~~~~~~~ The key files used for repository encryption are optionally passphrase protected. This command can be used to change this passphrase. Please note that this command only changes the passphrase, but not any secret protected by it (like e.g. encryption/MAC keys or chunker seed). Thus, changing the passphrase after passphrase and borg key got compromised does not protect future (nor past) backups to the same repository. borgbackup-1.0.13/docs/usage/check.rst.inc0000644000175000017500000000774313431513133020314 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_check: borg check ---------- :: usage: borg check [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [--repository-only] [--archives-only] [--repair] [--save-space] [--last N] [-P PREFIX] [REPOSITORY_OR_ARCHIVE] Check repository consistency positional arguments: REPOSITORY_OR_ARCHIVE repository or archive to check consistency of optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") --repository-only only perform repository checks --archives-only only perform archives checks --repair attempt to repair any inconsistencies found --save-space work slower, but using less space --last N only check last N archives (Default: all) -P PREFIX, --prefix PREFIX only consider archive names starting with this prefix Description ~~~~~~~~~~~ The check command verifies the consistency of a repository and the corresponding archives. First, the underlying repository data files are checked: - For all segments the segment magic (header) is checked - For all objects stored in the segments, all metadata (e.g. crc and size) and all data is read. The read data is checked by size and CRC. Bit rot and other types of accidental damage can be detected this way. - If we are in repair mode and a integrity error is detected for a segment, we try to recover as many objects from the segment as possible. - In repair mode, it makes sure that the index is consistent with the data stored in the segments. - If you use a remote repo server via ssh:, the repo check is executed on the repo server without causing significant network traffic. - The repository check can be skipped using the --archives-only option. Second, the consistency and correctness of the archive metadata is verified: - Is the repo manifest present? If not, it is rebuilt from archive metadata chunks (this requires reading and decrypting of all metadata and data). - Check if archive metadata chunk is present. if not, remove archive from manifest. - For all files (items) in the archive, for all chunks referenced by these files, check if chunk is present. If a chunk is not present and we are in repair mode, replace it with a same-size replacement chunk of zeros. If a previously lost chunk reappears (e.g. via a later backup) and we are in repair mode, the all-zero replacement chunk will be replaced by the correct chunk. This requires reading of archive and file metadata, but not data. - If we are in repair mode and we checked all the archives: delete orphaned chunks from the repo. - if you use a remote repo server via ssh:, the archive check is executed on the client machine (because if encryption is enabled, the checks will require decryption and this is always done client-side, because key access will be required). - The archive checks can be time consuming, they can be skipped using the --repository-only option. borgbackup-1.0.13/docs/usage/create.rst.inc0000644000175000017500000001461613431513133020477 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_create: borg create ----------- :: usage: borg create [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [-s] [-p] [--list] [--filter STATUSCHARS] [-e PATTERN] [--exclude-from EXCLUDEFILE] [--exclude-caches] [--exclude-if-present FILENAME] [--keep-tag-files] [-c SECONDS] [-x] [--numeric-owner] [--noatime] [--noctime] [--timestamp yyyy-mm-ddThh:mm:ss] [--chunker-params CHUNK_MIN_EXP,CHUNK_MAX_EXP,HASH_MASK_BITS,HASH_WINDOW_SIZE] [--ignore-inode] [-C COMPRESSION] [--read-special] [-n] ARCHIVE PATH [PATH ...] Create new archive positional arguments: ARCHIVE name of archive to create (must be also a valid directory name) PATH paths to archive optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") -s, --stats print statistics for the created archive. Requires -v/--verbose. -p, --progress show progress display while creating the archive, showing Original, Compressed and Deduplicated sizes, followed by the Number of files seen and the path being processed, default: False --list output verbose list of items (files, dirs, ...). Requires -v/--verbose. --filter STATUSCHARS only display items with the given status characters -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --exclude-caches exclude directories that contain a CACHEDIR.TAG file (http://www.brynosaurus.com/cachedir/spec.html) --exclude-if-present FILENAME exclude directories that contain the specified file --keep-tag-files keep tag files of excluded caches/directories -c SECONDS, --checkpoint-interval SECONDS write checkpoint every SECONDS seconds (Default: 1800) -x, --one-file-system stay in same file system --numeric-owner only store numeric user and group identifiers --noatime do not store atime into archive --noctime do not store ctime into archive --timestamp yyyy-mm-ddThh:mm:ss manually specify the archive creation date/time (UTC). alternatively, give a reference file/directory. --chunker-params CHUNK_MIN_EXP,CHUNK_MAX_EXP,HASH_MASK_BITS,HASH_WINDOW_SIZE specify the chunker parameters. default: 19,23,21,4095 --ignore-inode ignore inode data in the file metadata cache used to detect unchanged files. -C COMPRESSION, --compression COMPRESSION select compression algorithm (and level): none == no compression (default), lz4 == lz4, zlib == zlib (default level 6), zlib,0 .. zlib,9 == zlib (with level 0..9), lzma == lzma (default level 6), lzma,0 .. lzma,9 == lzma (with level 0..9). --read-special open and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files. -n, --dry-run do not create a backup archive Description ~~~~~~~~~~~ This command creates a backup archive containing all files found while recursively traversing all paths specified. Paths are added to the archive as they are given, that means if relative paths are desired, the command has to be run from the correct directory. When giving '-' as path, borg will read data from standard input and create a file 'stdin' in the created archive from that data. The archive will consume almost no disk space for files or parts of files that have already been stored in other archives. The archive name needs to be unique. It must not end in '.checkpoint' or '.checkpoint.N' (with N being a number), because these names are used for checkpoints and treated in special ways. In the archive name, you may use the following placeholders: {now}, {utcnow}, {fqdn}, {hostname}, {user} and some others. To speed up pulling backups over sshfs and similar network file systems which do not provide correct inode information the --ignore-inode flag can be used. This potentially decreases reliability of change detection, while avoiding always reading all files on these file systems. The mount points of filesystems or filesystem snapshots should be the same for every creation of a new archive to ensure fast operation. This is because the file cache that is used to determine changed files quickly uses absolute filenames. If this is not possible, consider creating a bind mount to a stable location. See the output of the "borg help patterns" command for more help on exclude patterns. See the output of the "borg help placeholders" command for more help on placeholders. Borg respects the nodump flag. Files flagged nodump will be marked as excluded (x) in ``--list`` output. borgbackup-1.0.13/docs/usage/delete.rst.inc0000644000175000017500000000414713431513133020474 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_delete: borg delete ----------- :: usage: borg delete [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [-p] [-s] [-c] [--force] [--save-space] [TARGET] Delete an existing repository or archive positional arguments: TARGET archive or repository to delete optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") -p, --progress show progress display while deleting a single archive -s, --stats print statistics for the deleted archive. Requires -v/--verbose. -c, --cache-only delete only the local cache for the given repository --force force deletion of corrupted archives, use --force --force in case --force does not work. --save-space work slower, but using less space Description ~~~~~~~~~~~ This command deletes an archive from the repository or the complete repository. Disk space is reclaimed accordingly. If you delete the complete repository, the local cache for it (if any) is also deleted. borgbackup-1.0.13/docs/usage/extract.rst.inc0000644000175000017500000000545313431513133020705 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_extract: borg extract ------------ :: usage: borg extract [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [--list] [-n] [-e PATTERN] [--exclude-from EXCLUDEFILE] [--numeric-owner] [--strip-components NUMBER] [--stdout] [--sparse] ARCHIVE [PATH [PATH ...]] Extract archive contents positional arguments: ARCHIVE archive to extract PATH paths to extract; patterns are supported optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") --list output verbose list of items (files, dirs, ...). Requires -v/--verbose. -n, --dry-run do not actually change any files -e PATTERN, --exclude PATTERN exclude paths matching PATTERN --exclude-from EXCLUDEFILE read exclude patterns from EXCLUDEFILE, one per line --numeric-owner only obey numeric user and group identifiers --strip-components NUMBER Remove the specified number of leading path elements. Pathnames with fewer elements will be silently skipped. --stdout write all extracted data to stdout --sparse create holes in output sparse file from all-zero chunks Description ~~~~~~~~~~~ This command extracts the contents of an archive. By default the entire archive is extracted but a subset of files and directories can be selected by passing a list of ``PATHs`` as arguments. The file selection can further be restricted by using the ``--exclude`` option. See the output of the "borg help patterns" command for more help on exclude patterns. borgbackup-1.0.13/docs/usage/help.rst.inc0000644000175000017500000001270113431513133020155 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_patterns: borg help patterns ~~~~~~~~~~~~~~~~~~ Exclusion patterns support four separate styles, fnmatch, shell, regular expressions and path prefixes. By default, fnmatch is used. If followed by a colon (':') the first two characters of a pattern are used as a style selector. Explicit style selection is necessary when a non-default style is desired or when the desired pattern starts with two alphanumeric characters followed by a colon (i.e. `aa:something/*`). `Fnmatch `_, selector `fm:` This is the default style. These patterns use a variant of shell pattern syntax, with '*' matching any number of characters, '?' matching any single character, '[...]' matching any single character specified, including ranges, and '[!...]' matching any character not specified. For the purpose of these patterns, the path separator ('\' for Windows and '/' on other systems) is not treated specially. Wrap meta-characters in brackets for a literal match (i.e. `[?]` to match the literal character `?`). For a path to match a pattern, it must completely match from start to end, or must match from the start to just before a path separator. Except for the root path, paths will never end in the path separator when matching is attempted. Thus, if a given pattern ends in a path separator, a '*' is appended before matching is attempted. Shell-style patterns, selector `sh:` Like fnmatch patterns these are similar to shell patterns. The difference is that the pattern may include `**/` for matching zero or more directory levels, `*` for matching zero or more arbitrary characters with the exception of any path separator. Regular expressions, selector `re:` Regular expressions similar to those found in Perl are supported. Unlike shell patterns regular expressions are not required to match the complete path and any substring match is sufficient. It is strongly recommended to anchor patterns to the start ('^'), to the end ('$') or both. Path separators ('\' for Windows and '/' on other systems) in paths are always normalized to a forward slash ('/') before applying a pattern. The regular expression syntax is described in the `Python documentation for the re module `_. Prefix path, selector `pp:` This pattern style is useful to match whole sub-directories. The pattern `pp:/data/bar` matches `/data/bar` and everything therein. .. note:: `re:`, `sh:` and `fm:` patterns are all implemented on top of the Python SRE engine. It is very easy to formulate patterns for each of these types which requires an inordinate amount of time to match paths. If untrusted users are able to supply patterns, ensure they cannot supply `re:` patterns. Further, ensure that `sh:` and `fm:` patterns only contain a handful of wildcards at most. Exclusions can be passed via the command line option `--exclude`. When used from within a shell the patterns should be quoted to protect them from expansion. The `--exclude-from` option permits loading exclusion patterns from a text file with one pattern per line. Lines empty or starting with the number sign ('#') after removing whitespace on both ends are ignored. The optional style selector prefix is also supported for patterns loaded from a file. Due to whitespace removal paths with whitespace at the beginning or end can only be excluded using regular expressions. Examples:: # Exclude '/home/user/file.o' but not '/home/user/file.odt': $ borg create -e '*.o' backup / # Exclude '/home/user/junk' and '/home/user/subdir/junk' but # not '/home/user/importantjunk' or '/etc/junk': $ borg create -e '/home/*/junk' backup / # Exclude the contents of '/home/user/cache' but not the directory itself: $ borg create -e /home/user/cache/ backup / # The file '/home/user/cache/important' is *not* backed up: $ borg create -e /home/user/cache/ backup / /home/user/cache/important # The contents of directories in '/home' are not backed up when their name # ends in '.tmp' $ borg create --exclude 're:^/home/[^/]+\.tmp/' backup / # Load exclusions from file $ cat >exclude.txt < repokey positional arguments: REPOSITORY optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") Description ~~~~~~~~~~~ This command migrates a repository from passphrase mode (not supported any more) to repokey mode. You will be first asked for the repository passphrase (to open it in passphrase mode). This is the same passphrase as you used to use for this repo before 1.0. It will then derive the different secrets from this passphrase. Then you will be asked for a new passphrase (twice, for safety). This passphrase will be used to protect the repokey (which contains these same secrets in encrypted form). You may use the same passphrase as you used to use, but you may also use a different one. After migrating to repokey mode, you can change the passphrase at any time. But please note: the secrets will always stay the same and they could always be derived from your (old) passphrase-mode passphrase. borgbackup-1.0.13/docs/usage/mount.rst.inc0000644000175000017500000000560213431513133020371 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_mount: borg mount ---------- :: usage: borg mount [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [-f] [-o OPTIONS] REPOSITORY_OR_ARCHIVE MOUNTPOINT Mount archive or an entire repository as a FUSE filesystem positional arguments: REPOSITORY_OR_ARCHIVE repository/archive to mount MOUNTPOINT where to mount filesystem optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") -f, --foreground stay in foreground, do not daemonize -o OPTIONS Extra mount options Description ~~~~~~~~~~~ This command mounts an archive as a FUSE filesystem. This can be useful for browsing an archive or restoring individual files. Unless the ``--foreground`` option is given the command will run in the background until the filesystem is ``umounted``. The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users to tweak the performance. It sets the number of cached data chunks; additional memory usage can be up to ~8 MiB times this number. The default is the number of CPU cores. For FUSE configuration and mount options, see the mount.fuse(8) manual page. Additional mount options supported by borg: - allow_damaged_files: by default damaged files (where missing chunks were replaced with runs of zeros by borg check --repair) are not readable and return EIO (I/O error). Set this option to read such files. - ignore_permissions: for security reasons the "default_permissions" mount option is internally enforced by borg. "ignore_permissions" can be given to not enforce "default_permissions". When the daemonized process receives a signal or crashes, it does not unmount. Unmounting in these cases could cause an active rsync or similar process to unintentionally delete data. When running in the foreground ^C/SIGINT unmounts cleanly, but other signals or crashes do not. borgbackup-1.0.13/docs/usage/prune.rst.inc0000644000175000017500000001013713431513133020357 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_prune: borg prune ---------- :: usage: borg prune [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [-n] [--force] [-s] [--list] [--keep-within INTERVAL] [-H HOURLY] [-d DAILY] [-w WEEKLY] [-m MONTHLY] [-y YEARLY] [-P PREFIX] [--save-space] [REPOSITORY] Prune repository archives according to specified rules positional arguments: REPOSITORY repository to prune optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") -n, --dry-run do not change repository --force force pruning of corrupted archives -s, --stats print statistics for the deleted archive. Requires -v/--verbose. --list output verbose list of archives it keeps/prunes. Requires -v/--verbose. --keep-within INTERVAL keep all archives within this time interval -H HOURLY, --keep-hourly HOURLY number of hourly archives to keep -d DAILY, --keep-daily DAILY number of daily archives to keep -w WEEKLY, --keep-weekly WEEKLY number of weekly archives to keep -m MONTHLY, --keep-monthly MONTHLY number of monthly archives to keep -y YEARLY, --keep-yearly YEARLY number of yearly archives to keep -P PREFIX, --prefix PREFIX only consider archive names starting with this prefix --save-space work slower, but using less space Description ~~~~~~~~~~~ The prune command prunes a repository by deleting all archives not matching any of the specified retention options. This command is normally used by automated backup scripts wanting to keep a certain number of historic backups. As an example, "-d 7" means to keep the latest backup on each day, up to 7 most recent days with backups (days without backups do not count). The rules are applied from hourly to yearly, and backups selected by previous rules do not count towards those of later rules. The time that each backup starts is used for pruning purposes. Dates and times are interpreted in the local timezone, and weeks go from Monday to Sunday. Specifying a negative number of archives to keep means that there is no limit. The "--keep-within" option takes an argument of the form "", where char is "H", "d", "w", "m", "y". For example, "--keep-within 2d" means to keep all archives that were created within the past 48 hours. "1m" is taken to mean "31d". The archives kept with this option do not count towards the totals specified by any other options. If a prefix is set with -P, then only archives that start with the prefix are considered for deletion and only those archives count towards the totals specified by the rules. Otherwise, *all* archives in the repository are candidates for deletion! There is no automatic distinction between archives representing different contents. These need to be distinguished by specifying matching prefixes. borgbackup-1.0.13/docs/usage/rename.rst.inc0000644000175000017500000000270613431513133020500 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_rename: borg rename ----------- :: usage: borg rename [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] ARCHIVE NEWNAME Rename an existing archive positional arguments: ARCHIVE archive to rename NEWNAME the new archive name to use optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") Description ~~~~~~~~~~~ This command renames an archive in the repository. borgbackup-1.0.13/docs/usage/serve.rst.inc0000644000175000017500000000361013431513133020350 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_serve: borg serve ---------- :: usage: borg serve [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [--restrict-to-path PATH] [--append-only] Start in server mode. This command is usually not used manually. optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") --restrict-to-path PATH restrict repository access to PATH. Can be specified multiple times to allow the client access to several directories. Access to all sub-directories is granted implicitly; PATH doesn't need to directly point to a repository. --append-only only allow appending to repository segment files Description ~~~~~~~~~~~ This command starts a repository server process. This command is usually not used manually. borgbackup-1.0.13/docs/usage/umount.rst.inc0000644000175000017500000000310113431513133020546 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_umount: borg umount ----------- :: usage: borg umount [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] MOUNTPOINT un-mount the FUSE filesystem positional arguments: MOUNTPOINT mountpoint of the filesystem to umount optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") Description ~~~~~~~~~~~ This command un-mounts a FUSE filesystem that was mounted with ``borg mount``. This is a convenience wrapper that just calls the platform-specific shell command - usually this is either umount or fusermount -u. borgbackup-1.0.13/docs/usage/upgrade.rst.inc0000644000175000017500000001223013431513133020651 0ustar useruser00000000000000.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit! .. _borg_upgrade: borg upgrade ------------ :: usage: borg upgrade [-h] [--critical] [--error] [--warning] [--info] [--debug] [--lock-wait N] [--show-rc] [--no-files-cache] [--umask M] [--remote-path PATH] [-p] [-n] [-i] [--force] [--tam] [--disable-tam] [REPOSITORY] upgrade a repository from a previous version positional arguments: REPOSITORY path to the repository to be upgraded optional arguments: -h, --help show this help message and exit --critical work on log level CRITICAL --error work on log level ERROR --warning work on log level WARNING (default) --info, -v, --verbose work on log level INFO --debug work on log level DEBUG --lock-wait N wait for the lock, but max. N seconds (default: 1). --show-rc show/log the return code (rc) --no-files-cache do not load/update the file metadata cache used to detect unchanged files --umask M set umask to M (local and remote, default: 0077) --remote-path PATH use PATH as borg executable on the remote (default: "borg") -p, --progress show progress display while upgrading the repository -n, --dry-run do not change repository -i, --inplace rewrite repository in place, with no chance of going back to older versions of the repository. --force Force upgrade --tam Enable manifest authentication (in key and cache) (Borg 1.0.9 and later) --disable-tam Disable manifest authentication (in key and cache) Description ~~~~~~~~~~~ Upgrade an existing, local Borg repository. When you do not need borg upgrade +++++++++++++++++++++++++++++++++ Not every change requires that you run ``borg upgrade``. You do **not** need to run it when: - moving your repository to a different place - upgrading to another point release (like 1.0.x to 1.0.y), except when noted otherwise in the changelog - upgrading from 1.0.x to 1.1.x, except when noted otherwise in the changelog Borg 1.x.y upgrades +++++++++++++++++++ Use ``borg upgrade --tam REPO`` to require manifest authentication introduced with Borg 1.0.9 to address security issues. This means that modifying the repository after doing this with a version prior to 1.0.9 will raise a validation error, so only perform this upgrade after updating all clients using the repository to 1.0.9 or newer. This upgrade should be done on each client for safety reasons. If a repository is accidentally modified with a pre-1.0.9 client after this upgrade, use ``borg upgrade --tam --force REPO`` to remedy it. If you routinely do this you might not want to enable this upgrade (which will leave you exposed to the security issue). You can reverse the upgrade by issuing ``borg upgrade --disable-tam REPO``. See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability for details. Attic and Borg 0.xx to Borg 1.x +++++++++++++++++++++++++++++++ This currently supports converting an Attic repository to Borg and also helps with converting Borg 0.xx to 1.0. Currently, only LOCAL repositories can be upgraded (issue #465). Please note that ``borg create`` (since 1.0.0) uses bigger chunks by default than old borg or attic did, so the new chunks won't deduplicate with the old chunks in the upgraded repository. See ``--chunker-params`` option of ``borg create`` and ``borg recreate``. ``borg upgrade`` will change the magic strings in the repository's segments to match the new Borg magic strings. The keyfiles found in $ATTIC_KEYS_DIR or ~/.attic/keys/ will also be converted and copied to $BORG_KEYS_DIR or ~/.config/borg/keys. The cache files are converted, from $ATTIC_CACHE_DIR or ~/.cache/attic to $BORG_CACHE_DIR or ~/.cache/borg, but the cache layout between Borg and Attic changed, so it is possible the first backup after the conversion takes longer than expected due to the cache resync. Upgrade should be able to resume if interrupted, although it will still iterate over all segments. If you want to start from scratch, use `borg delete` over the copied repository to make sure the cache files are also removed: borg delete borg Unless ``--inplace`` is specified, the upgrade process first creates a backup copy of the repository, in REPOSITORY.upgrade-DATETIME, using hardlinks. This takes longer than in place upgrades, but is much safer and gives progress information (as opposed to ``cp -al``). Once you are satisfied with the conversion, you can safely destroy the backup copy. WARNING: Running the upgrade in place will make the current copy unusable with older version, with no way of going back to previous versions. This can PERMANENTLY DAMAGE YOUR REPOSITORY! Attic CAN NOT READ BORG REPOSITORIES, as the magic strings have changed. You have been warned.borgbackup-1.0.13/docs/usage.rst0000644000175000017500000012172713431513133016466 0ustar useruser00000000000000.. include:: global.rst.inc .. highlight:: none .. _detailed_usage: Usage ===== |project_name| consists of a number of commands. Each command accepts a number of arguments and options. The following sections will describe each command in detail. General ------- Positional Arguments and Options: Order matters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Borg only supports taking options (``-v`` and ``--list`` in the example) to the left or right of all positional arguments (``repo::archive`` and ``path`` in the example), but not in between them: :: borg extract -v --list repo::archive path # good and preferred borg extract repo::archive path -v --list # also works borg extract -v repo::archive path --list # works, but ugly borg extract repo::archive -v --list path # BAD This is due to a problem in the argparse module: http://bugs.python.org/issue15112 Repository URLs ~~~~~~~~~~~~~~~ **Local filesystem** (or locally mounted network filesystem): ``/path/to/repo`` - filesystem path to repo directory, absolute path ``path/to/repo`` - filesystem path to repo directory, relative path Also, stuff like ``~/path/to/repo`` or ``~other/path/to/repo`` works (this is expanded by your shell). Note: you may also prepend a ``file://`` to a filesystem path to get URL style. **Remote repositories** accessed via ssh user@host: ``user@host:/path/to/repo`` - remote repo, absolute path ``ssh://user@host:port/path/to/repo`` - same, alternative syntax, port can be given **Remote repositories with relative pathes** can be given using this syntax: ``user@host:path/to/repo`` - path relative to current directory ``user@host:~/path/to/repo`` - path relative to user's home directory ``user@host:~other/path/to/repo`` - path relative to other's home directory Note: giving ``user@host:/./path/to/repo`` or ``user@host:/~/path/to/repo`` or ``user@host:/~other/path/to/repo`` is also supported, but not required here. **Remote repositories with relative pathes, alternative syntax with port**: ``ssh://user@host:port/./path/to/repo`` - path relative to current directory ``ssh://user@host:port/~/path/to/repo`` - path relative to user's home directory ``ssh://user@host:port/~other/path/to/repo`` - path relative to other's home directory If you frequently need the same repo URL, it is a good idea to set the ``BORG_REPO`` environment variable to set a default for the repo URL: :: export BORG_REPO='ssh://user@host:port/path/to/repo' Then just leave away the repo URL if only a repo URL is needed and you want to use the default - it will be read from BORG_REPO then. Use ``::`` syntax to give the repo URL when syntax requires giving a positional argument for the repo (e.g. ``borg mount :: /mnt``). Repository / Archive Locations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Many commands want either a repository (just give the repo URL, see above) or an archive location, which is a repo URL followed by ``::archive_name``. Archive names must not contain the ``/`` (slash) character. For simplicity, maybe also avoid blanks or other characters that have special meaning on the shell or in a filesystem (borg mount will use the archive name as directory name). If you have set BORG_REPO (see above) and an archive location is needed, use ``::archive_name`` - the repo URL part is then read from BORG_REPO. Type of log output ~~~~~~~~~~~~~~~~~~ The log level of the builtin logging configuration defaults to WARNING. This is because we want |project_name| to be mostly silent and only output warnings, errors and critical messages. Log levels: DEBUG < INFO < WARNING < ERROR < CRITICAL Use ``--debug`` to set DEBUG log level - to get debug, info, warning, error and critical level output. Use ``--info`` (or ``-v`` or ``--verbose``) to set INFO log level - to get info, warning, error and critical level output. Use ``--warning`` (default) to set WARNING log level - to get warning, error and critical level output. Use ``--error`` to set ERROR log level - to get error and critical level output. Use ``--critical`` to set CRITICAL log level - to get critical level output. While you can set misc. log levels, do not expect that every command will give different output on different log levels - it's just a possibility. .. warning:: Options --critical and --error are provided for completeness, their usage is not recommended as you might miss important information. .. warning:: While some options (like ``--stats`` or ``--list``) will emit more informational messages, you have to use INFO (or lower) log level to make them show up in log output. Use ``-v`` or a logging configuration. Return codes ~~~~~~~~~~~~ |project_name| can exit with the following return codes (rc): :: 0 = success (logged as INFO) 1 = warning (operation reached its normal end, but there were warnings - you should check the log, logged as WARNING) 2 = error (like a fatal error, a local or remote exception, the operation did not reach its normal end, logged as ERROR) 128+N = killed by signal N (e.g. 137 == kill -9) If you use ``--show-rc``, the return code is also logged at the indicated level as the last log entry. Environment Variables ~~~~~~~~~~~~~~~~~~~~~ |project_name| uses some environment variables for automation: General: BORG_REPO When set, use the value to give the default repository location. If a command needs an archive parameter, you can abbreviate as `::archive`. If a command needs a repository parameter, you can either leave it away or abbreviate as `::`, if a positional parameter is required. BORG_PASSPHRASE When set, use the value to answer the passphrase question for encrypted repositories. It is used when a passphrase is needed to access an encrypted repo as well as when a new passphrase should be initially set when initializing an encrypted repo. BORG_PASSCOMMAND When set, use the standard output of the command (trailing newlines are stripped) to answer the passphrase question for encrypted repositories. It is used when a passphrase is needed to access an encrypted repo as well as when a new passphrase should be initially set when initializing an encrypted repo. If BORG_PASSPHRASE is also set, it takes precedence. BORG_PASSPHRASE_FD When set, specifies a file descriptor to read a passphrase from. Programs starting borg may choose to open an anonymous pipe and use it to pass a passphrase. This is safer than passing via BORG_PASSPHRASE, because on some systems (e.g. Linux) environment can be examined by other processes. If BORG_PASSPHRASE or BORG_PASSCOMMAND are also set, they take precedence. BORG_DISPLAY_PASSPHRASE When set, use the value to answer the "display the passphrase for verification" question when defining a new passphrase for encrypted repositories. BORG_LOGGING_CONF When set, use the given filename as INI_-style logging configuration. BORG_RSH When set, use this command instead of ``ssh``. This can be used to specify ssh options, such as a custom identity file ``ssh -i /path/to/private/key``. See ``man ssh`` for other options. BORG_REMOTE_PATH When set, use the given path as borg executable on the remote (defaults to "borg" if unset). Using ``--remote-path PATH`` commandline option overrides the environment variable. BORG_FILES_CACHE_TTL When set to a numeric value, this determines the maximum "time to live" for the files cache entries (default: 20). The files cache is used to quickly determine whether a file is unchanged. The FAQ explains this more detailed in: :ref:`always_chunking` TMPDIR where temporary files are stored (might need a lot of temporary space for some operations) Some automatic "answerers" (if set, they automatically answer confirmation questions): BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no (or =yes) For "Warning: Attempting to access a previously unknown unencrypted repository" BORG_RELOCATED_REPO_ACCESS_IS_OK=no (or =yes) For "Warning: The repository at location ... was previously located at ..." BORG_CHECK_I_KNOW_WHAT_I_AM_DOING=NO (or =YES) For "Warning: 'check --repair' is an experimental feature that might result in data loss." BORG_DELETE_I_KNOW_WHAT_I_AM_DOING=NO (or =YES) For "You requested to completely DELETE the repository *including* all archives it contains:" Note: answers are case sensitive. setting an invalid answer value might either give the default answer or ask you interactively, depending on whether retries are allowed (they by default are allowed). So please test your scripts interactively before making them a non-interactive script. Directories: BORG_KEYS_DIR Default to '~/.config/borg/keys'. This directory contains keys for encrypted repositories. BORG_SECURITY_DIR Default to '~/.config/borg/security'. This directory is used by Borg to track various pieces of security-related data. BORG_CACHE_DIR Default to '~/.cache/borg'. This directory contains the local cache and might need a lot of space for dealing with big repositories). Building: BORG_OPENSSL_PREFIX Adds given OpenSSL header file directory to the default locations (setup.py). BORG_LZ4_PREFIX Adds given LZ4 header file directory to the default locations (setup.py). Please note: - be very careful when using the "yes" sayers, the warnings with prompt exist for your / your data's security/safety - also be very careful when putting your passphrase into a script, make sure it has appropriate file permissions (e.g. mode 600, root:root). .. _INI: https://docs.python.org/3.4/library/logging.config.html#configuration-file-format Resource Usage ~~~~~~~~~~~~~~ |project_name| might use a lot of resources depending on the size of the data set it is dealing with. If one uses |project_name| in a client/server way (with a ssh: repository), the resource usage occurs in part on the client and in another part on the server. If one uses |project_name| as a single process (with a filesystem repo), all the resource usage occurs in that one process, so just add up client + server to get the approximate resource usage. CPU client: borg create: does chunking, hashing, compression, crypto (high CPU usage) chunks cache sync: quite heavy on CPU, doing lots of hashtable operations. borg extract: crypto, decompression (medium to high CPU usage) borg check: similar to extract, but depends on options given. borg prune / borg delete archive: low to medium CPU usage borg delete repo: done on the server It won't go beyond 100% of 1 core as the code is currently single-threaded. Especially higher zlib and lzma compression levels use significant amounts of CPU cycles. Crypto might be cheap on the CPU (if hardware accelerated) or expensive (if not). CPU server: It usually doesn't need much CPU, it just deals with the key/value store (repository) and uses the repository index for that. borg check: the repository check computes the checksums of all chunks (medium CPU usage) borg delete repo: low CPU usage CPU (only for client/server operation): When using borg in a client/server way with a ssh:-type repo, the ssh processes used for the transport layer will need some CPU on the client and on the server due to the crypto they are doing - esp. if you are pumping big amounts of data. Memory (RAM) client: The chunks index and the files index are read into memory for performance reasons. Might need big amounts of memory (see below). Compression, esp. lzma compression with high levels might need substantial amounts of memory. Memory (RAM) server: The server process will load the repository index into memory. Might need considerable amounts of memory, but less than on the client (see below). Chunks index (client only): Proportional to the amount of data chunks in your repo. Lots of chunks in your repo imply a big chunks index. It is possible to tweak the chunker params (see create options). Files index (client only): Proportional to the amount of files in your last backups. Can be switched off (see create options), but next backup might be much slower if you do. The speed benefit of using the files cache is proportional to file size. Repository index (server only): Proportional to the amount of data chunks in your repo. Lots of chunks in your repo imply a big repository index. It is possible to tweak the chunker params (see create options) to influence the amount of chunks being created. Temporary files (client): Reading data and metadata from a FUSE mounted repository will consume up to the size of all deduplicated, small chunks in the repository. Big chunks won't be locally cached. Temporary files (server): None. Cache files (client only): Contains the chunks index and files index (plus a collection of single- archive chunk indexes which might need huge amounts of disk space, depending on archive count and size - see FAQ about how to reduce). Network (only for client/server operation): If your repository is remote, all deduplicated (and optionally compressed/ encrypted) data of course has to go over the connection (ssh: repo url). If you use a locally mounted network filesystem, additionally some copy operations used for transaction support also go over the connection. If you backup multiple sources to one target repository, additional traffic happens for cache resynchronization. In case you are interested in more details (like formulas), please see :ref:`internals`. Units ~~~~~ To display quantities, |project_name| takes care of respecting the usual conventions of scale. Disk sizes are displayed in `decimal `_, using powers of ten (so ``kB`` means 1000 bytes). For memory usage, `binary prefixes `_ are used, and are indicated using the `IEC binary prefixes `_, using powers of two (so ``KiB`` means 1024 bytes). Date and Time ~~~~~~~~~~~~~ We format date and time conforming to ISO-8601, that is: YYYY-MM-DD and HH:MM:SS (24h clock). For more information about that, see: https://xkcd.com/1179/ Unless otherwise noted, we display local date and time. Internally, we store and process date and time as UTC. .. include:: usage/init.rst.inc Examples ~~~~~~~~ :: # Local repository (default is to use encryption in repokey mode) $ borg init /path/to/repo # Local repository (no encryption) $ borg init --encryption=none /path/to/repo # Remote repository (accesses a remote borg via ssh) $ borg init user@hostname:backup # Remote repository (store the key your home dir) $ borg init --encryption=keyfile user@hostname:backup Important notes about encryption: It is not recommended to disable encryption. Repository encryption protects you e.g. against the case that an attacker has access to your backup repository. But be careful with the key / the passphrase: If you want "passphrase-only" security, use the ``repokey`` mode. The key will be stored inside the repository (in its "config" file). In above mentioned attack scenario, the attacker will have the key (but not the passphrase). If you want "passphrase and having-the-key" security, use the ``keyfile`` mode. The key will be stored in your home directory (in ``.config/borg/keys``). In the attack scenario, the attacker who has just access to your repo won't have the key (and also not the passphrase). Make a backup copy of the key file (``keyfile`` mode) or repo config file (``repokey`` mode) and keep it at a safe place, so you still have the key in case it gets corrupted or lost. Also keep the passphrase at a safe place. The backup that is encrypted with that key won't help you with that, of course. Make sure you use a good passphrase. Not too short, not too simple. The real encryption / decryption key is encrypted with / locked by your passphrase. If an attacker gets your key, he can't unlock and use it without knowing the passphrase. Be careful with special or non-ascii characters in your passphrase: - |project_name| processes the passphrase as unicode (and encodes it as utf-8), so it does not have problems dealing with even the strangest characters. - BUT: that does not necessarily apply to your OS / VM / keyboard configuration. So better use a long passphrase made from simple ascii chars than one that includes non-ascii stuff or characters that are hard/impossible to enter on a different keyboard layout. You can change your passphrase for existing repos at any time, it won't affect the encryption/decryption key or other secrets. .. include:: usage/create.rst.inc Examples ~~~~~~~~ :: # Backup ~/Documents into an archive named "my-documents" $ borg create /path/to/repo::my-documents ~/Documents # same, but verbosely list all files as we process them $ borg create -v --list /path/to/repo::my-documents ~/Documents # Backup ~/Documents and ~/src but exclude pyc files $ borg create /path/to/repo::my-files \ ~/Documents \ ~/src \ --exclude '*.pyc' # Backup home directories excluding image thumbnails (i.e. only # /home/*/.thumbnails is excluded, not /home/*/*/.thumbnails) $ borg create /path/to/repo::my-files /home \ --exclude 're:^/home/[^/]+/\.thumbnails/' # Do the same using a shell-style pattern $ borg create /path/to/repo::my-files /home \ --exclude 'sh:/home/*/.thumbnails' # Backup the root filesystem into an archive named "root-YYYY-MM-DD" # use zlib compression (good, but slow) - default is no compression $ borg create -C zlib,6 --one-file-system /path/to/repo::root-{now:%Y-%m-%d} / # Make a big effort in fine granular deduplication (big chunk management # overhead, needs a lot of RAM and disk space, see formula in internals # docs - same parameters as borg < 1.0 or attic): $ borg create --chunker-params 10,23,16,4095 /path/to/repo::small /smallstuff # Backup a raw device (must not be active/in use/mounted at that time) $ dd if=/dev/sdx bs=10M | borg create /path/to/repo::my-sdx - # No compression (default) $ borg create /path/to/repo::arch ~ # Super fast, low compression $ borg create --compression lz4 /path/to/repo::arch ~ # Less fast, higher compression (N = 0..9) $ borg create --compression zlib,N /path/to/repo::arch ~ # Even slower, even higher compression (N = 0..9) $ borg create --compression lzma,N /path/to/repo::arch ~ # Use short hostname, user name and current time in archive name $ borg create /path/to/repo::{hostname}-{user}-{now} ~ # Similar, use the same datetime format as borg 1.1 will have as default $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S} ~ # As above, but add nanoseconds $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f} ~ # Backing up relative paths by moving into the correct directory first $ cd /home/user/Documents # The root directory of the archive will be "projectA" $ borg create /path/to/repo::daily-projectA-{now:%Y-%m-%d} projectA Notes ~~~~~ - the --exclude patterns are not like tar. In tar --exclude .bundler/gems will exclude foo/.bundler/gems. In borg it will not, you need to use --exclude '\*/.bundler/gems' to get the same effect. See ``borg help patterns`` for more information. .. include:: usage/extract.rst.inc Examples ~~~~~~~~ :: # Extract entire archive $ borg extract /path/to/repo::my-files # Extract entire archive and list files while processing $ borg extract -v --list /path/to/repo::my-files # Extract the "src" directory $ borg extract /path/to/repo::my-files home/USERNAME/src # Extract the "src" directory but exclude object files $ borg extract /path/to/repo::my-files home/USERNAME/src --exclude '*.o' # Restore a raw device (must not be active/in use/mounted at that time) $ borg extract --stdout /path/to/repo::my-sdx | dd of=/dev/sdx bs=10M Note: currently, extract always writes into the current working directory ("."), so make sure you ``cd`` to the right place before calling ``borg extract``. .. include:: usage/check.rst.inc .. include:: usage/rename.rst.inc Examples ~~~~~~~~ :: $ borg create /path/to/repo::archivename ~ $ borg list /path/to/repo archivename Mon, 2016-02-15 19:50:19 $ borg rename /path/to/repo::archivename newname $ borg list /path/to/repo newname Mon, 2016-02-15 19:50:19 .. include:: usage/list.rst.inc Examples ~~~~~~~~ :: $ borg list /path/to/repo Monday Mon, 2016-02-15 19:15:11 repo Mon, 2016-02-15 19:26:54 root-2016-02-15 Mon, 2016-02-15 19:36:29 newname Mon, 2016-02-15 19:50:19 ... $ borg list /path/to/repo::root-2016-02-15 drwxr-xr-x root root 0 Mon, 2016-02-15 17:44:27 . drwxrwxr-x root root 0 Mon, 2016-02-15 19:04:49 bin -rwxr-xr-x root root 1029624 Thu, 2014-11-13 00:08:51 bin/bash lrwxrwxrwx root root 0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff -rwxr-xr-x root root 2140 Fri, 2015-03-27 20:24:22 bin/bzdiff ... $ borg list /path/to/repo::archiveA --list-format="{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NEWLINE}" drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 . drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code/myproject -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext ... # see what is changed between archives, based on file modification time, size and file path $ borg list /path/to/repo::archiveA --list-format="{mtime:%s}{TAB}{size}{TAB}{path}{LF}" |sort -n > /tmp/list.archiveA $ borg list /path/to/repo::archiveB --list-format="{mtime:%s}{TAB}{size}{TAB}{path}{LF}" |sort -n > /tmp/list.archiveB $ diff -y /tmp/list.archiveA /tmp/list.archiveB 1422781200 0 . 1422781200 0 . 1422781200 0 code 1422781200 0 code 1422781200 0 code/myproject 1422781200 0 code/myproject 1422781200 1416192 code/myproject/file.ext | 1454664653 1416192 code/myproject/file.ext ... .. include:: usage/delete.rst.inc Examples ~~~~~~~~ :: # delete a single backup archive: $ borg delete /path/to/repo::Monday # delete the whole repository and the related local cache: $ borg delete /path/to/repo You requested to completely DELETE the repository *including* all archives it contains: repo Mon, 2016-02-15 19:26:54 root-2016-02-15 Mon, 2016-02-15 19:36:29 newname Mon, 2016-02-15 19:50:19 Type 'YES' if you understand this and want to continue: YES .. include:: usage/prune.rst.inc Examples ~~~~~~~~ Be careful, prune is a potentially dangerous command, it will remove backup archives. The default of prune is to apply to **all archives in the repository** unless you restrict its operation to a subset of the archives using ``--prefix``. When using ``--prefix``, be careful to choose a good prefix - e.g. do not use a prefix "foo" if you do not also want to match "foobar". It is strongly recommended to always run ``prune -v --list --dry-run ...`` first so you will see what it would do without it actually doing anything. There is also a visualized prune example in ``docs/misc/prune-example.txt``. :: # Keep 7 end of day and 4 additional end of week archives. # Do a dry-run without actually deleting anything. $ borg prune -v --list --dry-run --keep-daily=7 --keep-weekly=4 /path/to/repo # Same as above but only apply to archive names starting with the hostname # of the machine followed by a "-" character: $ borg prune -v --list --keep-daily=7 --keep-weekly=4 --prefix='{hostname}-' /path/to/repo # Keep 7 end of day, 4 additional end of week archives, # and an end of month archive for every month: $ borg prune -v --list --keep-daily=7 --keep-weekly=4 --keep-monthly=-1 /path/to/repo # Keep all backups in the last 10 days, 4 additional end of week archives, # and an end of month archive for every month: $ borg prune -v --list --keep-within=10d --keep-weekly=4 --keep-monthly=-1 /path/to/repo .. include:: usage/info.rst.inc Examples ~~~~~~~~ :: $ borg info /path/to/repo::root-2016-02-15 Name: root-2016-02-15 Fingerprint: 57c827621f21b000a8d363c1e163cc55983822b3afff3a96df595077a660be50 Hostname: myhostname Username: root Time (start): Mon, 2016-02-15 19:36:29 Time (end): Mon, 2016-02-15 19:39:26 Command line: /usr/local/bin/borg create -v --list -C zlib,6 /path/to/repo::root-2016-02-15 / --one-file-system Number of files: 38100 Original size Compressed size Deduplicated size This archive: 1.33 GB 613.25 MB 571.64 MB All archives: 1.63 GB 853.66 MB 584.12 MB Unique chunks Total chunks Chunk index: 36858 48844 .. include:: usage/mount.rst.inc .. include:: usage/umount.rst.inc Examples ~~~~~~~~ :: $ borg mount /path/to/repo::root-2016-02-15 /tmp/mymountpoint $ ls /tmp/mymountpoint bin boot etc home lib lib64 lost+found media mnt opt root sbin srv tmp usr var $ borg umount /tmp/mymountpoint .. include:: usage/key_export.rst.inc .. include:: usage/key_import.rst.inc .. include:: usage/change-passphrase.rst.inc Examples ~~~~~~~~ :: # Create a key file protected repository $ borg init --encryption=keyfile -v /path/to/repo Initializing repository at "/path/to/repo" Enter new passphrase: Enter same passphrase again: Remember your passphrase. Your data will be inaccessible without it. Key in "/root/.config/borg/keys/mnt_backup" created. Keep this key safe. Your data will be inaccessible without it. Synchronizing chunks cache... Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0. Done. # Change key file passphrase $ borg change-passphrase -v /path/to/repo Enter passphrase for key /root/.config/borg/keys/mnt_backup: Enter new passphrase: Enter same passphrase again: Remember your passphrase. Your data will be inaccessible without it. Key updated .. include:: usage/serve.rst.inc Examples ~~~~~~~~ borg serve has special support for ssh forced commands (see ``authorized_keys`` example below): it will detect that you use such a forced command and extract the value of the ``--restrict-to-path`` option(s). It will then parse the original command that came from the client, makes sure that it is also ``borg serve`` and enforce path restriction(s) as given by the forced command. That way, other options given by the client (like ``--info`` or ``--umask``) are preserved (and are not fixed by the forced command). :: # Allow an SSH keypair to only run borg, and only have access to /path/to/repo. # Use key options to disable unneeded and potentially dangerous SSH functionality. # This will help to secure an automated remote backup system. $ cat ~/.ssh/authorized_keys command="borg serve --restrict-to-path /path/to/repo",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-rsa AAAAB3[...] .. include:: usage/upgrade.rst.inc Examples ~~~~~~~~ :: # Upgrade the borg repository to the most recent version. $ borg upgrade -v /path/to/repo making a hardlink copy in /path/to/repo.upgrade-2016-02-15-20:51:55 opening attic repository with borg and converting no key file found for repository converting repo index /path/to/repo/index.0 converting 1 segments... converting borg 0.xx to borg current no key file found for repository Upgrading a passphrase encrypted attic repo ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ attic offered a "passphrase" encryption mode, but this was removed in borg 1.0 and replaced by the "repokey" mode (which stores the passphrase-protected encryption key into the repository config). Thus, to upgrade a "passphrase" attic repo to a "repokey" borg repo, 2 steps are needed, in this order: - borg upgrade repo - borg migrate-to-repokey repo .. include:: usage/break-lock.rst.inc Miscellaneous Help ------------------ .. include:: usage/help.rst.inc Debug Commands -------------- There are some more commands (all starting with "debug-") which are all **not intended for normal use** and **potentially very dangerous** if used incorrectly. For example, ``borg debug-put-obj`` and ``borg debug-delete-obj`` will only do what their name suggests: put objects into repo / delete objects from repo. Please note: - they will not update the chunks cache (chunks index) about the object - they will not update the manifest (so no automatic chunks index resync is triggered) - they will not check whether the object is in use (e.g. before delete-obj) - they will not update any metadata which may point to the object They exist to improve debugging capabilities without direct system access, e.g. in case you ever run into some severe malfunction. Use them only if you know what you are doing or if a trusted |project_name| developer tells you what to do. Additional Notes ---------------- Here are misc. notes about topics that are maybe not covered in enough detail in the usage section. Item flags ~~~~~~~~~~ ``borg create -v --list`` outputs a verbose list of all files, directories and other file system items it considered (no matter whether they had content changes or not). For each item, it prefixes a single-letter flag that indicates type and/or status of the item. If you are interested only in a subset of that output, you can give e.g. ``--filter=AME`` and it will only show regular files with A, M or E status (see below). A uppercase character represents the status of a regular file relative to the "files" cache (not relative to the repo -- this is an issue if the files cache is not used). Metadata is stored in any case and for 'A' and 'M' also new data chunks are stored. For 'U' all data chunks refer to already existing chunks. - 'A' = regular file, added (see also :ref:`a_status_oddity` in the FAQ) - 'M' = regular file, modified - 'U' = regular file, unchanged - 'E' = regular file, an error happened while accessing/reading *this* file A lowercase character means a file type other than a regular file, borg usually just stores their metadata: - 'd' = directory - 'b' = block device - 'c' = char device - 'h' = regular file, hardlink (to already seen inodes) - 's' = symlink - 'f' = fifo Other flags used include: - 'i' = backup data was read from standard input (stdin) - '-' = dry run, item was *not* backed up - '?' = missing status code (if you see this, please file a bug report!) --chunker-params ~~~~~~~~~~~~~~~~ The chunker params influence how input files are cut into pieces (chunks) which are then considered for deduplication. They also have a big impact on resource usage (RAM and disk space) as the amount of resources needed is (also) determined by the total amount of chunks in the repository (see `Indexes / Caches memory usage` for details). ``--chunker-params=10,23,16,4095`` results in a fine-grained deduplication and creates a big amount of chunks and thus uses a lot of resources to manage them. This is good for relatively small data volumes and if the machine has a good amount of free RAM and disk space. ``--chunker-params=19,23,21,4095`` (default) results in a coarse-grained deduplication and creates a much smaller amount of chunks and thus uses less resources. This is good for relatively big data volumes and if the machine has a relatively low amount of free RAM and disk space. If you already have made some archives in a repository and you then change chunker params, this of course impacts deduplication as the chunks will be cut differently. In the worst case (all files are big and were touched in between backups), this will store all content into the repository again. Usually, it is not that bad though: - usually most files are not touched, so it will just re-use the old chunks it already has in the repo - files smaller than the (both old and new) minimum chunksize result in only one chunk anyway, so the resulting chunks are same and deduplication will apply If you switch chunker params to save resources for an existing repo that already has some backup archives, you will see an increasing effect over time, when more and more files have been touched and stored again using the bigger chunksize **and** all references to the smaller older chunks have been removed (by deleting / pruning archives). If you want to see an immediate big effect on resource usage, you better start a new repository when changing chunker params. For more details, see :ref:`chunker_details`. ``--noatime / --noctime`` ~~~~~~~~~~~~~~~~~~~~~~~~~ You can use these ``borg create`` options to not store the respective timestamp into the archive, in case you do not really need it. Besides saving a little space for the not archived timestamp, it might also affect metadata stream deduplication: if only this timestamp changes between backups and is stored into the metadata stream, the metadata stream chunks won't deduplicate just because of that. --umask ~~~~~~~ If you use ``--umask``, make sure that all repository-modifying borg commands (create, delete, prune) that access the repository in question use the same ``--umask`` value. If multiple machines access the same repository, this should hold true for all of them. --read-special ~~~~~~~~~~~~~~ The --read-special option is special - you do not want to use it for normal full-filesystem backups, but rather after carefully picking some targets for it. The option ``--read-special`` triggers special treatment for block and char device files as well as FIFOs. Instead of storing them as such a device (or FIFO), they will get opened, their content will be read and in the backup archive they will show up like a regular file. Symlinks will also get special treatment if (and only if) they point to such a special file: instead of storing them as a symlink, the target special file will get processed as described above. One intended use case of this is backing up the contents of one or multiple block devices, like e.g. LVM snapshots or inactive LVs or disk partitions. You need to be careful about what you include when using ``--read-special``, e.g. if you include ``/dev/zero``, your backup will never terminate. Restoring such files' content is currently only supported one at a time via ``--stdout`` option (and you have to redirect stdout to where ever it shall go, maybe directly into an existing device file of your choice or indirectly via ``dd``). To some extent, mounting a backup archive with the backups of special files via ``borg mount`` and then loop-mounting the image files from inside the mount point will work. If you plan to access a lot of data in there, it likely will scale and perform better if you do not work via the FUSE mount. Example +++++++ Imagine you have made some snapshots of logical volumes (LVs) you want to backup. .. note:: For some scenarios, this is a good method to get "crash-like" consistency (I call it crash-like because it is the same as you would get if you just hit the reset button or your machine would abrubtly and completely crash). This is better than no consistency at all and a good method for some use cases, but likely not good enough if you have databases running. Then you create a backup archive of all these snapshots. The backup process will see a "frozen" state of the logical volumes, while the processes working in the original volumes continue changing the data stored there. You also add the output of ``lvdisplay`` to your backup, so you can see the LV sizes in case you ever need to recreate and restore them. After the backup has completed, you remove the snapshots again. :: $ # create snapshots here $ lvdisplay > lvdisplay.txt $ borg create --read-special /path/to/repo::arch lvdisplay.txt /dev/vg0/*-snapshot $ # remove snapshots here Now, let's see how to restore some LVs from such a backup. :: $ borg extract /path/to/repo::arch lvdisplay.txt $ # create empty LVs with correct sizes here (look into lvdisplay.txt). $ # we assume that you created an empty root and home LV and overwrite it now: $ borg extract --stdout /path/to/repo::arch dev/vg0/root-snapshot > /dev/vg0/root $ borg extract --stdout /path/to/repo::arch dev/vg0/home-snapshot > /dev/vg0/home .. _append_only_mode: Append-only mode ~~~~~~~~~~~~~~~~ A repository can be made "append-only", which means that Borg will never overwrite or delete committed data. This is useful for scenarios where multiple machines back up to a central backup server using ``borg serve``, since a hacked machine cannot delete backups permanently. To activate append-only mode, edit the repository ``config`` file and add a line ``append_only=1`` to the ``[repository]`` section (or edit the line if it exists). In append-only mode Borg will create a transaction log in the ``transactions`` file, where each line is a transaction and a UTC timestamp. In addition, ``borg serve`` can act as if a repository is in append-only mode with its option ``--append-only``. This can be very useful for fine-tuning access control in ``.ssh/authorized_keys`` :: command="borg serve --append-only ..." ssh-rsa command="borg serve ..." ssh-rsa Please note that if you run ``borg init`` via a ``borg serve --append-only`` server, the repository config will be created with a ``append_only=1`` entry. This behaviour is subject to change in a later borg version. So, be aware of it for now, but do not rely on it. Example +++++++ Suppose an attacker remotely deleted all backups, but your repository was in append-only mode. A transaction log in this situation might look like this: :: transaction 1, UTC time 2016-03-31T15:53:27.383532 transaction 5, UTC time 2016-03-31T15:53:52.588922 transaction 11, UTC time 2016-03-31T15:54:23.887256 transaction 12, UTC time 2016-03-31T15:55:54.022540 transaction 13, UTC time 2016-03-31T15:55:55.472564 From your security logs you conclude the attacker gained access at 15:54:00 and all the backups where deleted or replaced by compromised backups. From the log you know that transactions 11 and later are compromised. Note that the transaction ID is the name of the *last* file in the transaction. For example, transaction 11 spans files 6 to 11. In a real attack you'll likely want to keep the compromised repository intact to analyze what the attacker tried to achieve. It's also a good idea to make this copy just in case something goes wrong during the recovery. Since recovery is done by deleting some files, a hard link copy (``cp -al``) is sufficient. The first step to reset the repository to transaction 5, the last uncompromised transaction, is to remove the ``hints.N`` and ``index.N`` files in the repository (these two files are always expendable). In this example N is 13. Then remove or move all segment files from the segment directories in ``data/`` starting with file 6:: rm data/**/{6..13} That's all to it. Drawbacks +++++++++ As data is only appended, and nothing removed, commands like ``prune`` or ``delete`` won't free disk space, they merely tag data as deleted in a new transaction. Be aware that as soon as you write to the repo in non-append-only mode (e.g. prune, delete or create archives from an admin machine), it will remove the deleted objects permanently (including the ones that were already marked as deleted, but not removed, in append-only mode). Note that you can go back-and-forth between normal and append-only operation by editing the configuration file, it's not a "one way trip". Further considerations ++++++++++++++++++++++ Append-only mode is not respected by tools other than Borg. ``rm`` still works on the repository. Make sure that backup client machines only get to access the repository via ``borg serve``. Ensure that no remote access is possible if the repository is temporarily set to normal mode for e.g. regular pruning. Further protections can be implemented, but are outside of Borgs scope. For example, file system snapshots or wrapping ``borg serve`` to set special permissions or ACLs on new data files. SSH batch mode ~~~~~~~~~~~~~~ When running |project_name| using an automated script, ``ssh`` might still ask for a password, even if there is an SSH key for the target server. Use this to make scripts more robust:: export BORG_RSH='ssh -oBatchMode=yes' borgbackup-1.0.13/requirements.d/0000755000175000017500000000000013431514213016633 5ustar useruser00000000000000borgbackup-1.0.13/requirements.d/development.lock.txt0000644000175000017500000000023413431513133022644 0ustar useruser00000000000000virtualenv==13.1.2 setuptools-scm==3.1.0 pluggy==0.6.0 tox==3.1.3 pytest==3.5.1 pytest-xdist==1.24.0 pytest-cov==2.6.0 pytest-benchmark==3.1.1 Cython==0.29 borgbackup-1.0.13/requirements.d/development.txt0000644000175000017500000000021413431513133021713 0ustar useruser00000000000000setuptools setuptools_scm pip virtualenv<14.0 pluggy<0.7 tox<3.2.0 pytest<3.6.0 pytest-xdist pytest-cov pytest-benchmark Cython!=0.27 twine borgbackup-1.0.13/requirements.d/docs.txt0000644000175000017500000000000113431513133020313 0ustar useruser00000000000000 borgbackup-1.0.13/requirements.d/fuse.txt0000644000175000017500000000016213431513133020335 0ustar useruser00000000000000# low-level FUSE support library for "borg mount" # please see the comments in setup.py about llfuse. llfuse<2.0 borgbackup-1.0.13/scripts/0000755000175000017500000000000013431514213015355 5ustar useruser00000000000000borgbackup-1.0.13/scripts/borg.exe.spec0000644000175000017500000000216413431513133017745 0ustar useruser00000000000000# -*- mode: python -*- # this pyinstaller spec file is used to build borg binaries on posix platforms import os, sys basepath = '/vagrant/borg/borg' block_cipher = None a = Analysis([os.path.join(basepath, 'borg/__main__.py'), ], pathex=[basepath, ], binaries=[], datas=[ ('../borg/paperkey.html', 'borg'), ], hiddenimports=['borg.platform.posix'], hookspath=[], runtime_hooks=[], excludes=[], win_no_prefer_redirects=False, win_private_assemblies=False, cipher=block_cipher) if sys.platform == 'darwin': # do not bundle the osxfuse libraries, so we do not get a version # mismatch to the installed kernel driver of osxfuse. a.binaries = [b for b in a.binaries if 'libosxfuse' not in b[0]] pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher) exe = EXE(pyz, a.scripts, a.binaries, a.zipfiles, a.datas, name='borg.exe', debug=False, strip=False, upx=True, console=True ) borgbackup-1.0.13/scripts/glibc_check.py0000755000175000017500000000323613431513133020153 0ustar useruser00000000000000#!/usr/bin/env python3 """ Check if all given binaries work with the given glibc version. glibc_check.py 2.11 bin [bin ...] rc = 0 means "yes", rc = 1 means "no". """ import re import subprocess import sys verbose = True objdump = "objdump -T %s" glibc_re = re.compile(r'GLIBC_([0-9]\.[0-9]+)') def parse_version(v): major, minor = v.split('.') return int(major), int(minor) def format_version(version): return "%d.%d" % version def main(): given = parse_version(sys.argv[1]) filenames = sys.argv[2:] overall_versions = set() for filename in filenames: try: output = subprocess.check_output(objdump % filename, shell=True, stderr=subprocess.STDOUT) output = output.decode('utf-8') versions = set(parse_version(match.group(1)) for match in glibc_re.finditer(output)) requires_glibc = max(versions) overall_versions.add(requires_glibc) if verbose: print("%s %s" % (filename, format_version(requires_glibc))) except subprocess.CalledProcessError as e: if verbose: print("%s errored." % filename) wanted = max(overall_versions) ok = given >= wanted if verbose: if ok: print("The binaries work with the given glibc %s." % format_version(given)) else: print("The binaries do not work with the given glibc %s. " "Minimum is: %s" % (format_version(given), format_version(wanted))) return ok if __name__ == '__main__': ok = main() sys.exit(0 if ok else 1) borgbackup-1.0.13/scripts/hash_sizes.py0000644000175000017500000000543713431513133020100 0ustar useruser00000000000000""" Compute hashtable sizes with nices properties - prime sizes (for small to medium sizes) - 2 prime-factor sizes (for big sizes) - fast growth for small sizes - slow growth for big sizes Note: this is just a tool for developers. within borgbackup, it is just used to generate hash_sizes definition for _hashindex.c. """ from collections import namedtuple K, M, G = 2**10, 2**20, 2**30 # hash table size (in number of buckets) start, end_p1, end_p2 = 1 * K, 127 * M, 2 * G - 10 * M # stay well below 2^31 - 1 Policy = namedtuple("Policy", "upto grow") policies = [ # which growth factor to use when growing a hashtable of size < upto # grow fast (*2.0) at the start so we do not have to resize too often (expensive). # grow slow (*1.1) for huge hash tables (do not jump too much in memory usage) Policy(256*K, 2.0), Policy(2*M, 1.7), Policy(16*M, 1.4), Policy(128*M, 1.2), Policy(2*G-1, 1.1), ] # slightly modified version of: # http://www.macdevcenter.com/pub/a/python/excerpt/pythonckbk_chap1/index1.html?page=2 def eratosthenes(): """Yields the sequence of prime numbers via the Sieve of Eratosthenes.""" D = {} # map each composite integer to its first-found prime factor q = 2 # q gets 2, 3, 4, 5, ... ad infinitum while True: p = D.pop(q, None) if p is None: # q not a key in D, so q is prime, therefore, yield it yield q # mark q squared as not-prime (with q as first-found prime factor) D[q * q] = q else: # let x <- smallest (N*p)+q which wasn't yet known to be composite # we just learned x is composite, with p first-found prime factor, # since p is the first-found prime factor of q -- find and mark it x = p + q while x in D: x += p D[x] = p q += 1 def two_prime_factors(pfix=65537): """Yields numbers with 2 prime factors pfix and p.""" for p in eratosthenes(): yield pfix * p def get_grow_factor(size): for p in policies: if size < p.upto: return p.grow def find_bigger_prime(gen, i): while True: p = next(gen) if p >= i: return p def main(): sizes = [] i = start gen = eratosthenes() while i < end_p1: grow_factor = get_grow_factor(i) p = find_bigger_prime(gen, i) sizes.append(p) i = int(i * grow_factor) gen = two_prime_factors() # for lower ram consumption while i < end_p2: grow_factor = get_grow_factor(i) p = find_bigger_prime(gen, i) sizes.append(p) i = int(i * grow_factor) print("""\ static int hash_sizes[] = { %s }; """ % ', '.join(str(size) for size in sizes)) if __name__ == '__main__': main() borgbackup-1.0.13/scripts/sdist-sign0000755000175000017500000000046713431513133017376 0ustar useruser00000000000000#!/bin/bash R=$1 if [ "$R" = "" ]; then echo "Usage: sdist-sign 1.2.3" exit fi if [ "$QUBES_GPG_DOMAIN" = "" ]; then GPG=gpg else GPG=qubes-gpg-client-wrapper fi python setup.py sdist D=dist/borgbackup-$R.tar.gz $GPG --detach-sign --local-user "Thomas Waldmann" --armor --output $D.asc $D borgbackup-1.0.13/scripts/sign-binaries0000755000175000017500000000051413431513133020035 0ustar useruser00000000000000#!/bin/bash D=$1 if [ "$D" = "" ]; then echo "Usage: sign-binaries 201912312359" exit fi if [ "$QUBES_GPG_DOMAIN" = "" ]; then GPG=gpg else GPG=qubes-gpg-client-wrapper fi for file in dist/borg-*; do $GPG --local-user "Thomas Waldmann" --armor --detach-sign --output $file.asc $file done touch -t $D dist/* borgbackup-1.0.13/scripts/upload-pypi0000755000175000017500000000043113431513133017544 0ustar useruser00000000000000#!/bin/bash R=$1 if [ "$R" = "" ]; then echo "Usage: upload-pypi 1.2.3 [test]" exit fi if [ "$2" = "test" ]; then export TWINE_REPOSITORY_URL=https://test.pypi.org/legacy/ else export TWINE_REPOSITORY_URL= fi D=dist/borgbackup-$R.tar.gz twine upload $D.asc $D borgbackup-1.0.13/setup.cfg0000644000175000017500000000036113431514213015507 0ustar useruser00000000000000[pytest] python_files = testsuite/*.py [flake8] ignore = E122,E123,E125,E126,E127,E128,E226,E402,E722,E741,F401,F405,F811 max-line-length = 255 exclude = build,dist,.git,.idea,.cache,.tox,docs/conf.py [egg_info] tag_build = tag_date = 0 borgbackup-1.0.13/setup.py0000644000175000017500000002607513431513133015412 0ustar useruser00000000000000# -*- encoding: utf-8 *-* import os import re import sys from glob import glob from distutils.command.build import build from distutils.core import Command min_python = (3, 4) my_python = sys.version_info if my_python < min_python: print("Borg requires Python %d.%d or later" % min_python) sys.exit(1) # Are we building on ReadTheDocs? on_rtd = os.environ.get('READTHEDOCS') install_requires = [ # msgpack pure python data corruption was fixed in 0.4.6. # msgpack 0.5.0 was a bit of a troublemaker. # also, msgpack dropped py34 support at 0.5.0. 'msgpack-python>=0.4.6,<0.5.0', ] # note for package maintainers: if you package borgbackup for distribution, # please add llfuse as a *requirement* on all platforms that have a working # llfuse package. "borg mount" needs llfuse to work. # if you do not have llfuse, do not require it, most of borgbackup will work. extras_require = { # llfuse 0.40 (tested, proven, ok), needs FUSE version >= 2.8.0 # llfuse 0.41 (tested shortly, looks ok), needs FUSE version >= 2.8.0 # llfuse 0.41.1 (tested shortly, looks ok), needs FUSE version >= 2.8.0 # llfuse 0.42 (tested shortly, looks ok), needs FUSE version >= 2.8.0 # llfuse 1.0 (tested shortly, looks ok), needs FUSE version >= 2.8.0 # llfuse 1.1.1 (tested shortly, looks ok), needs FUSE version >= 2.8.0 # llfuse 2.0 will break API 'fuse': ['llfuse<2.0', ], } if sys.platform.startswith('freebsd'): # llfuse was frequently broken / did not build on freebsd # llfuse 0.41.1, 1.1 are ok extras_require['fuse'] = ['llfuse <2.0, !=0.42.*, !=0.43, !=1.0', ] from setuptools import setup, Extension from setuptools.command.sdist import sdist compress_source = 'borg/compress.pyx' crypto_source = 'borg/crypto.pyx' chunker_source = 'borg/chunker.pyx' hashindex_source = 'borg/hashindex.pyx' platform_linux_source = 'borg/platform_linux.pyx' platform_darwin_source = 'borg/platform_darwin.pyx' platform_freebsd_source = 'borg/platform_freebsd.pyx' try: from Cython.Distutils import build_ext import Cython.Compiler.Main as cython_compiler class Sdist(sdist): def __init__(self, *args, **kwargs): for src in glob('borg/*.pyx'): cython_compiler.compile(src, cython_compiler.default_options) super().__init__(*args, **kwargs) def make_distribution(self): self.filelist.extend([ 'borg/compress.c', 'borg/crypto.c', 'borg/chunker.c', 'borg/_chunker.c', 'borg/hashindex.c', 'borg/_hashindex.c', 'borg/platform_linux.c', 'borg/platform_freebsd.c', 'borg/platform_darwin.c', ]) super().make_distribution() except ImportError: class Sdist(sdist): def __init__(self, *args, **kwargs): raise Exception('Cython is required to run sdist') compress_source = compress_source.replace('.pyx', '.c') crypto_source = crypto_source.replace('.pyx', '.c') chunker_source = chunker_source.replace('.pyx', '.c') hashindex_source = hashindex_source.replace('.pyx', '.c') platform_linux_source = platform_linux_source.replace('.pyx', '.c') platform_freebsd_source = platform_freebsd_source.replace('.pyx', '.c') platform_darwin_source = platform_darwin_source.replace('.pyx', '.c') from distutils.command.build_ext import build_ext if not on_rtd and not all(os.path.exists(path) for path in [ compress_source, crypto_source, chunker_source, hashindex_source, platform_linux_source, platform_freebsd_source, platform_darwin_source]): raise ImportError('The GIT version of Borg needs Cython. Install Cython or use a released version.') def detect_openssl(prefixes): for prefix in prefixes: filename = os.path.join(prefix, 'include', 'openssl', 'evp.h') if os.path.exists(filename): with open(filename, 'rb') as fd: if b'PKCS5_PBKDF2_HMAC(' in fd.read(): return prefix def detect_lz4(prefixes): for prefix in prefixes: filename = os.path.join(prefix, 'include', 'lz4.h') if os.path.exists(filename): with open(filename, 'rb') as fd: if b'LZ4_decompress_safe' in fd.read(): return prefix include_dirs = [] library_dirs = [] possible_openssl_prefixes = ['/usr', '/usr/local', '/usr/local/opt/openssl', '/usr/local/ssl', '/usr/local/openssl', '/usr/local/borg', '/opt/local'] if os.environ.get('BORG_OPENSSL_PREFIX'): possible_openssl_prefixes.insert(0, os.environ.get('BORG_OPENSSL_PREFIX')) ssl_prefix = detect_openssl(possible_openssl_prefixes) if not ssl_prefix: raise Exception('Unable to find OpenSSL >= 1.0 headers. (Looked here: {})'.format(', '.join(possible_openssl_prefixes))) include_dirs.append(os.path.join(ssl_prefix, 'include')) library_dirs.append(os.path.join(ssl_prefix, 'lib')) possible_lz4_prefixes = ['/usr', '/usr/local', '/usr/local/opt/lz4', '/usr/local/lz4', '/usr/local/borg', '/opt/local'] if os.environ.get('BORG_LZ4_PREFIX'): possible_lz4_prefixes.insert(0, os.environ.get('BORG_LZ4_PREFIX')) lz4_prefix = detect_lz4(possible_lz4_prefixes) if lz4_prefix: include_dirs.append(os.path.join(lz4_prefix, 'include')) library_dirs.append(os.path.join(lz4_prefix, 'lib')) elif not on_rtd: raise Exception('Unable to find LZ4 headers. (Looked here: {})'.format(', '.join(possible_lz4_prefixes))) with open('README.rst', 'r') as fd: long_description = fd.read() # remove badges long_description = re.compile(r'^\.\. start-badges.*^\.\. end-badges', re.M | re.S).sub('', long_description) # remove |substitutions| long_description = re.compile(r'\|screencast\|').sub('', long_description) # remove unknown directives long_description = re.compile(r'^\.\. highlight:: \w+$', re.M).sub('', long_description) class build_usage(Command): description = "generate usage for each command" user_options = [ ('output=', 'O', 'output directory'), ] def initialize_options(self): pass def finalize_options(self): pass def run(self): print('generating usage docs') if not os.path.exists('docs/usage'): os.mkdir('docs/usage') # allows us to build docs without the C modules fully loaded during help generation from borg.archiver import Archiver parser = Archiver().build_parser(prog='borg') self.generate_level("", parser, Archiver) def generate_level(self, prefix, parser, Archiver): is_subcommand = False choices = {} for action in parser._actions: if action.choices is not None and 'SubParsersAction' in str(action.__class__): is_subcommand = True for cmd, parser in action.choices.items(): choices[prefix + cmd] = parser if prefix and not choices: return print('found commands: %s' % list(choices.keys())) for command, parser in sorted(choices.items()): if command.startswith('debug'): print('skipping', command) continue print('generating help for %s' % command) if self.generate_level(command + " ", parser, Archiver): continue with open('docs/usage/%s.rst.inc' % command.replace(" ", "_"), 'w') as doc: doc.write(".. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!\n\n") if command == 'help': for topic in Archiver.helptext: params = {"topic": topic, "underline": '~' * len('borg help ' + topic)} doc.write(".. _borg_{topic}:\n\n".format(**params)) doc.write("borg help {topic}\n{underline}\n\n".format(**params)) doc.write(Archiver.helptext[topic]) else: params = {"command": command, "command_": command.replace(' ', '_'), "underline": '-' * len('borg ' + command)} doc.write(".. _borg_{command_}:\n\n".format(**params)) doc.write("borg {command}\n{underline}\n::\n\n".format(**params)) epilog = parser.epilog parser.epilog = None doc.write(re.sub("^", " ", parser.format_help(), flags=re.M)) doc.write("\nDescription\n~~~~~~~~~~~\n") doc.write(epilog) return is_subcommand cmdclass = { 'build_ext': build_ext, 'build_usage': build_usage, 'sdist': Sdist } ext_modules = [] if not on_rtd: ext_modules += [ Extension('borg.compress', [compress_source], libraries=['lz4'], include_dirs=include_dirs, library_dirs=library_dirs), Extension('borg.crypto', [crypto_source], libraries=['crypto'], include_dirs=include_dirs, library_dirs=library_dirs), Extension('borg.chunker', [chunker_source]), Extension('borg.hashindex', [hashindex_source]) ] if sys.platform == 'linux': ext_modules.append(Extension('borg.platform_linux', [platform_linux_source], libraries=['acl'])) elif sys.platform.startswith('freebsd'): ext_modules.append(Extension('borg.platform_freebsd', [platform_freebsd_source])) elif sys.platform == 'darwin': ext_modules.append(Extension('borg.platform_darwin', [platform_darwin_source])) setup( name='borgbackup', use_scm_version={ 'write_to': 'borg/_version.py', }, author='The Borg Collective (see AUTHORS file)', author_email='borgbackup@python.org', url='https://borgbackup.readthedocs.io/', description='Deduplicated, encrypted, authenticated and compressed backups', long_description=long_description, license='BSD', platforms=['Linux', 'MacOS X', 'FreeBSD', 'OpenBSD', 'NetBSD', ], classifiers=[ 'Development Status :: 4 - Beta', 'Environment :: Console', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: BSD License', 'Operating System :: POSIX :: BSD :: FreeBSD', 'Operating System :: POSIX :: BSD :: OpenBSD', 'Operating System :: POSIX :: BSD :: NetBSD', 'Operating System :: MacOS :: MacOS X', 'Operating System :: POSIX :: Linux', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Topic :: Security :: Cryptography', 'Topic :: System :: Archiving :: Backup', ], packages=['borg', 'borg.testsuite', ], entry_points={ 'console_scripts': [ 'borg = borg.archiver:main', ] }, include_package_data=True, package_data={ 'borg': ['paperkey.html'], 'borg.testsuite': ['attic.tar.gz'], }, cmdclass=cmdclass, ext_modules=ext_modules, setup_requires=['setuptools_scm>=1.7'], install_requires=install_requires, extras_require=extras_require, ) borgbackup-1.0.13/tox.ini0000644000175000017500000000120213431513133015174 0ustar useruser00000000000000# tox configuration - if you change anything here, run this to verify: # fakeroot -u tox --recreate [tox] envlist = py{34,35,36},flake8 [testenv] # Change dir to avoid import problem for cython code. The directory does # not really matter, should be just different from the toplevel dir. changedir = {toxworkdir} deps = -rrequirements.d/development.lock.txt -rrequirements.d/fuse.txt commands = py.test -n {env:XDISTN:4} -rs --cov=borg --cov-config=../.coveragerc --benchmark-skip --pyargs {posargs:borg.testsuite} # fakeroot -u needs some env vars: passenv = * [testenv:flake8] changedir = deps = flake8<3.6.0 commands = flake8