././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9739063 tuspy-1.0.3/0000755000076500000240000000000014536326100012122 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/AUTHORS0000644000076500000240000000007014536311414013171 0ustar00mariusstaffAuthors: Ifedapo Olarewaju ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/LICENSE0000644000076500000240000000206614536311414013135 0ustar00mariusstaffMIT License Copyright (c) 2016 Ifedapo .A. Olarewaju Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/MANIFEST.in0000644000076500000240000000002214536311414013654 0ustar00mariusstaffinclude README.md ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9737072 tuspy-1.0.3/PKG-INFO0000644000076500000240000001023114536326100013214 0ustar00mariusstaffMetadata-Version: 2.1 Name: tuspy Version: 1.0.3 Summary: A Python client for the tus resumable upload protocol -> http://tus.io Home-page: http://github.com/tus/tus-py-client/ Author: Ifedapo Olarewaju Author-email: ifedapoolarewaju@gmail.com License: MIT Platform: any Classifier: Programming Language :: Python Classifier: Natural Language :: English Classifier: Environment :: Web Environment Classifier: Intended Audience :: Developers Classifier: Development Status :: 3 - Alpha Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Internet :: File Transfer Protocol (FTP) Classifier: Topic :: Communications :: File Sharing Requires-Python: >=3.5.3 Description-Content-Type: text/markdown License-File: LICENSE License-File: AUTHORS Requires-Dist: requests>=2.18.4 Requires-Dist: tinydb>=3.5.0 Requires-Dist: aiohttp>=3.6.2 Provides-Extra: test Requires-Dist: responses>=0.5.1; extra == "test" Requires-Dist: aioresponses>=0.6.2; extra == "test" Requires-Dist: coverage>=4.2; extra == "test" Requires-Dist: pytest>=3.0.3; extra == "test" Requires-Dist: pytest-cov<2.6,>=2.3.1; extra == "test" Requires-Dist: parametrize>=0.1.1; extra == "test" Provides-Extra: dev Requires-Dist: tox>=2.3.1; extra == "dev" Requires-Dist: sphinx-autobuild==2021.3.14; extra == "dev" Requires-Dist: Sphinx==1.7.1; extra == "dev" # tus-py-client [![Build Status](https://github.com/tus/tus-py-client/actions/workflows/CI.yml/badge.svg)](https://github.com/tus/tus-py-client/actions/workflows/CI.yml) > **tus** is a protocol based on HTTP for _resumable file uploads_. Resumable > means that an upload can be interrupted at any moment and can be resumed without > re-uploading the previous data again. An interruption may happen willingly, if > the user wants to pause, or by accident in case of a network issue or server > outage. **tus-py-client** is a Python client for uploading files using the _tus_ protocol to any remote server supporting it. ## Documentation See documentation here: http://tus-py-client.readthedocs.io/en/latest/ ## Get started ```bash pip install tuspy ``` Now you are ready to use the api. ```python from tusclient import client # Set Authorization headers if it is required # by the tus server. my_client = client.TusClient('http://tusd.tusdemo.net/files/', headers={'Authorization': 'Basic xxyyZZAAbbCC='}) # Set more headers. my_client.set_headers({'HEADER_NAME': 'HEADER_VALUE'}) uploader = my_client.uploader('path/to/file.ext', chunk_size=200) # A file stream may also be passed in place of a file path. fs = open('path/to/file.ext', mode=) uploader = my_client.uploader(file_stream=fs, chunk_size=200) # Upload a chunk i.e 200 bytes. uploader.upload_chunk() # Uploads the entire file. # This uploads chunk by chunk. uploader.upload() # you could increase the chunk size to reduce the # number of upload_chunk cycles. uploader.chunk_size = 800 uploader.upload() # Continue uploading chunks till total chunks uploaded reaches 1000 bytes. uploader.upload(stop_at=1000) ``` If the upload url is known and the client headers are not required, uploaders can also be used standalone. ```python from tusclient.uploader import Uploader my_uploader = Uploader('path/to/file.ext', url='http://tusd.tusdemo.net/files/abcdef123456', chunk_size=200) ``` ## Development If you want to work on tus-py-client internally, follow these few steps: 1. Setup virtual environment and install dependencies ```bash python -m venv env/ source env/bin/activate pip install -e .[test] ``` 2. Running tests ```bash pytest ``` 3. Releasing a new version (see https://realpython.com/pypi-publish-python-package/) ```bash # Update version in tusclient/__init__.py vim tusclient/__init__.py # Update changelogs vim CHANGELOG.md pytest # Commit and tag git commit -m 'v1.2.3' git tag v1.2.3 # Build and release pip install build twine python -m build twine check dist/* twine upload dist/* # Then: make release on GitHub ``` ## License MIT ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/README.md0000644000076500000240000000534714536311414013414 0ustar00mariusstaff# tus-py-client [![Build Status](https://github.com/tus/tus-py-client/actions/workflows/CI.yml/badge.svg)](https://github.com/tus/tus-py-client/actions/workflows/CI.yml) > **tus** is a protocol based on HTTP for _resumable file uploads_. Resumable > means that an upload can be interrupted at any moment and can be resumed without > re-uploading the previous data again. An interruption may happen willingly, if > the user wants to pause, or by accident in case of a network issue or server > outage. **tus-py-client** is a Python client for uploading files using the _tus_ protocol to any remote server supporting it. ## Documentation See documentation here: http://tus-py-client.readthedocs.io/en/latest/ ## Get started ```bash pip install tuspy ``` Now you are ready to use the api. ```python from tusclient import client # Set Authorization headers if it is required # by the tus server. my_client = client.TusClient('http://tusd.tusdemo.net/files/', headers={'Authorization': 'Basic xxyyZZAAbbCC='}) # Set more headers. my_client.set_headers({'HEADER_NAME': 'HEADER_VALUE'}) uploader = my_client.uploader('path/to/file.ext', chunk_size=200) # A file stream may also be passed in place of a file path. fs = open('path/to/file.ext', mode=) uploader = my_client.uploader(file_stream=fs, chunk_size=200) # Upload a chunk i.e 200 bytes. uploader.upload_chunk() # Uploads the entire file. # This uploads chunk by chunk. uploader.upload() # you could increase the chunk size to reduce the # number of upload_chunk cycles. uploader.chunk_size = 800 uploader.upload() # Continue uploading chunks till total chunks uploaded reaches 1000 bytes. uploader.upload(stop_at=1000) ``` If the upload url is known and the client headers are not required, uploaders can also be used standalone. ```python from tusclient.uploader import Uploader my_uploader = Uploader('path/to/file.ext', url='http://tusd.tusdemo.net/files/abcdef123456', chunk_size=200) ``` ## Development If you want to work on tus-py-client internally, follow these few steps: 1. Setup virtual environment and install dependencies ```bash python -m venv env/ source env/bin/activate pip install -e .[test] ``` 2. Running tests ```bash pytest ``` 3. Releasing a new version (see https://realpython.com/pypi-publish-python-package/) ```bash # Update version in tusclient/__init__.py vim tusclient/__init__.py # Update changelogs vim CHANGELOG.md pytest # Commit and tag git commit -m 'v1.2.3' git tag v1.2.3 # Build and release pip install build twine python -m build twine check dist/* twine upload dist/* # Then: make release on GitHub ``` ## License MIT ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1702472767.973945 tuspy-1.0.3/setup.cfg0000644000076500000240000000004614536326100013743 0ustar00mariusstaff[egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469036.0 tuspy-1.0.3/setup.py0000644000076500000240000000315114536316654013651 0ustar00mariusstafffrom setuptools import setup import tusclient setup( name='tuspy', version=tusclient.__version__, url='http://github.com/tus/tus-py-client/', license='MIT', author='Ifedapo Olarewaju', install_requires=[ 'requests>=2.18.4', 'tinydb>=3.5.0', 'aiohttp>=3.6.2' ], extras_require={ 'test': [ 'responses>=0.5.1', 'aioresponses>=0.6.2', 'coverage>=4.2', 'pytest>=3.0.3', 'pytest-cov>=2.3.1,<2.6', 'parametrize>=0.1.1' ], 'dev': [ 'tox>=2.3.1', 'sphinx-autobuild==2021.3.14', 'Sphinx==1.7.1' ] }, author_email='ifedapoolarewaju@gmail.com', description='A Python client for the tus resumable upload protocol -> http://tus.io', long_description=open('README.md', encoding='utf-8').read(), long_description_content_type='text/markdown', packages=['tusclient', 'tusclient.fingerprint', 'tusclient.storage', 'tusclient.uploader'], include_package_data=True, platforms='any', classifiers=[ 'Programming Language :: Python', 'Natural Language :: English', 'Environment :: Web Environment', 'Intended Audience :: Developers', 'Development Status :: 3 - Alpha', 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Topic :: Software Development :: Libraries :: Python Modules', 'Topic :: Internet :: File Transfer Protocol (FTP)', 'Topic :: Communications :: File Sharing', ], python_requires=">=3.5.3", ) ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9704688 tuspy-1.0.3/tests/0000755000076500000240000000000014536326100013264 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472593.0 tuspy-1.0.3/tests/test_async_uploader.py0000644000076500000240000001147714536325621017726 0ustar00mariusstaffimport io import unittest from unittest import mock import asyncio from aioresponses import aioresponses, CallbackResult import responses import pytest from tusclient import exceptions, client class AsyncUploaderTest(unittest.TestCase): @responses.activate def setUp(self): self.client = client.TusClient('http://tusd.tusdemo.net/files/') self.url = 'http://tusd.tusdemo.net/files/15acd89eabdf5738ffc' responses.add(responses.HEAD, self.url, adding_headers={"upload-offset": "0"}) self.loop = asyncio.new_event_loop() self.async_uploader = self.client.async_uploader( './LICENSE', url=self.url) def tearDown(self): self.loop.stop() def _validate_request(self, url, **kwargs): self.assertEqual(self.url, str(url)) req_headers = kwargs['headers'] self.assertEqual(req_headers.get('Tus-Resumable'), '1.0.0') body = kwargs['data'] with open('./LICENSE', 'rb') as stream: expected_content = stream.read() self.assertEqual(expected_content, body) response_headers = { 'upload-offset': str(self.async_uploader.offset + self.async_uploader.get_request_length())} return CallbackResult(status=204, headers=response_headers) def test_upload_chunk(self): with aioresponses() as resps: resps.patch(self.url, callback=self._validate_request) request_length = self.async_uploader.get_request_length() self.loop.run_until_complete(self.async_uploader.upload_chunk()) self.assertEqual(self.async_uploader.offset, request_length) def test_upload_chunk_with_creation(self): with aioresponses() as resps: resps.post( self.client.url, status=201, headers={ "location": f"{self.client.url}hello" } ) resps.patch( f"{self.client.url}hello", headers={ "upload-offset": "5" } ) uploader = self.client.async_uploader( file_stream=io.BytesIO(b"hello") ) self.loop.run_until_complete(uploader.upload_chunk()) self.assertEqual(uploader.url, f"{self.client.url}hello") def test_upload(self): with aioresponses() as resps: resps.patch(self.url, callback=self._validate_request) self.loop.run_until_complete(self.async_uploader.upload()) self.assertEqual(self.async_uploader.offset, self.async_uploader.get_file_size()) def test_upload_empty(self): with aioresponses() as resps: resps.post( self.client.url, status=200, headers={ "upload-offset": "0", "location": f"{self.client.url}this-is-not-used" } ) resps.patch( f"{self.client.url}this-is-not-used", exception=ValueError( "PATCH request not allowed for empty file" ) ) # Upload an empty file async_uploader = self.client.async_uploader( file_stream=io.BytesIO(b"") ) self.loop.run_until_complete(async_uploader.upload()) # Upload URL being set means the POST request was sent and the empty # file was uploaded without a single PATCH request. self.assertTrue(async_uploader.url) def test_upload_retry(self): num_of_retries = 3 self.async_uploader.retries = num_of_retries self.async_uploader.retry_delay = 3 with aioresponses() as resps: resps.patch(self.url, status=00) self.assertEqual(self.async_uploader._retried, 0) with pytest.raises(exceptions.TusCommunicationError): self.loop.run_until_complete( self.async_uploader.upload_chunk()) self.assertEqual(self.async_uploader._retried, num_of_retries) def test_upload_verify_tls_cert(self): self.async_uploader.verify_tls_cert = False with aioresponses() as resps: ssl = None def validate_verify_tls_cert(url, **kwargs): nonlocal ssl ssl = kwargs['ssl'] response_headers = { 'upload-offset': str(self.async_uploader.offset + self.async_uploader.get_request_length()) } return CallbackResult(status=204, headers=response_headers) resps.patch( self.url, status=204, callback=validate_verify_tls_cert ) self.loop.run_until_complete(self.async_uploader.upload()) self.assertEqual(ssl, False) ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/tests/test_client.py0000644000076500000240000000323114536311414016154 0ustar00mariusstaffimport unittest import responses from tusclient import client from tusclient.uploader import Uploader, AsyncUploader class TusClientTest(unittest.TestCase): def setUp(self): self.client = client.TusClient('http://tusd.tusdemo.net/files/', headers={'foo': 'bar'}) def test_instance_attributes(self): self.assertEqual(self.client.url, 'http://tusd.tusdemo.net/files/') self.assertEqual(self.client.headers, {'foo': 'bar'}) def test_set_headers(self): self.client.set_headers({'foo': 'bar tender'}) self.assertEqual(self.client.headers, {'foo': 'bar tender'}) # uploader headers must update when client headers change self.client.set_headers({'food': 'at the bar'}) self.assertEqual(self.client.headers, {'foo': 'bar tender', 'food': 'at the bar'}) @responses.activate def test_uploader(self): url = 'http://tusd.tusdemo.net/files/15acd89eabdf5738ffc' responses.add(responses.HEAD, url, adding_headers={"upload-offset": "0"}) uploader = self.client.uploader('./LICENSE', url=url) self.assertIsInstance(uploader, Uploader) self.assertEqual(uploader.client, self.client) @responses.activate def test_async_uploader(self): url = 'http://tusd.tusdemo.net/files/15acd89eabdf5738ffc' responses.add(responses.HEAD, url, adding_headers={"upload-offset": "0"}) async_uploader = self.client.async_uploader('./LICENSE', url=url) self.assertIsInstance(async_uploader, AsyncUploader) self.assertEqual(async_uploader.client, self.client) ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/tests/test_filestorage.py0000644000076500000240000000153214536311414017204 0ustar00mariusstaffimport unittest import os from tusclient.storage import filestorage class FileStorageTest(unittest.TestCase): def setUp(self): self.storage_path = 'storage.json' self.storage = filestorage.FileStorage(self.storage_path) def tearDown(self): self.storage.close() os.remove(self.storage_path) def test_set_get_remove_item(self): url = 'http://tusd.tusdemo.net/files/unique_file_id' key = 'unique_key' url_2 = 'http://tusd.tusdemo.net/files/unique_file_id_2' key_2 = 'unique_key_2' self.storage.set_item(key, url) self.storage.set_item(key_2, url_2) self.assertEqual(self.storage.get_item(key), url) self.assertEqual(self.storage.get_item(key_2), url_2) self.storage.remove_item(key) self.assertIsNone(self.storage.get_item(key)) ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469036.0 tuspy-1.0.3/tests/test_fingerprint.py0000644000076500000240000000261014536316654017240 0ustar00mariusstaffimport io import unittest from parametrize import parametrize from tusclient.fingerprint import fingerprint FILEPATH_TEXT = "tests/sample_files/text.txt" FILEPATH_BINARY = "tests/sample_files/binary.png" class FileStorageTest(unittest.TestCase): def setUp(self): self.fingerprinter = fingerprint.Fingerprint() @parametrize( "filename", [FILEPATH_TEXT, FILEPATH_BINARY], ) def test_get_fingerpint(self, filename: str): with open(filename, "rb") as f: content = f.read() buff = io.BytesIO() buff.write(content) buff.seek(0) # reset buffer postion before reading with open(filename, "rb") as f: self.assertEqual( self.fingerprinter.get_fingerprint(buff), self.fingerprinter.get_fingerprint(f) ) @parametrize( "filename", [FILEPATH_TEXT, FILEPATH_BINARY], ) def test_unique_fingerprint(self, filename: str): with open(filename, "rb") as f: content = f.read() buff = io.BytesIO() buff.write(content + b's') # add some salt to change value buff.seek(0) # reset buffer postion before reading with open(filename, "rb") as f: self.assertNotEqual( self.fingerprinter.get_fingerprint(buff), self.fingerprinter.get_fingerprint(f) ) ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702468564.0 tuspy-1.0.3/tests/test_request.py0000644000076500000240000000433214536315724016401 0ustar00mariusstaffimport base64 import hashlib from parametrize import parametrize import responses from tusclient import request from tests import mixin FILEPATH_TEXT = "tests/sample_files/text.txt" FILEPATH_BINARY = "tests/sample_files/binary.png" class TusRequestTest(mixin.Mixin): def setUp(self): super(TusRequestTest, self).setUp() self.request = request.TusRequest(self.uploader) @parametrize( "filename", [FILEPATH_TEXT, FILEPATH_BINARY], ) def test_perform(self, filename: str): with open(FILEPATH_TEXT, "rb") as stream, responses.RequestsMock() as resps: size = stream.tell() resps.add(responses.PATCH, self.url, adding_headers={'upload-offset': str(size)}, status=204) self.request.perform() self.assertEqual(str(size), self.request.response_headers['upload-offset']) def test_perform_checksum(self): self.uploader.upload_checksum = True tus_request = request.TusRequest(self.uploader) with open(FILEPATH_TEXT, "rb") as stream, responses.RequestsMock() as resps: content = stream.read() expected_checksum = "sha1 " + \ base64.standard_b64encode(hashlib.sha1( content).digest()).decode("ascii") sent_checksum = '' def validate_headers(req): nonlocal sent_checksum sent_checksum = req.headers['upload-checksum'] return (204, {}, None) resps.add_callback(responses.PATCH, self.url, callback=validate_headers) tus_request.perform() self.assertEqual(sent_checksum, expected_checksum) def test_verify_tls_cert(self): self.uploader.verify_tls_cert = False tus_request = request.TusRequest(self.uploader) with responses.RequestsMock() as resps: verify = None def validate_verify(req): nonlocal verify verify = req.req_kwargs['verify'] return (204, {}, None) resps.add_callback(responses.PATCH, self.url, callback=validate_verify) tus_request.perform() self.assertEqual(verify, False) ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472593.0 tuspy-1.0.3/tests/test_uploader.py0000644000076500000240000001722514536325621016526 0ustar00mariusstaffimport os import io from base64 import b64encode from unittest import mock import responses from parametrize import parametrize import pytest from tusclient import exceptions from tusclient.storage import filestorage from tests import mixin FILEPATH_TEXT = "tests/sample_files/text.txt" FILEPATH_BINARY = "tests/sample_files/binary.png" class UploaderTest(mixin.Mixin): def mock_request(self, request_mock): request_mock = request_mock.return_value request_mock.status_code = 204 request_mock.response_headers = { 'upload-offset': self.uploader.offset + self.uploader.get_request_length()} request_mock.perform.return_value = None return request_mock def test_instance_attributes(self): self.assertEqual(self.uploader.chunk_size, self.uploader.DEFAULT_CHUNK_SIZE) self.assertEqual(self.uploader.client, self.client) self.assertEqual(self.uploader.offset, 0) def test_headers(self): self.assertEqual(self.uploader.get_headers(), {"Tus-Resumable": "1.0.0"}) self.client.set_headers({'foo': 'bar'}) self.assertEqual(self.uploader.get_headers(), {"Tus-Resumable": "1.0.0", 'foo': 'bar'}) @responses.activate def test_get_offset(self): responses.add(responses.HEAD, self.uploader.url, adding_headers={"upload-offset": "300"}) self.assertEqual(self.uploader.get_offset(), 300) def test_encode_metadata(self): self.uploader.metadata = {'foo': 'bar', 'red': 'blue'} encoded_metadata = ['foo' + ' ' + b64encode(b'bar').decode('ascii'), 'red' + ' ' + b64encode(b'blue').decode('ascii')] self.assertCountEqual(self.uploader.encode_metadata(), encoded_metadata) with pytest.raises(ValueError): self.uploader.metadata = {'foo, ': 'bar'} self.uploader.encode_metadata() def test_encode_metadata_utf8(self): self.uploader.metadata = {'foo': 'bär', 'red': '🔵'} self.uploader.metadata_encoding = 'utf-8' encoded_metadata = [ 'foo ' + b64encode('bär'.encode('utf-8')).decode('ascii'), 'red ' + b64encode('🔵'.encode('utf-8')).decode('ascii') ] self.assertCountEqual(self.uploader.encode_metadata(), encoded_metadata) @responses.activate def test_create_url_absolute(self): responses.add(responses.POST, self.client.url, adding_headers={"location": 'http://tusd.tusdemo.net/files/foo'}) self.assertEqual(self.uploader.create_url(), 'http://tusd.tusdemo.net/files/foo') @responses.activate def test_create_url_relative(self): responses.add(responses.POST, self.client.url, adding_headers={"location": "/files/foo"}) self.assertEqual(self.uploader.create_url(), 'http://tusd.tusdemo.net/files/foo') @parametrize( "filename", [FILEPATH_TEXT, FILEPATH_BINARY], ) @responses.activate def test_url(self, filename: str): # test for stored urls responses.add(responses.HEAD, 'http://tusd.tusdemo.net/files/foo_bar', adding_headers={"upload-offset": "10"}) storage_path = '{}/storage_file'.format(os.path.dirname(os.path.abspath(__file__))) resumable_uploader = self.client.uploader( file_path=filename, store_url=True, url_storage=filestorage.FileStorage(storage_path) ) self.assertEqual(resumable_uploader.url, "http://tusd.tusdemo.net/files/foo_bar") self.assertEqual(resumable_uploader.offset, 10) def test_request_length(self): self.uploader.chunk_size = 200 self.assertEqual(self.uploader.get_request_length(), 200) self.uploader.chunk_size = self.uploader.get_file_size() + 3000 self.assertEqual(self.uploader.get_request_length(), self.uploader.get_file_size()) @parametrize( "filename", [FILEPATH_TEXT, FILEPATH_BINARY], ) def test_get_file_stream(self, filename: str): with open(filename, "rb") as fs: self.uploader.file_stream = fs self.uploader.file_path = None self.assertEqual(self.uploader.file_stream, self.uploader.get_file_stream()) with open(filename, "rb") as fs: self.uploader.file_stream = None self.uploader.file_path = filename with self.uploader.get_file_stream() as stream: self.assertEqual(fs.read(), stream.read()) @parametrize( "filename", [FILEPATH_TEXT, FILEPATH_BINARY], ) def test_file_size(self, filename: str): self.assertEqual(self.uploader.get_file_size(), os.path.getsize(self.uploader.file_path)) with open(filename, "rb") as fs: self.uploader.file_stream = fs self.uploader.file_path = None self.assertEqual(self.uploader.get_file_size(), os.path.getsize(filename)) @mock.patch('tusclient.uploader.uploader.TusRequest') def test_upload_chunk(self, request_mock): self.mock_request(request_mock) self.uploader.offset = 0 request_length = self.uploader.get_request_length() self.uploader.upload_chunk() self.assertEqual(self.uploader.offset, request_length) @responses.activate def test_upload_chunk_with_creation(self): responses.add( responses.POST, self.client.url, adding_headers={ "location": f"{self.client.url}hello" } ) responses.add( responses.PATCH, f"{self.client.url}hello", adding_headers={ "upload-offset": "5" } ) uploader = self.client.uploader( file_stream=io.BytesIO(b"hello") ) uploader.upload_chunk() self.assertEqual(uploader.url, f"{self.client.url}hello") @mock.patch('tusclient.uploader.uploader.TusRequest') def test_upload(self, request_mock): self.mock_request(request_mock) self.uploader.upload() self.assertEqual(self.uploader.offset, self.uploader.get_file_size()) @mock.patch('tusclient.uploader.uploader.TusRequest') def test_upload_retry(self, request_mock): num_of_retries = 3 self.uploader.retries = num_of_retries self.uploader.retry_delay = 3 request_mock = self.mock_request(request_mock) request_mock.status_code = 00 self.assertEqual(self.uploader._retried, 0) with pytest.raises(exceptions.TusCommunicationError): self.uploader.upload_chunk() self.assertEqual(self.uploader._retried, num_of_retries) @responses.activate def test_upload_empty(self): responses.add( responses.POST, self.client.url, adding_headers={ "upload-offset": "0", "location": f"{self.client.url}this-is-not-used" } ) responses.add( responses.PATCH, f"{self.client.url}this-is-not-used", body=ValueError("PATCH request not allowed for empty file") ) # Upload an empty file uploader = self.client.uploader( file_stream=io.BytesIO(b"") ) uploader.upload() # Upload URL being set means the POST request was sent and the empty # file was uploaded without a single PATCH request. self.assertTrue(uploader.url) @mock.patch('tusclient.uploader.uploader.TusRequest') def test_upload_checksum(self, request_mock): self.mock_request(request_mock) self.uploader.upload_checksum = True self.uploader.upload() self.assertEqual(self.uploader.offset, self.uploader.get_file_size()) ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1702472767.971068 tuspy-1.0.3/tusclient/0000755000076500000240000000000014536326100014134 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472631.0 tuspy-1.0.3/tusclient/__init__.py0000644000076500000240000000002614536325667016264 0ustar00mariusstaff__version__ = "1.0.3" ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469162.0 tuspy-1.0.3/tusclient/client.py0000644000076500000240000000366014536317052015777 0ustar00mariusstafffrom typing import Dict, Optional from tusclient.uploader import Uploader, AsyncUploader class TusClient: """ Object representation of Tus client. :Attributes: - url (str): represents the tus server's create extension url. On instantiation this argument must be passed to the constructor. - headers (dict): This can be used to set the server specific headers. These headers would be sent along with every request made by the cleint to the server. This may be used to set authentication headers. These headers should not include headers required by tus protocol. If not set this defaults to an empty dictionary. :Constructor Args: - url (str) - headers (Optiional[dict]) """ def __init__(self, url: str, headers: Optional[Dict[str, str]] = None): self.url = url self.headers = headers or {} def set_headers(self, headers: Dict[str, str]): """ Set tus client headers. Update and/or set new headers that would be sent along with every request made to the server. :Args: - headers (dict): key, value pairs of the headers to be set. This argument is required. """ self.headers.update(headers) def uploader(self, *args, **kwargs) -> Uploader: """ Return uploader instance pointing at current client instance. Return uploader instance with which you can control the upload of a specific file. The current instance of the tus client is passed to the uploader on creation. :Args: see tusclient.uploader.Uploader for required and optional arguments. """ kwargs["client"] = self return Uploader(*args, **kwargs) def async_uploader(self, *args, **kwargs) -> AsyncUploader: kwargs["client"] = self return AsyncUploader(*args, **kwargs) ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469162.0 tuspy-1.0.3/tusclient/exceptions.py0000644000076500000240000000210014536317052016666 0ustar00mariusstaff""" Global Tusclient exception and warning classes. """ class TusCommunicationError(Exception): """ Should be raised when communications with tus-server behaves unexpectedly. :Attributes: - message (str): Main message of the exception - status_code (int): Status code of response indicating an error - response_content (str): Content of response indicating an error :Constructor Args: - message (Optional[str]) - status_code (Optional[int]) - response_content (Optional[str]) """ def __init__(self, message, status_code=None, response_content=None): default_message = "Communication with tus server failed with status {}".format( status_code ) message = message or default_message super(TusCommunicationError, self).__init__(message) self.status_code = status_code self.response_content = response_content class TusUploadFailed(TusCommunicationError): """Should be raised when an attempted upload fails""" ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9714243 tuspy-1.0.3/tusclient/fingerprint/0000755000076500000240000000000014536326100016463 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1687250684.0 tuspy-1.0.3/tusclient/fingerprint/__init__.py0000644000076500000240000000000014444263374020575 0ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469162.0 tuspy-1.0.3/tusclient/fingerprint/fingerprint.py0000644000076500000240000000224314536317052021373 0ustar00mariusstaff""" An implementation of of , using the hashlib to generate an md5 hash based on the file content """ from typing import IO import hashlib import os from . import interface class Fingerprint(interface.Fingerprint): BLOCK_SIZE = 65536 def get_fingerprint(self, fs: IO): """ Return a unique fingerprint string value based on the file stream recevied :Args: - fs[IO]: The file stream instance of the file for which a fingerprint would be generated. :Returns: fingerprint[str] """ hasher = hashlib.md5() # we encode the content to avoid python 3 uncicode errors buf = self._encode_data(fs.read(self.BLOCK_SIZE)) hasher.update(buf) # add in the file size to minimize chances of collision fs.seek(0, os.SEEK_END) file_size = fs.tell() return "size:{}--md5:{}".format(file_size, hasher.hexdigest()) def _encode_data(self, data): try: return data.encode("utf-8") except AttributeError: # in case the content is already binary, this failure would happen. return data ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702466316.0 tuspy-1.0.3/tusclient/fingerprint/interface.py0000644000076500000240000000103614536311414020777 0ustar00mariusstaff""" Interface module defining a fingerprint generator based on file content. """ from typing import IO import abc class Fingerprint(abc.ABC): """An interface specifying the requirements of a file fingerprint""" @abc.abstractmethod def get_fingerprint(self, fs: IO): """ Return a unique fingerprint string value based on the file stream recevied :Args: - fs[IO]: The file stream instance of the file for which a fingerprint would be generated. :Returns: fingerprint[str] """ ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1701773049.0 tuspy-1.0.3/tusclient/py.typed0000644000076500000240000000000014533577371015640 0ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469162.0 tuspy-1.0.3/tusclient/request.py0000644000076500000240000000755114536317052016214 0ustar00mariusstafffrom typing import Optional import base64 import asyncio from functools import wraps import requests import aiohttp from tusclient.exceptions import TusUploadFailed, TusCommunicationError # Catches requests exceptions and throws custom tuspy errors. def catch_requests_error(func): """Deocrator to catch requests exceptions""" @wraps(func) def _wrapper(*args, **kwargs): try: return func(*args, **kwargs) except requests.exceptions.RequestException as error: raise TusCommunicationError(error) return _wrapper class BaseTusRequest: """ Http Request Abstraction. Sets up tus custom http request on instantiation. requires argument 'uploader' an instance of tusclient.uploader.Uploader on instantiation. :Attributes: - response_headers (dict) - file (file): The file that is being uploaded. """ def __init__(self, uploader): self._url = uploader.url self.response_headers = {} self.status_code = None self.response_content = None self.verify_tls_cert = bool(uploader.verify_tls_cert) self.file = uploader.get_file_stream() self.file.seek(uploader.offset) self._request_headers = { "upload-offset": str(uploader.offset), "Content-Type": "application/offset+octet-stream", } self._request_headers.update(uploader.get_headers()) self._content_length = uploader.get_request_length() self._upload_checksum = uploader.upload_checksum self._checksum_algorithm = uploader.checksum_algorithm self._checksum_algorithm_name = uploader.checksum_algorithm_name def add_checksum(self, chunk: bytes): if self._upload_checksum: self._request_headers["upload-checksum"] = " ".join( ( self._checksum_algorithm_name, base64.b64encode(self._checksum_algorithm(chunk).digest()).decode( "ascii" ), ) ) class TusRequest(BaseTusRequest): """Class to handle async Tus upload requests""" def perform(self): """ Perform actual request. """ try: chunk = self.file.read(self._content_length) self.add_checksum(chunk) resp = requests.patch( self._url, data=chunk, headers=self._request_headers, verify=self.verify_tls_cert, ) self.status_code = resp.status_code self.response_content = resp.content self.response_headers = {k.lower(): v for k, v in resp.headers.items()} except requests.exceptions.RequestException as error: raise TusUploadFailed(error) class AsyncTusRequest(BaseTusRequest): """Class to handle async Tus upload requests""" def __init__( self, *args, io_loop: Optional[asyncio.AbstractEventLoop] = None, **kwargs ): self.io_loop = io_loop super().__init__(*args, **kwargs) async def perform(self): """ Perform actual request. """ chunk = self.file.read(self._content_length) self.add_checksum(chunk) try: async with aiohttp.ClientSession(loop=self.io_loop) as session: ssl = None if self.verify_tls_cert else False async with session.patch( self._url, data=chunk, headers=self._request_headers, ssl=ssl ) as resp: self.status_code = resp.status self.response_headers = { k.lower(): v for k, v in resp.headers.items() } self.response_content = await resp.content.read() except aiohttp.ClientError as error: raise TusUploadFailed(error) ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9717393 tuspy-1.0.3/tusclient/storage/0000755000076500000240000000000014536326100015600 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1687250684.0 tuspy-1.0.3/tusclient/storage/__init__.py0000644000076500000240000000000014444263374017712 0ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469162.0 tuspy-1.0.3/tusclient/storage/filestorage.py0000644000076500000240000000262214536317052020466 0ustar00mariusstaff""" An implementation of , using a file as storage. """ from tinydb import TinyDB, Query from . import interface class FileStorage(interface.Storage): def __init__(self, fp): self._db = TinyDB(fp) self._urls = Query() def get_item(self, key: str): """ Return the tus url of a file, identified by the key specified. :Args: - key[str]: The unique id for the stored item (in this case, url) :Returns: url[str] """ result = self._db.search(self._urls.key == key) return result[0].get("url") if result else None def set_item(self, key: str, url: str): """ Store the url value under the unique key. :Args: - key[str]: The unique id to which the item (in this case, url) would be stored. - value[str]: The actual url value to be stored. """ if self._db.search(self._urls.key == key): self._db.update({"url": url}, self._urls.key == key) else: self._db.insert({"key": key, "url": url}) def remove_item(self, key: str): """ Remove/Delete the url value under the unique key from storage. """ self._db.remove(self._urls.key == key) def close(self): """ Close the file storage and release all opened files. """ self._db.close() ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702469036.0 tuspy-1.0.3/tusclient/storage/interface.py0000644000076500000240000000155514536316654020135 0ustar00mariusstaff""" Interface module defining a url storage API. """ import abc class Storage(object, metaclass=abc.ABCMeta): @abc.abstractmethod def get_item(self, key): """ Return the tus url of a file, identified by the key specified. :Args: - key[str]: The unique id for the stored item (in this case, url) :Returns: url[str] """ pass @abc.abstractmethod def set_item(self, key, value): """ Store the url value under the unique key. :Args: - key[str]: The unique id to which the item (in this case, url) would be stored. - value[str]: The actual url value to be stored. """ pass @abc.abstractmethod def remove_item(self, key): """ Remove/Delete the url value under the unique key from storage. """ pass ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9720669 tuspy-1.0.3/tusclient/uploader/0000755000076500000240000000000014536326100015747 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1687250684.0 tuspy-1.0.3/tusclient/uploader/__init__.py0000644000076500000240000000010014444263374020062 0ustar00mariusstafffrom tusclient.uploader.uploader import AsyncUploader, Uploader ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702471528.0 tuspy-1.0.3/tusclient/uploader/baseuploader.py0000644000076500000240000002517614536323550021010 0ustar00mariusstafffrom typing import Optional, IO, Dict, TYPE_CHECKING import os import re from base64 import b64encode from sys import maxsize as MAXSIZE import hashlib import requests from tusclient.exceptions import TusCommunicationError from tusclient.request import TusRequest, catch_requests_error from tusclient.fingerprint import fingerprint, interface from tusclient.storage.interface import Storage if TYPE_CHECKING: from tusclient.client import TusClient class BaseUploader: """ Object to control upload related functions. :Attributes: - file_path (str): This is the path(absolute/relative) to the file that is intended for upload to the tus server. On instantiation this attribute is required. - file_stream (file): As an alternative to the `file_path`, an instance of the file to be uploaded can be passed to the constructor as `file_stream`. Do note that either the `file_stream` or the `file_path` must be passed on instantiation. - url (str): If the upload url for the file is known, it can be passed to the constructor. This may happen when you resume an upload. - client (): An instance of `tusclient.client.TusClient`. This would tell the uploader instance what client it is operating with. Although this argument is optional, it is only optional if the 'url' argument is specified. - chunk_size (int): This tells the uploader what chunk size(in bytes) should be uploaded when the method `upload_chunk` is called. This defaults to the maximum possible integer if not specified. - metadata (dict): A dictionary containing the upload-metadata. This would be encoded internally by the method `encode_metadata` to conform with the tus protocol. - metadata_encoding (str): Encoding used for each upload-metadata value. This defaults to 'utf-8'. - offset (int): The offset value of the upload indicates the current position of the file upload. - stop_at (int): At what offset value the upload should stop. - request (): A http Request instance of the last chunk uploaded. - retries (int): The number of attempts the uploader should make in the case of a failed upload. If not specified, it defaults to 0. - retry_delay (int): How long (in seconds) the uploader should wait before retrying a failed upload attempt. If not specified, it defaults to 30. - verify_tls_cert (bool): Whether or not to verify the TLS certificate of the server. If not specified, it defaults to True. - store_url (bool): Determines whether or not url should be stored, and uploads should be resumed. - url_storage (): An implementation of which is an API for URL storage. This value must be set if store_url is set to true. A ready to use implementation exists atbe used out of the box. But you can implement your own custom storage API and pass an instace of it as value. - fingerprinter (): An implementation of which is an API to generate a unique fingerprint for the uploaded file. This is used for url storage when resumability is enabled. if store_url is set to true, the default fingerprint module () would be used. But you can set your own custom fingerprint module by passing it to the constructor. - upload_checksum (bool): Whether or not to supply the Upload-Checksum header along with each chunk. Defaults to False. :Constructor Args: - file_path (str) - file_stream (Optional[file]) - url (Optional[str]) - client (Optional []) - chunk_size (Optional[int]) - metadata (Optional[dict]) - metadata_encoding (Optional[str]) - retries (Optional[int]) - retry_delay (Optional[int]) - verify_tls_cert (Optional[bool]) - store_url (Optional[bool]) - url_storage (Optinal []) - fingerprinter (Optional []) - upload_checksum (Optional[bool]) """ DEFAULT_HEADERS = {"Tus-Resumable": "1.0.0"} DEFAULT_CHUNK_SIZE = MAXSIZE CHECKSUM_ALGORITHM_PAIR = ( "sha1", hashlib.sha1, ) def __init__( self, file_path: Optional[str] = None, file_stream: Optional[IO] = None, url: Optional[str] = None, client: Optional["TusClient"] = None, chunk_size: int = MAXSIZE, metadata: Optional[Dict] = None, metadata_encoding: Optional[str] = "utf-8", retries: int = 0, retry_delay: int = 30, verify_tls_cert: bool = True, store_url=False, url_storage: Optional[Storage] = None, fingerprinter: Optional[interface.Fingerprint] = None, upload_checksum=False, ): if file_path is None and file_stream is None: raise ValueError("Either 'file_path' or 'file_stream' cannot be None.") if url is None and client is None: raise ValueError("Either 'url' or 'client' cannot be None.") if store_url and url_storage is None: raise ValueError( "Please specify a storage instance to enable resumablility." ) self.verify_tls_cert = verify_tls_cert self.file_path = file_path self.file_stream = file_stream self.stop_at = self.get_file_size() self.client = client self.metadata = metadata or {} self.metadata_encoding = metadata_encoding self.store_url = store_url self.url_storage = url_storage self.fingerprinter = fingerprinter or fingerprint.Fingerprint() self.offset = 0 self.url = None self.__init_url_and_offset(url) self.chunk_size = chunk_size self.retries = retries self.request = None self._retried = 0 self.retry_delay = retry_delay self.upload_checksum = upload_checksum ( self.__checksum_algorithm_name, self.__checksum_algorithm, ) = self.CHECKSUM_ALGORITHM_PAIR def get_headers(self): """ Return headers of the uploader instance. This would include the headers of the client instance. """ client_headers = getattr(self.client, "headers", {}) return dict(self.DEFAULT_HEADERS, **client_headers) def get_url_creation_headers(self): """Return headers required to create upload url""" headers = self.get_headers() headers["upload-length"] = str(self.get_file_size()) headers["upload-metadata"] = ",".join(self.encode_metadata()) return headers @property def checksum_algorithm(self): """The checksum algorithm to be used for the Upload-Checksum extension.""" return self.__checksum_algorithm @property def checksum_algorithm_name(self): """The name of the checksum algorithm to be used for the Upload-Checksum extension. """ return self.__checksum_algorithm_name @catch_requests_error def get_offset(self): """ Return offset from tus server. This is different from the instance attribute 'offset' because this makes an http request to the tus server to retrieve the offset. """ resp = requests.head( self.url, headers=self.get_headers(), verify=self.verify_tls_cert ) offset = resp.headers.get("upload-offset") if offset is None: msg = "Attempt to retrieve offset fails with status {}".format( resp.status_code ) raise TusCommunicationError(msg, resp.status_code, resp.content) return int(offset) def encode_metadata(self): """ Return list of encoded metadata as defined by the Tus protocol. """ encoded_list = [] for key, value in self.metadata.items(): key_str = str(key) # dict keys may be of any object type. # confirm that the key does not contain unwanted characters. if re.search(r"^$|[\s,]+", key_str): msg = 'Upload-metadata key "{}" cannot be empty nor contain spaces or commas.' raise ValueError(msg.format(key_str)) value_bytes = value.encode(self.metadata_encoding) encoded_list.append( "{} {}".format(key_str, b64encode(value_bytes).decode("ascii")) ) return encoded_list def __init_url_and_offset(self, url: Optional[str] = None): """ Return the tus upload url. If resumability is enabled, this would try to get the url from storage if available, otherwise it would request a new upload url from the tus server. """ if url: self.set_url(url) if self.store_url and self.url_storage: key = self._get_fingerprint() self.set_url(self.url_storage.get_item(key)) if self.url: self.offset = self.get_offset() def _get_fingerprint(self): with self.get_file_stream() as stream: return self.fingerprinter.get_fingerprint(stream) def set_url(self, url: str): """Set the upload URL""" self.url = url if self.store_url and self.url_storage: key = self._get_fingerprint() self.url_storage.set_item(key, url) def get_request_length(self): """ Return length of next chunk upload. """ remainder = self.stop_at - self.offset return self.chunk_size if remainder > self.chunk_size else remainder def get_file_stream(self): """ Return a file stream instance of the upload. """ if self.file_stream: self.file_stream.seek(0) return self.file_stream elif os.path.isfile(self.file_path): return open(self.file_path, "rb") else: raise ValueError("invalid file {}".format(self.file_path)) def get_file_size(self): """ Return size of the file. """ stream = self.get_file_stream() stream.seek(0, os.SEEK_END) return stream.tell() ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472593.0 tuspy-1.0.3/tusclient/uploader/uploader.py0000644000076500000240000001417514536325621020153 0ustar00mariusstafffrom typing import Optional import time import asyncio from urllib.parse import urljoin import requests import aiohttp from tusclient.uploader.baseuploader import BaseUploader from tusclient.exceptions import TusUploadFailed, TusCommunicationError from tusclient.request import TusRequest, AsyncTusRequest, catch_requests_error def _verify_upload(request: TusRequest): if 200 <= request.status_code < 300: return True else: raise TusUploadFailed("", request.status_code, request.response_content) class Uploader(BaseUploader): def upload(self, stop_at: Optional[int] = None): """ Perform file upload. Performs continous upload of chunks of the file. The size uploaded at each cycle is the value of the attribute 'chunk_size'. :Args: - stop_at (Optional[int]): Determines at what offset value the upload should stop. If not specified this defaults to the file size. """ self.stop_at = stop_at or self.get_file_size() if not self.url: # Ensure the POST request is performed even for empty files. # This ensures even empty files can be uploaded; in this case # only the POST request needs to be performed. self.set_url(self.create_url()) self.offset = 0 while self.offset < self.stop_at: self.upload_chunk() def upload_chunk(self): """ Upload chunk of file. """ self._retried = 0 # Ensure that we have a URL, as this is behavior we allowed previously. # See https://github.com/tus/tus-py-client/issues/82. if not self.url: self.set_url(self.create_url()) self.offset = 0 self._do_request() self.offset = int(self.request.response_headers.get("upload-offset")) @catch_requests_error def create_url(self): """ Return upload url. Makes request to tus server to create a new upload url for the required file upload. """ resp = requests.post( self.client.url, headers=self.get_url_creation_headers(), verify=self.verify_tls_cert, ) url = resp.headers.get("location") if url is None: msg = "Attempt to retrieve create file url with status {}".format( resp.status_code ) raise TusCommunicationError(msg, resp.status_code, resp.content) return urljoin(self.client.url, url) def _do_request(self): self.request = TusRequest(self) try: self.request.perform() _verify_upload(self.request) except TusUploadFailed as error: self._retry_or_cry(error) def _retry_or_cry(self, error): if self.retries > self._retried: time.sleep(self.retry_delay) self._retried += 1 try: self.offset = self.get_offset() except TusCommunicationError as err: self._retry_or_cry(err) else: self._do_request() else: raise error class AsyncUploader(BaseUploader): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) async def upload(self, stop_at: Optional[int] = None): """ Perform file upload. Performs continous upload of chunks of the file. The size uploaded at each cycle is the value of the attribute 'chunk_size'. :Args: - stop_at (Optional[int]): Determines at what offset value the upload should stop. If not specified this defaults to the file size. """ self.stop_at = stop_at or self.get_file_size() if not self.url: self.set_url(await self.create_url()) self.offset = 0 while self.offset < self.stop_at: await self.upload_chunk() async def upload_chunk(self): """ Upload chunk of file. """ self._retried = 0 # Ensure that we have a URL, as this is behavior we allowed previously. # See https://github.com/tus/tus-py-client/issues/82. if not self.url: self.set_url(await self.create_url()) self.offset = 0 await self._do_request() self.offset = int(self.request.response_headers.get("upload-offset")) async def create_url(self): """ Return upload url. Makes request to tus server to create a new upload url for the required file upload. """ try: async with aiohttp.ClientSession() as session: headers = self.get_url_creation_headers() ssl = None if self.verify_tls_cert else False async with session.post( self.client.url, headers=headers, ssl=ssl ) as resp: url = resp.headers.get("location") if url is None: msg = ( "Attempt to retrieve create file url with status {}".format( resp.status ) ) raise TusCommunicationError( msg, resp.status, await resp.content.read() ) return urljoin(self.client.url, url) except aiohttp.ClientError as error: raise TusCommunicationError(error) async def _do_request(self): self.request = AsyncTusRequest(self) try: await self.request.perform() _verify_upload(self.request) except TusUploadFailed as error: await self._retry_or_cry(error) async def _retry_or_cry(self, error): if self.retries > self._retried: await asyncio.sleep(self.retry_delay) self._retried += 1 try: self.offset = self.get_offset() except TusCommunicationError as err: await self._retry_or_cry(err) else: await self._do_request() else: raise error ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1702472767.9730754 tuspy-1.0.3/tuspy.egg-info/0000755000076500000240000000000014536326100015000 5ustar00mariusstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472767.0 tuspy-1.0.3/tuspy.egg-info/PKG-INFO0000644000076500000240000001023114536326077016107 0ustar00mariusstaffMetadata-Version: 2.1 Name: tuspy Version: 1.0.3 Summary: A Python client for the tus resumable upload protocol -> http://tus.io Home-page: http://github.com/tus/tus-py-client/ Author: Ifedapo Olarewaju Author-email: ifedapoolarewaju@gmail.com License: MIT Platform: any Classifier: Programming Language :: Python Classifier: Natural Language :: English Classifier: Environment :: Web Environment Classifier: Intended Audience :: Developers Classifier: Development Status :: 3 - Alpha Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Internet :: File Transfer Protocol (FTP) Classifier: Topic :: Communications :: File Sharing Requires-Python: >=3.5.3 Description-Content-Type: text/markdown License-File: LICENSE License-File: AUTHORS Requires-Dist: requests>=2.18.4 Requires-Dist: tinydb>=3.5.0 Requires-Dist: aiohttp>=3.6.2 Provides-Extra: test Requires-Dist: responses>=0.5.1; extra == "test" Requires-Dist: aioresponses>=0.6.2; extra == "test" Requires-Dist: coverage>=4.2; extra == "test" Requires-Dist: pytest>=3.0.3; extra == "test" Requires-Dist: pytest-cov<2.6,>=2.3.1; extra == "test" Requires-Dist: parametrize>=0.1.1; extra == "test" Provides-Extra: dev Requires-Dist: tox>=2.3.1; extra == "dev" Requires-Dist: sphinx-autobuild==2021.3.14; extra == "dev" Requires-Dist: Sphinx==1.7.1; extra == "dev" # tus-py-client [![Build Status](https://github.com/tus/tus-py-client/actions/workflows/CI.yml/badge.svg)](https://github.com/tus/tus-py-client/actions/workflows/CI.yml) > **tus** is a protocol based on HTTP for _resumable file uploads_. Resumable > means that an upload can be interrupted at any moment and can be resumed without > re-uploading the previous data again. An interruption may happen willingly, if > the user wants to pause, or by accident in case of a network issue or server > outage. **tus-py-client** is a Python client for uploading files using the _tus_ protocol to any remote server supporting it. ## Documentation See documentation here: http://tus-py-client.readthedocs.io/en/latest/ ## Get started ```bash pip install tuspy ``` Now you are ready to use the api. ```python from tusclient import client # Set Authorization headers if it is required # by the tus server. my_client = client.TusClient('http://tusd.tusdemo.net/files/', headers={'Authorization': 'Basic xxyyZZAAbbCC='}) # Set more headers. my_client.set_headers({'HEADER_NAME': 'HEADER_VALUE'}) uploader = my_client.uploader('path/to/file.ext', chunk_size=200) # A file stream may also be passed in place of a file path. fs = open('path/to/file.ext', mode=) uploader = my_client.uploader(file_stream=fs, chunk_size=200) # Upload a chunk i.e 200 bytes. uploader.upload_chunk() # Uploads the entire file. # This uploads chunk by chunk. uploader.upload() # you could increase the chunk size to reduce the # number of upload_chunk cycles. uploader.chunk_size = 800 uploader.upload() # Continue uploading chunks till total chunks uploaded reaches 1000 bytes. uploader.upload(stop_at=1000) ``` If the upload url is known and the client headers are not required, uploaders can also be used standalone. ```python from tusclient.uploader import Uploader my_uploader = Uploader('path/to/file.ext', url='http://tusd.tusdemo.net/files/abcdef123456', chunk_size=200) ``` ## Development If you want to work on tus-py-client internally, follow these few steps: 1. Setup virtual environment and install dependencies ```bash python -m venv env/ source env/bin/activate pip install -e .[test] ``` 2. Running tests ```bash pytest ``` 3. Releasing a new version (see https://realpython.com/pypi-publish-python-package/) ```bash # Update version in tusclient/__init__.py vim tusclient/__init__.py # Update changelogs vim CHANGELOG.md pytest # Commit and tag git commit -m 'v1.2.3' git tag v1.2.3 # Build and release pip install build twine python -m build twine check dist/* twine upload dist/* # Then: make release on GitHub ``` ## License MIT ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472767.0 tuspy-1.0.3/tuspy.egg-info/SOURCES.txt0000644000076500000240000000134414536326077016703 0ustar00mariusstaffAUTHORS LICENSE MANIFEST.in README.md setup.py tests/test_async_uploader.py tests/test_client.py tests/test_filestorage.py tests/test_fingerprint.py tests/test_request.py tests/test_uploader.py tusclient/__init__.py tusclient/client.py tusclient/exceptions.py tusclient/py.typed tusclient/request.py tusclient/fingerprint/__init__.py tusclient/fingerprint/fingerprint.py tusclient/fingerprint/interface.py tusclient/storage/__init__.py tusclient/storage/filestorage.py tusclient/storage/interface.py tusclient/uploader/__init__.py tusclient/uploader/baseuploader.py tusclient/uploader/uploader.py tuspy.egg-info/PKG-INFO tuspy.egg-info/SOURCES.txt tuspy.egg-info/dependency_links.txt tuspy.egg-info/requires.txt tuspy.egg-info/top_level.txt././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472767.0 tuspy-1.0.3/tuspy.egg-info/dependency_links.txt0000644000076500000240000000000114536326077021063 0ustar00mariusstaff ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472767.0 tuspy-1.0.3/tuspy.egg-info/requires.txt0000644000076500000240000000033514536326077017416 0ustar00mariusstaffrequests>=2.18.4 tinydb>=3.5.0 aiohttp>=3.6.2 [dev] tox>=2.3.1 sphinx-autobuild==2021.3.14 Sphinx==1.7.1 [test] responses>=0.5.1 aioresponses>=0.6.2 coverage>=4.2 pytest>=3.0.3 pytest-cov<2.6,>=2.3.1 parametrize>=0.1.1 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1702472767.0 tuspy-1.0.3/tuspy.egg-info/top_level.txt0000644000076500000240000000001214536326077017540 0ustar00mariusstafftusclient