././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5533824 priority-2.0.0/0000755000076500000240000000000000000000000012674 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624788090.0 priority-2.0.0/CHANGELOG.rst0000644000076500000240000000536400000000000014725 0ustar00kriechistaffChangelog ========= 2.0.0 (2021-06-27) ------------------ **API Changes** - Python 3.6 is the minimal support Python version. - Support for Python 3.7 has been added. - Support for Python 3.8 has been added. - Support for Python 3.9 has been added. - Support for Python 2.7 has been removed. - Support for Python 3.3 has been removed. - Support for Python 3.4 has been removed. - Support for Python 3.5 has been removed. - Support for PyPy (Python 2.7 compatible) has been removed. - Add type hints throughout and support PEP 561 via a py.typed file. This should allow projects to type check their usage of this dependency. - Throw ``TypeError`` when creating a priority tree with a ``maximum_streams`` value that is not an integer. - Throw ``ValueError`` when creating a priority tree with a ``maximum_streams`` value that is not a positive integer. 1.3.0 (2017-01-27) ------------------ **API Changes** - Throw ``PriorityLoop`` when inserting or reprioritising a stream that depends on itself. - Throw ``BadWeightError`` when creating or reprioritising a stream with a weight that is not an integer between 1 and 256, inclusive. - Throw ``PseudoStreamError`` when trying to reprioritise, remove, block or unblock stream 0. - Add a new ``PriorityError`` parent class for the exceptions that can be thrown by priority. 1.2.2 (2016-11-11) ------------------ **Bugfixes** - Allow ``insert_stream`` to be called with ``exclusive=True`` but no explicit ``depends_on`` value. 1.2.1 (2016-10-26) ------------------ **Bugfixes** - Allow insertion of streams that have parents in the idle or closed states. This would previously raise a KeyError. 1.2.0 (2016-08-04) ------------------ **Security Fixes** - CVE-2016-6580: All versions of this library prior to 1.2.0 are vulnerable to a denial of service attack whereby a remote peer can cause a user to insert an unbounded number of streams into the priority tree, eventually consuming all available memory. This version adds a ``TooManyStreamsError`` exception that is raised when too many streams are inserted into the priority tree. It also adds a keyword argument to the priority tree, ``maximum_streams``, which limits how many streams may be inserted. By default, this number is set to 1000. Implementations should strongly consider whether they can set this value lower. 1.1.1 (2016-05-28) ------------------ **Bugfixes** - 2.5x performance improvement by swapping from ``queue.PriorityQueue`` to ``heapq``. 1.1.0 (2016-01-08) ------------------ **API Changes** - Throw ``DuplicateStreamError`` when inserting a stream that is already in the tree. - Throw ``MissingStreamError`` when reprioritising a stream that is not in the tree. 1.0.0 (2015-12-07) ------------------ - Initial release. ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/CONTRIBUTORS.rst0000644000076500000240000000031200000000000015357 0ustar00kriechistaffPriority is written and maintained by Cory Benfield and various contributors: Development Lead ```````````````` - Cory Benfield Contributors ```````````` In chronological order: ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/LICENSE0000644000076500000240000000204000000000000013675 0ustar00kriechistaffCopyright (c) 2015 Cory Benfield Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624309761.0 priority-2.0.0/MANIFEST.in0000644000076500000240000000036100000000000014432 0ustar00kriechistaffgraft src/priority graft docs graft test graft visualizer graft examples prune docs/build recursive-include *.py include README.rst LICENSE CHANGELOG.rst CONTRIBUTORS.rst tox.ini global-exclude *.pyc *.pyo *.swo *.swp *.map *.yml *.DS_Store ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1624788894.553576 priority-2.0.0/PKG-INFO0000644000076500000240000001476200000000000014003 0ustar00kriechistaffMetadata-Version: 2.1 Name: priority Version: 2.0.0 Summary: A pure-Python implementation of the HTTP/2 priority tree Home-page: https://github.com/python-hyper/priority/ Author: Cory Benfield Author-email: cory@lukasa.co.uk License: MIT License Project-URL: Documentation, https://python-hyper.org/projects/priority/ Project-URL: Source, https://github.com/python-hyper/priority/ Project-URL: Tracker, https://github.com/python-hyper/priority/issues Project-URL: Changelog, https://github.com/python-hyper/priority/blob/master/HISTORY.rst Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Requires-Python: >=3.6.1 Description-Content-Type: text/x-rst License-File: LICENSE ========================================== Priority: A HTTP/2 Priority Implementation ========================================== .. image:: https://github.com/python-hyper/priority/workflows/CI/badge.svg :target: https://github.com/python-hyper/priority/actions :alt: Build Status .. image:: https://codecov.io/gh/python-hyper/priority/branch/master/graph/badge.svg :target: https://codecov.io/gh/python-hyper/priority :alt: Code Coverage .. image:: https://readthedocs.org/projects/priority/badge/?version=latest :target: https://priority.readthedocs.io/en/latest/ :alt: Documentation Status .. image:: https://img.shields.io/badge/chat-join_now-brightgreen.svg :target: https://gitter.im/python-hyper/community :alt: Chat community .. image:: https://raw.github.com/python-hyper/documentation/master/source/logo/hyper-black-bg-white.png Priority is a pure-Python implementation of the priority logic for HTTP/2, set out in `RFC 7540 Section 5.3 (Stream Priority)`_. This logic allows for clients to express a preference for how the server allocates its (limited) resources to the many outstanding HTTP requests that may be running over a single HTTP/2 connection. Specifically, this Python implementation uses a variant of the implementation used in the excellent `H2O`_ project. This original implementation is also the inspiration for `nghttp2's`_ priority implementation, and generally produces a very clean and even priority stream. The only notable changes from H2O's implementation are small modifications to allow the priority implementation to work cleanly as a separate implementation, rather than being embedded in a HTTP/2 stack directly. While priority information in HTTP/2 is only a suggestion, rather than an enforceable constraint, where possible servers should respect the priority requests of their clients. Using Priority -------------- Priority has a simple API. Streams are inserted into the tree: when they are inserted, they may optionally have a weight, depend on another stream, or become an exclusive dependent of another stream. .. code-block:: python >>> p = priority.PriorityTree() >>> p.insert_stream(stream_id=1) >>> p.insert_stream(stream_id=3) >>> p.insert_stream(stream_id=5, depends_on=1) >>> p.insert_stream(stream_id=7, weight=32) >>> p.insert_stream(stream_id=9, depends_on=7, weight=8) >>> p.insert_stream(stream_id=11, depends_on=7, exclusive=True) Once streams are inserted, the stream priorities can be requested. This allows the server to make decisions about how to allocate resources. Iterating The Tree ~~~~~~~~~~~~~~~~~~ The tree in this algorithm acts as a gate. Its goal is to allow one stream "through" at a time, in such a manner that all the active streams are served as evenly as possible in proportion to their weights. This is handled in Priority by iterating over the tree. The tree itself is an iterator, and each time it is advanced it will yield a stream ID. This is the ID of the stream that should next send data. This looks like this: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) If each stream only sends when it is 'ungated' by this mechanism, the server will automatically be emitting stream data in conformance to RFC 7540. Updating The Tree ~~~~~~~~~~~~~~~~~ If for any reason a stream is unable to proceed (for example, it is blocked on HTTP/2 flow control, or it is waiting for more data from another service), that stream is *blocked*. The ``PriorityTree`` should be informed that the stream is blocked so that other dependent streams get a chance to proceed. This can be done by calling the ``block`` method of the tree with the stream ID that is currently unable to proceed. This will automatically update the tree, and it will adjust on the fly to correctly allow any streams that were dependent on the blocked one to progress. For example: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) ... if blocked(stream_id): ... p.block(stream_id) When a stream goes from being blocked to being unblocked, call the ``unblock`` method to place it back into the sequence. Both the ``block`` and ``unblock`` methods are idempotent and safe to call repeatedly. Additionally, the priority of a stream may change. When it does, the ``reprioritize`` method can be used to update the tree in the wake of that change. ``reprioritize`` has the same signature as ``insert_stream``, but applies only to streams already in the tree. Removing Streams ~~~~~~~~~~~~~~~~ A stream can be entirely removed from the tree by calling ``remove_stream``. Note that this is not idempotent. Further, calling ``remove_stream`` and then re-adding it *may* cause a substantial change in the shape of the priority tree, and *will* cause the iteration order to change. License ------- Priority is made available under the MIT License. For more details, see the LICENSE file in the repository. Authors ------- Priority is maintained by Cory Benfield, with contributions from others. For more details about the contributors, please see CONTRIBUTORS.rst in the repository. .. _RFC 7540 Section 5.3 (Stream Priority): https://tools.ietf.org/html/rfc7540#section-5.3 .. _nghttp2's: https://nghttp2.org/blog/2015/11/11/stream-scheduling-utilizing-http2-priority/ .. _H2O: https://h2o.examp1e.net/ ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624136383.0 priority-2.0.0/README.rst0000644000076500000240000001244600000000000014372 0ustar00kriechistaff========================================== Priority: A HTTP/2 Priority Implementation ========================================== .. image:: https://github.com/python-hyper/priority/workflows/CI/badge.svg :target: https://github.com/python-hyper/priority/actions :alt: Build Status .. image:: https://codecov.io/gh/python-hyper/priority/branch/master/graph/badge.svg :target: https://codecov.io/gh/python-hyper/priority :alt: Code Coverage .. image:: https://readthedocs.org/projects/priority/badge/?version=latest :target: https://priority.readthedocs.io/en/latest/ :alt: Documentation Status .. image:: https://img.shields.io/badge/chat-join_now-brightgreen.svg :target: https://gitter.im/python-hyper/community :alt: Chat community .. image:: https://raw.github.com/python-hyper/documentation/master/source/logo/hyper-black-bg-white.png Priority is a pure-Python implementation of the priority logic for HTTP/2, set out in `RFC 7540 Section 5.3 (Stream Priority)`_. This logic allows for clients to express a preference for how the server allocates its (limited) resources to the many outstanding HTTP requests that may be running over a single HTTP/2 connection. Specifically, this Python implementation uses a variant of the implementation used in the excellent `H2O`_ project. This original implementation is also the inspiration for `nghttp2's`_ priority implementation, and generally produces a very clean and even priority stream. The only notable changes from H2O's implementation are small modifications to allow the priority implementation to work cleanly as a separate implementation, rather than being embedded in a HTTP/2 stack directly. While priority information in HTTP/2 is only a suggestion, rather than an enforceable constraint, where possible servers should respect the priority requests of their clients. Using Priority -------------- Priority has a simple API. Streams are inserted into the tree: when they are inserted, they may optionally have a weight, depend on another stream, or become an exclusive dependent of another stream. .. code-block:: python >>> p = priority.PriorityTree() >>> p.insert_stream(stream_id=1) >>> p.insert_stream(stream_id=3) >>> p.insert_stream(stream_id=5, depends_on=1) >>> p.insert_stream(stream_id=7, weight=32) >>> p.insert_stream(stream_id=9, depends_on=7, weight=8) >>> p.insert_stream(stream_id=11, depends_on=7, exclusive=True) Once streams are inserted, the stream priorities can be requested. This allows the server to make decisions about how to allocate resources. Iterating The Tree ~~~~~~~~~~~~~~~~~~ The tree in this algorithm acts as a gate. Its goal is to allow one stream "through" at a time, in such a manner that all the active streams are served as evenly as possible in proportion to their weights. This is handled in Priority by iterating over the tree. The tree itself is an iterator, and each time it is advanced it will yield a stream ID. This is the ID of the stream that should next send data. This looks like this: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) If each stream only sends when it is 'ungated' by this mechanism, the server will automatically be emitting stream data in conformance to RFC 7540. Updating The Tree ~~~~~~~~~~~~~~~~~ If for any reason a stream is unable to proceed (for example, it is blocked on HTTP/2 flow control, or it is waiting for more data from another service), that stream is *blocked*. The ``PriorityTree`` should be informed that the stream is blocked so that other dependent streams get a chance to proceed. This can be done by calling the ``block`` method of the tree with the stream ID that is currently unable to proceed. This will automatically update the tree, and it will adjust on the fly to correctly allow any streams that were dependent on the blocked one to progress. For example: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) ... if blocked(stream_id): ... p.block(stream_id) When a stream goes from being blocked to being unblocked, call the ``unblock`` method to place it back into the sequence. Both the ``block`` and ``unblock`` methods are idempotent and safe to call repeatedly. Additionally, the priority of a stream may change. When it does, the ``reprioritize`` method can be used to update the tree in the wake of that change. ``reprioritize`` has the same signature as ``insert_stream``, but applies only to streams already in the tree. Removing Streams ~~~~~~~~~~~~~~~~ A stream can be entirely removed from the tree by calling ``remove_stream``. Note that this is not idempotent. Further, calling ``remove_stream`` and then re-adding it *may* cause a substantial change in the shape of the priority tree, and *will* cause the iteration order to change. License ------- Priority is made available under the MIT License. For more details, see the LICENSE file in the repository. Authors ------- Priority is maintained by Cory Benfield, with contributions from others. For more details about the contributors, please see CONTRIBUTORS.rst in the repository. .. _RFC 7540 Section 5.3 (Stream Priority): https://tools.ietf.org/html/rfc7540#section-5.3 .. _nghttp2's: https://nghttp2.org/blog/2015/11/11/stream-scheduling-utilizing-http2-priority/ .. _H2O: https://h2o.examp1e.net/ ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5448263 priority-2.0.0/docs/0000755000076500000240000000000000000000000013624 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/Makefile0000644000076500000240000001517300000000000015273 0ustar00kriechistaff# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/priority.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/priority.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/priority" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/priority" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/make.bat0000644000076500000240000001507200000000000015236 0ustar00kriechistaff@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% source set I18NSPHINXOPTS=%SPHINXOPTS% source if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. xml to make Docutils-native XML files echo. pseudoxml to make pseudoxml-XML files for display purposes echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) %SPHINXBUILD% 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\priority.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\priority.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdf" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdfja" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf-ja cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) if "%1" == "xml" ( %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml if errorlevel 1 exit /b 1 echo. echo.Build finished. The XML files are in %BUILDDIR%/xml. goto end ) if "%1" == "pseudoxml" ( %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml if errorlevel 1 exit /b 1 echo. echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. goto end ) :end ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5481043 priority-2.0.0/docs/source/0000755000076500000240000000000000000000000015124 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/api.rst0000644000076500000240000000071700000000000016434 0ustar00kriechistaffPriority API ============ Priority Tree ------------- .. autoclass:: priority.PriorityTree :members: Exceptions ---------- .. autoclass:: priority.PriorityError .. autoclass:: priority.DeadlockError .. autoclass:: priority.PriorityLoop .. autoclass:: priority.DuplicateStreamError .. autoclass:: priority.MissingStreamError .. autoclass:: priority.TooManyStreamsError .. autoclass:: priority.BadWeightError .. autoclass:: priority.PseudoStreamError ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/authors.rst0000644000076500000240000000007600000000000017346 0ustar00kriechistaffContributors ============ .. include:: ../../CONTRIBUTORS.rst././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624136383.0 priority-2.0.0/docs/source/conf.py0000644000076500000240000000445000000000000016426 0ustar00kriechistaff# Configuration file for the Sphinx documentation builder. # # This file only contains a selection of the most common options. For a full # list see the documentation: # https://www.sphinx-doc.org/en/master/usage/configuration.html # -- Path setup -------------------------------------------------------------- # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # import os import sys import re sys.path.insert(0, os.path.abspath('../..')) PROJECT_ROOT = os.path.dirname(__file__) # Get the version version_regex = r'__version__ = ["\']([^"\']*)["\']' with open(os.path.join(PROJECT_ROOT, '../../', 'src/priority/__init__.py')) as file_: text = file_.read() match = re.search(version_regex, text) version = match.group(1) # -- Project information ----------------------------------------------------- project = 'priority' copyright = '2021, Cory Benfield' author = 'Cory Benfield' release = version # -- General configuration ------------------------------------------------ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.viewcode', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The master toctree document. master_doc = 'index' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [] # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = {'http://docs.python.org/': None} # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'default' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". # html_static_path = ['_static'] ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/index.rst0000644000076500000240000000236700000000000016775 0ustar00kriechistaffPriority: A pure-Python HTTP/2 Priority implementation ====================================================== Priority is a pure-Python implementation of the priority logic for HTTP/2, set out in `RFC 7540 Section 5.3 (Stream Priority)`_. This logic allows for clients to express a preference for how the server allocates its (limited) resources to the many outstanding HTTP requests that may be running over a single HTTP/2 connection. Specifically, this Python implementation uses a variant of the implementation used in the excellent `H2O`_ project. This original implementation is also the inspiration for `nghttp2's`_ priority implementation, and generally produces a very clean and even priority stream. The only notable changes from H2O's implementation are small modifications to allow the priority implementation to work cleanly as a separate implementation, rather than being embedded in a HTTP/2 stack directly. Contents: .. toctree:: :maxdepth: 2 installation using-priority api security/index license authors .. _RFC 7540 Section 5.3 (Stream Priority): https://tools.ietf.org/html/rfc7540#section-5.3 .. _nghttp2's: https://nghttp2.org/blog/2015/11/11/stream-scheduling-utilizing-http2-priority/ .. _H2O: https://h2o.examp1e.net/ ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/installation.rst0000644000076500000240000000032200000000000020354 0ustar00kriechistaffInstallation ============ Priority is a pure-Python project. This means installing it is extremely simple. To get the latest release from PyPI, simply run: .. code-block:: console $ pip install priority ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/license.rst0000644000076500000240000000005400000000000017277 0ustar00kriechistaffLicense ======= .. include:: ../../LICENSE ././@PaxHeader0000000000000000000000000000003200000000000010210 xustar0026 mtime=1624788894.54903 priority-2.0.0/docs/source/security/0000755000076500000240000000000000000000000016773 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/security/CVE-2016-6580.rst0000644000076500000240000000404600000000000021134 0ustar00kriechistaff:orphan: DoS via Unlimited Stream Insertion ================================== Hyper Project security advisory, August 4th 2016. Vulnerability ------------- A HTTP/2 implementation built using the priority library could be targetted by a malicious peer by having that peer assign priority information for every possible HTTP/2 stream ID. The priority tree would happily continue to store the priority information for each stream, and would therefore allocate unbounded amounts of memory. Attempting to actually *use* a tree like this would also cause extremely high CPU usage to maintain the tree. We are not aware of any active exploits of this vulnerability, but as this class of attack was publicly described in `this report`_, users should assume that they are at imminent risk of this kind of attack. Info ---- This issue has been given the name CVE-2016-6580. Affected Versions ----------------- This issue affects all versions of the priority library prior to 1.2.0. The Solution ------------ In version 1.2.0, the priority library limits the maximum number of streams that can be inserted into the tree. By default this limit is 1000, but it is user-configurable. If it is necessary to backport a patch, the patch can be found in `this GitHub pull request`_. Recommendations --------------- We suggest you take the following actions immediately, in order of preference: 1. Update priority to 1.2.0 immediately, and consider revising the maximum number of streams downward to a suitable value for your application. 2. Backport the patch made available on GitHub. 3. Manually enforce a limit on the number of priority settings you'll allow at once. Timeline -------- This class of vulnerability was publicly reported in `this report`_ on the 3rd of August. We requested a CVE ID from Mitre the same day. Priority 1.2.0 was released on the 4th of August, at the same time as the publication of this advisory. .. _this report: http://www.imperva.com/docs/Imperva_HII_HTTP2.pdf .. _this GitHub pull request: https://github.com/python-hyper/priority/pull/23 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/security/index.rst0000644000076500000240000000207700000000000020642 0ustar00kriechistaffVulnerability Notifications =========================== This section of the page contains all known vulnerabilities in the priority library. These vulnerabilities have all been reported to us via our `vulnerability disclosure policy`_. Known Vulnerabilities --------------------- +----+---------------------------+----------------+---------------+--------------+---------------+ | \# | Vulnerability | Date Announced | First Version | Last Version | CVE | +====+===========================+================+===============+==============+===============+ | 1 | :doc:`DoS via unlimited | 2016-08-04 | 1.0.0 | 1.1.1 | CVE-2016-6580 | | | stream insertion. | | | | | | | ` | | | | | +----+---------------------------+----------------+---------------+--------------+---------------+ .. _vulnerability disclosure policy: http://python-hyper.org/en/latest/security.html#vulnerability-disclosure ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1618595124.0 priority-2.0.0/docs/source/using-priority.rst0000644000076500000240000000663500000000000020674 0ustar00kriechistaffUsing Priority ============== Priority has a simple API. Streams are inserted into the tree: when they are inserted, they may optionally have a weight, depend on another stream, or become an exclusive dependent of another stream. To manipulate the tree, we use a :class:`PriorityTree ` object. .. code-block:: python >>> p = priority.PriorityTree() >>> p.insert_stream(stream_id=1) >>> p.insert_stream(stream_id=3) >>> p.insert_stream(stream_id=5, depends_on=1) >>> p.insert_stream(stream_id=7, weight=32) >>> p.insert_stream(stream_id=9, depends_on=7, weight=8) >>> p.insert_stream(stream_id=11, depends_on=7, exclusive=True) Once streams are inserted, the stream priorities can be requested. This allows the server to make decisions about how to allocate resources. Iterating The Tree ------------------ The tree in this algorithm acts as a gate. Its goal is to allow one stream "through" at a time, in such a manner that all the active streams are served as evenly as possible in proportion to their weights. This is handled in Priority by iterating over the tree. The tree itself is an iterator, and each time it is advanced it will yield a stream ID. This is the ID of the stream that should next send data. This looks like this: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) If each stream only sends when it is 'ungated' by this mechanism, the server will automatically be emitting stream data in conformance to RFC 7540. Updating The Tree ----------------- If for any reason a stream is unable to proceed (for example, it is blocked on HTTP/2 flow control, or it is waiting for more data from another service), that stream is *blocked*. The :class:`PriorityTree ` should be informed that the stream is blocked so that other dependent streams get a chance to proceed. This can be done by calling the :meth:`block ` method of the tree with the stream ID that is currently unable to proceed. This will automatically update the tree, and it will adjust on the fly to correctly allow any streams that were dependent on the blocked one to progress. For example: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) ... if blocked(stream_id): ... p.block(stream_id) When a stream goes from being blocked to being unblocked, call the :meth:`unblock ` method to place it back into the sequence. Both the :meth:`block ` and :meth:`unblock ` methods are idempotent and safe to call repeatedly. Additionally, the priority of a stream may change. When it does, the :meth:`reprioritize ` method can be used to update the tree in the wake of that change. :meth:`reprioritize ` has the same signature as :meth:`insert_stream `, but applies only to streams already in the tree. Removing Streams ---------------- A stream can be entirely removed from the tree by calling :meth:`remove_stream `. Note that this is *not* idempotent. Further, calling :meth:`remove_stream ` and then re-adding it *may* cause a substantial change in the shape of the priority tree, and *will* cause the iteration order to change. ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5545695 priority-2.0.0/setup.cfg0000644000076500000240000000115100000000000014513 0ustar00kriechistaff[tool:pytest] testpaths = test [coverage:run] branch = True source = priority [coverage:report] fail_under = 100 show_missing = True exclude_lines = pragma: no cover assert False, "Should not be reachable" .*:.* # Python \d.* .*:.* # Platform-specific: [coverage:paths] source = src .tox/*/site-packages [flake8] max-line-length = 120 max-complexity = 10 [mypy] strict = true warn_unused_configs = true show_error_codes = true [mypy-test_priority] allow_untyped_defs = true check_untyped_defs = false ignore_missing_imports = true disallow_subclassing_any = false [egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624136769.0 priority-2.0.0/setup.py0000644000076500000240000000365300000000000014415 0ustar00kriechistaff#!/usr/bin/env python3 import os import re from setuptools import setup, find_packages PROJECT_ROOT = os.path.dirname(__file__) with open(os.path.join(PROJECT_ROOT, 'README.rst')) as file_: long_description = file_.read() version_regex = r'__version__ = ["\']([^"\']*)["\']' with open(os.path.join(PROJECT_ROOT, 'src/priority/__init__.py')) as file_: text = file_.read() match = re.search(version_regex, text) if match: version = match.group(1) else: raise RuntimeError("No version number found!") setup( name='priority', version=version, description='A pure-Python implementation of the HTTP/2 priority tree', long_description=long_description, long_description_content_type='text/x-rst', author='Cory Benfield', author_email='cory@lukasa.co.uk', url='https://github.com/python-hyper/priority/', project_urls={ 'Documentation': 'https://python-hyper.org/projects/priority/', 'Source': 'https://github.com/python-hyper/priority/', 'Tracker': 'https://github.com/python-hyper/priority/issues', 'Changelog': 'https://github.com/python-hyper/priority/blob/master/HISTORY.rst', }, packages=find_packages(where='src'), package_data={'priority': ['py.typed']}, package_dir={'': 'src'}, python_requires='>=3.6.1', license='MIT License', classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.9', 'Programming Language :: Python :: Implementation :: CPython', 'Programming Language :: Python :: Implementation :: PyPy', ], ) ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5407236 priority-2.0.0/src/0000755000076500000240000000000000000000000013463 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5503647 priority-2.0.0/src/priority/0000755000076500000240000000000000000000000015344 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624787802.0 priority-2.0.0/src/priority/__init__.py0000644000076500000240000000053200000000000017455 0ustar00kriechistaff# -*- coding: utf-8 -*- """ priority: HTTP/2 priority implementation for Python """ from .priority import ( # noqa Stream, PriorityTree, DeadlockError, PriorityLoop, PriorityError, DuplicateStreamError, MissingStreamError, TooManyStreamsError, BadWeightError, PseudoStreamError, ) __version__ = "2.0.0" ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624309761.0 priority-2.0.0/src/priority/priority.py0000644000076500000240000003731200000000000017605 0ustar00kriechistaff# -*- coding: utf-8 -*- """ priority/tree ~~~~~~~~~~~~~ Implementation of the Priority tree data structure. """ import heapq from typing import List, Tuple, Optional class PriorityError(Exception): """ The base class for all ``priority`` exceptions. """ class DeadlockError(PriorityError): """ Raised when there are no streams that can make progress: all streams are blocked. """ pass class PriorityLoop(PriorityError): """ An unexpected priority loop has been detected. The tree is invalid. """ pass class DuplicateStreamError(PriorityError): """ An attempt was made to insert a stream that already exists. """ pass class MissingStreamError(KeyError, PriorityError): """ An operation was attempted on a stream that is not present in the tree. """ pass class TooManyStreamsError(PriorityError): """ An attempt was made to insert a dangerous number of streams into the priority tree at the same time. .. versionadded:: 1.2.0 """ pass class BadWeightError(PriorityError): """ An attempt was made to create a stream with an invalid weight. .. versionadded:: 1.3.0 """ pass class PseudoStreamError(PriorityError): """ An operation was attempted on stream 0. .. versionadded:: 1.3.0 """ pass class Stream: """ Priority information for a given stream. :param stream_id: The stream ID for the new stream. :param weight: (optional) The stream weight. Defaults to 16. """ def __init__(self, stream_id: int, weight: int = 16) -> None: self.stream_id = stream_id self.weight = weight self.children: List[Stream] = [] self.parent: Optional[Stream] = None self.child_queue: List[Tuple[int, Stream]] = [] self.active = True self.last_weight = 0 self._deficit = 0 @property def weight(self) -> int: return self._weight @weight.setter def weight(self, value: int) -> None: # RFC 7540 § 5.3.2: "All dependent streams are allocated an integer # weight between 1 and 256 (inclusive)." if not isinstance(value, int): raise BadWeightError("Stream weight should be an integer") elif not (1 <= value <= 256): raise BadWeightError("Stream weight must be between 1 and 256 (inclusive)") self._weight = value def add_child(self, child: "Stream") -> None: """ Add a stream that depends on this one. :param child: A ``Stream`` object that depends on this one. """ child.parent = self self.children.append(child) heapq.heappush(self.child_queue, (self.last_weight, child)) def add_child_exclusive(self, child: "Stream") -> None: """ Add a stream that exclusively depends on this one. :param child: A ``Stream`` object that exclusively depends on this one. """ old_children = self.children self.children = [] self.child_queue = [] self.last_weight = 0 self.add_child(child) for old_child in old_children: child.add_child(old_child) def remove_child( self, child: "Stream", strip_children: bool = True, ) -> None: """ Removes a child stream from this stream. This is a potentially somewhat expensive operation. :param child: The child stream to remove. :param strip_children: Whether children of the removed stream should become children of this stream. """ # To do this we do the following: # # - remove the child stream from the list of children # - build a new priority queue, filtering out the child when we find # it in the old one self.children.remove(child) new_queue: List[Tuple[int, Stream]] = [] while self.child_queue: level, stream = heapq.heappop(self.child_queue) if stream == child: continue heapq.heappush(new_queue, (level, stream)) self.child_queue = new_queue if strip_children: for new_child in child.children: self.add_child(new_child) def schedule(self) -> int: """ Returns the stream ID of the next child to schedule. Potentially recurses down the tree of priorities. """ # Cannot be called on active streams. assert not self.active next_stream = None popped_streams = [] # Spin looking for the next active stream. Everything we pop off has # to be rescheduled, even if it turns out none of them were active at # this time. try: while next_stream is None: # If the queue is empty, immediately fail. val = heapq.heappop(self.child_queue) popped_streams.append(val) level, child = val if child.active: next_stream = child.stream_id else: # Guard against the possibility that the child also has no # suitable children. try: next_stream = child.schedule() except IndexError: continue finally: for level, child in popped_streams: self.last_weight = level level += (256 + child._deficit) // child.weight child._deficit = (256 + child._deficit) % child.weight heapq.heappush(self.child_queue, (level, child)) return next_stream # Custom repr def __repr__(self) -> str: return "Stream" % (self.stream_id, self.weight) # Custom comparison def __eq__(self, other: object) -> bool: if not isinstance(other, Stream): # pragma: no cover return False return self.stream_id == other.stream_id def __ne__(self, other: object) -> bool: return not self.__eq__(other) def __lt__(self, other: "Stream") -> bool: if not isinstance(other, Stream): # pragma: no cover return NotImplemented return self.stream_id < other.stream_id def __le__(self, other: "Stream") -> bool: if not isinstance(other, Stream): # pragma: no cover return NotImplemented return self.stream_id <= other.stream_id def __gt__(self, other: "Stream") -> bool: if not isinstance(other, Stream): # pragma: no cover return NotImplemented return self.stream_id > other.stream_id def __ge__(self, other: "Stream") -> bool: if not isinstance(other, Stream): # pragma: no cover return NotImplemented return self.stream_id >= other.stream_id def _stream_cycle(new_parent: Stream, current: Stream) -> bool: """ Reports whether the new parent depends on the current stream. """ parent = new_parent # Don't iterate forever, but instead assume that the tree doesn't # get more than 100 streams deep. This should catch accidental # tree loops. This is the definition of defensive programming. for _ in range(100): parent = parent.parent # type: ignore[assignment] if parent.stream_id == current.stream_id: return True elif parent.stream_id == 0: return False raise PriorityLoop( "Stream %d is in a priority loop." % new_parent.stream_id ) # pragma: no cover class PriorityTree: """ A HTTP/2 Priority Tree. This tree stores HTTP/2 streams according to their HTTP/2 priorities. .. versionchanged:: 1.2.0 Added ``maximum_streams`` keyword argument. :param maximum_streams: The maximum number of streams that may be active in the priority tree at any one time. If this number is exceeded, the priority tree will raise a :class:`TooManyStreamsError ` and will refuse to insert the stream. This parameter exists to defend against the possibility of DoS attack by attempting to overfill the priority tree. If any endpoint is attempting to manage the priority of this many streams at once it is probably trying to screw with you, so it is sensible to simply refuse to play ball at that point. While we allow the user to configure this, we don't really *expect* them too, unless they want to be even more conservative than we are by default. :type maximum_streams: ``int`` """ def __init__(self, maximum_streams: int = 1000) -> None: # This flat array keeps hold of all the streams that are logically # dependent on stream 0. self._root_stream = Stream(stream_id=0, weight=1) self._root_stream.active = False self._streams = {0: self._root_stream} if not isinstance(maximum_streams, int): raise TypeError("maximum_streams must be an int.") if maximum_streams <= 0: raise ValueError("maximum_streams must be a positive integer.") self._maximum_streams = maximum_streams def _get_or_insert_parent(self, parent_stream_id: int) -> Stream: """ When inserting or reprioritizing a stream it is possible to make it dependent on a stream that is no longer in the tree. In this situation, rather than bail out, we should insert the parent stream into the tree with default priority and mark it as blocked. """ try: return self._streams[parent_stream_id] except KeyError: self.insert_stream(parent_stream_id) self.block(parent_stream_id) return self._streams[parent_stream_id] def _exclusive_insert( self, parent_stream: Stream, inserted_stream: Stream, ) -> None: """ Insert ``inserted_stream`` beneath ``parent_stream``, obeying the semantics of exclusive insertion. """ parent_stream.add_child_exclusive(inserted_stream) def insert_stream( self, stream_id: int, depends_on: Optional[int] = None, weight: int = 16, exclusive: bool = False, ) -> None: """ Insert a stream into the tree. :param stream_id: The stream ID of the stream being inserted. :param depends_on: (optional) The ID of the stream that the new stream depends on, if any. :param weight: (optional) The weight to give the new stream. Defaults to 16. :param exclusive: (optional) Whether this new stream should be an exclusive dependency of the parent. """ if stream_id in self._streams: raise DuplicateStreamError("Stream %d already in tree" % stream_id) if (len(self._streams) + 1) > self._maximum_streams: raise TooManyStreamsError( "Refusing to insert %d streams into priority tree at once" % (self._maximum_streams + 1) ) stream = Stream(stream_id, weight) if not depends_on: depends_on = 0 elif depends_on == stream_id: raise PriorityLoop("Stream %d must not depend on itself." % stream_id) if exclusive: parent_stream = self._get_or_insert_parent(depends_on) self._exclusive_insert(parent_stream, stream) self._streams[stream_id] = stream return parent = self._get_or_insert_parent(depends_on) parent.add_child(stream) self._streams[stream_id] = stream def reprioritize( self, stream_id: int, depends_on: Optional[int] = None, weight: int = 16, exclusive: bool = False, ) -> None: """ Update the priority status of a stream already in the tree. :param stream_id: The stream ID of the stream being updated. :param depends_on: (optional) The ID of the stream that the stream now depends on. If ``None``, will be moved to depend on stream 0. :param weight: (optional) The new weight to give the stream. Defaults to 16. :param exclusive: (optional) Whether this stream should now be an exclusive dependency of the new parent. """ if stream_id == 0: raise PseudoStreamError("Cannot reprioritize stream 0") try: current_stream = self._streams[stream_id] except KeyError: raise MissingStreamError("Stream %d not in tree" % stream_id) # Update things in a specific order to make sure the calculation # behaves properly. Specifically, we first update the weight. Then, # we check whether this stream is being made dependent on one of its # own dependents. Then, we remove this stream from its current parent # and move it to its new parent, taking its children with it. if depends_on: if depends_on == stream_id: raise PriorityLoop("Stream %d must not depend on itself" % stream_id) new_parent = self._get_or_insert_parent(depends_on) cycle = _stream_cycle(new_parent, current_stream) else: new_parent = self._streams[0] cycle = False current_stream.weight = weight # Our new parent is currently dependent on us. We should remove it from # its parent, and make it a child of our current parent, and then # continue. if cycle: new_parent.parent.remove_child( # type: ignore[union-attr] new_parent, ) current_stream.parent.add_child( # type: ignore[union-attr] new_parent, ) current_stream.parent.remove_child( # type: ignore[union-attr] current_stream, strip_children=False ) if exclusive: new_parent.add_child_exclusive(current_stream) else: new_parent.add_child(current_stream) def remove_stream(self, stream_id: int) -> None: """ Removes a stream from the priority tree. :param stream_id: The ID of the stream to remove. """ if stream_id == 0: raise PseudoStreamError("Cannot remove stream 0") try: child = self._streams.pop(stream_id) except KeyError: raise MissingStreamError("Stream %d not in tree" % stream_id) parent = child.parent parent.remove_child(child) # type: ignore[union-attr] def block(self, stream_id: int) -> None: """ Marks a given stream as blocked, with no data to send. :param stream_id: The ID of the stream to block. """ if stream_id == 0: raise PseudoStreamError("Cannot block stream 0") try: self._streams[stream_id].active = False except KeyError: raise MissingStreamError("Stream %d not in tree" % stream_id) def unblock(self, stream_id: int) -> None: """ Marks a given stream as unblocked, with more data to send. :param stream_id: The ID of the stream to unblock. """ if stream_id == 0: raise PseudoStreamError("Cannot unblock stream 0") try: self._streams[stream_id].active = True except KeyError: raise MissingStreamError("Stream %d not in tree" % stream_id) # The iterator protocol def __iter__(self) -> "PriorityTree": # pragma: no cover return self def __next__(self) -> int: # pragma: no cover try: return self._root_stream.schedule() except IndexError: raise DeadlockError("No unblocked streams to schedule.") def next(self) -> int: # pragma: no cover return self.__next__() ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624309761.0 priority-2.0.0/src/priority/py.typed0000644000076500000240000000000000000000000017031 0ustar00kriechistaff././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1624788894.552094 priority-2.0.0/src/priority.egg-info/0000755000076500000240000000000000000000000017036 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624788894.0 priority-2.0.0/src/priority.egg-info/PKG-INFO0000644000076500000240000001476200000000000020145 0ustar00kriechistaffMetadata-Version: 2.1 Name: priority Version: 2.0.0 Summary: A pure-Python implementation of the HTTP/2 priority tree Home-page: https://github.com/python-hyper/priority/ Author: Cory Benfield Author-email: cory@lukasa.co.uk License: MIT License Project-URL: Documentation, https://python-hyper.org/projects/priority/ Project-URL: Source, https://github.com/python-hyper/priority/ Project-URL: Tracker, https://github.com/python-hyper/priority/issues Project-URL: Changelog, https://github.com/python-hyper/priority/blob/master/HISTORY.rst Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Requires-Python: >=3.6.1 Description-Content-Type: text/x-rst License-File: LICENSE ========================================== Priority: A HTTP/2 Priority Implementation ========================================== .. image:: https://github.com/python-hyper/priority/workflows/CI/badge.svg :target: https://github.com/python-hyper/priority/actions :alt: Build Status .. image:: https://codecov.io/gh/python-hyper/priority/branch/master/graph/badge.svg :target: https://codecov.io/gh/python-hyper/priority :alt: Code Coverage .. image:: https://readthedocs.org/projects/priority/badge/?version=latest :target: https://priority.readthedocs.io/en/latest/ :alt: Documentation Status .. image:: https://img.shields.io/badge/chat-join_now-brightgreen.svg :target: https://gitter.im/python-hyper/community :alt: Chat community .. image:: https://raw.github.com/python-hyper/documentation/master/source/logo/hyper-black-bg-white.png Priority is a pure-Python implementation of the priority logic for HTTP/2, set out in `RFC 7540 Section 5.3 (Stream Priority)`_. This logic allows for clients to express a preference for how the server allocates its (limited) resources to the many outstanding HTTP requests that may be running over a single HTTP/2 connection. Specifically, this Python implementation uses a variant of the implementation used in the excellent `H2O`_ project. This original implementation is also the inspiration for `nghttp2's`_ priority implementation, and generally produces a very clean and even priority stream. The only notable changes from H2O's implementation are small modifications to allow the priority implementation to work cleanly as a separate implementation, rather than being embedded in a HTTP/2 stack directly. While priority information in HTTP/2 is only a suggestion, rather than an enforceable constraint, where possible servers should respect the priority requests of their clients. Using Priority -------------- Priority has a simple API. Streams are inserted into the tree: when they are inserted, they may optionally have a weight, depend on another stream, or become an exclusive dependent of another stream. .. code-block:: python >>> p = priority.PriorityTree() >>> p.insert_stream(stream_id=1) >>> p.insert_stream(stream_id=3) >>> p.insert_stream(stream_id=5, depends_on=1) >>> p.insert_stream(stream_id=7, weight=32) >>> p.insert_stream(stream_id=9, depends_on=7, weight=8) >>> p.insert_stream(stream_id=11, depends_on=7, exclusive=True) Once streams are inserted, the stream priorities can be requested. This allows the server to make decisions about how to allocate resources. Iterating The Tree ~~~~~~~~~~~~~~~~~~ The tree in this algorithm acts as a gate. Its goal is to allow one stream "through" at a time, in such a manner that all the active streams are served as evenly as possible in proportion to their weights. This is handled in Priority by iterating over the tree. The tree itself is an iterator, and each time it is advanced it will yield a stream ID. This is the ID of the stream that should next send data. This looks like this: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) If each stream only sends when it is 'ungated' by this mechanism, the server will automatically be emitting stream data in conformance to RFC 7540. Updating The Tree ~~~~~~~~~~~~~~~~~ If for any reason a stream is unable to proceed (for example, it is blocked on HTTP/2 flow control, or it is waiting for more data from another service), that stream is *blocked*. The ``PriorityTree`` should be informed that the stream is blocked so that other dependent streams get a chance to proceed. This can be done by calling the ``block`` method of the tree with the stream ID that is currently unable to proceed. This will automatically update the tree, and it will adjust on the fly to correctly allow any streams that were dependent on the blocked one to progress. For example: .. code-block:: python >>> for stream_id in p: ... send_data(stream_id) ... if blocked(stream_id): ... p.block(stream_id) When a stream goes from being blocked to being unblocked, call the ``unblock`` method to place it back into the sequence. Both the ``block`` and ``unblock`` methods are idempotent and safe to call repeatedly. Additionally, the priority of a stream may change. When it does, the ``reprioritize`` method can be used to update the tree in the wake of that change. ``reprioritize`` has the same signature as ``insert_stream``, but applies only to streams already in the tree. Removing Streams ~~~~~~~~~~~~~~~~ A stream can be entirely removed from the tree by calling ``remove_stream``. Note that this is not idempotent. Further, calling ``remove_stream`` and then re-adding it *may* cause a substantial change in the shape of the priority tree, and *will* cause the iteration order to change. License ------- Priority is made available under the MIT License. For more details, see the LICENSE file in the repository. Authors ------- Priority is maintained by Cory Benfield, with contributions from others. For more details about the contributors, please see CONTRIBUTORS.rst in the repository. .. _RFC 7540 Section 5.3 (Stream Priority): https://tools.ietf.org/html/rfc7540#section-5.3 .. _nghttp2's: https://nghttp2.org/blog/2015/11/11/stream-scheduling-utilizing-http2-priority/ .. _H2O: https://h2o.examp1e.net/ ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624788894.0 priority-2.0.0/src/priority.egg-info/SOURCES.txt0000644000076500000240000000112200000000000020716 0ustar00kriechistaffCHANGELOG.rst CONTRIBUTORS.rst LICENSE MANIFEST.in README.rst setup.cfg setup.py tox.ini docs/Makefile docs/make.bat docs/source/api.rst docs/source/authors.rst docs/source/conf.py docs/source/index.rst docs/source/installation.rst docs/source/license.rst docs/source/using-priority.rst docs/source/security/CVE-2016-6580.rst docs/source/security/index.rst src/priority/__init__.py src/priority/priority.py src/priority/py.typed src/priority.egg-info/PKG-INFO src/priority.egg-info/SOURCES.txt src/priority.egg-info/dependency_links.txt src/priority.egg-info/top_level.txt test/test_priority.py././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624788894.0 priority-2.0.0/src/priority.egg-info/dependency_links.txt0000644000076500000240000000000100000000000023104 0ustar00kriechistaff ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624788894.0 priority-2.0.0/src/priority.egg-info/top_level.txt0000644000076500000240000000001100000000000021560 0ustar00kriechistaffpriority ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1624788894.5527163 priority-2.0.0/test/0000755000076500000240000000000000000000000013653 5ustar00kriechistaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624309761.0 priority-2.0.0/test/test_priority.py0000644000076500000240000005132000000000000017146 0ustar00kriechistaff# -*- coding: utf-8 -*- """ test_priority ~~~~~~~~~~~~~ Tests for the Priority trees """ import operator import collections import itertools import pytest from hypothesis import given, settings from hypothesis.stateful import invariant, RuleBasedStateMachine, rule from hypothesis.strategies import integers, lists, tuples, sampled_from import priority from typing import Iterable, List, Dict, Any STREAMS_AND_WEIGHTS = lists( elements=tuples(integers(min_value=1), integers(min_value=1, max_value=255)), unique_by=operator.itemgetter(0), ) BLOCKED_AND_ACTIVE = lists( elements=sampled_from([1, 3, 5, 7, 9, 11]), unique=True, ).map(lambda blocked: (blocked, active_readme_streams_from_filter(blocked))) UNBLOCKED_AND_ACTIVE = lists( elements=sampled_from([1, 3, 5, 7, 9, 11]), unique=True, ).map( lambda unblocked: ( unblocked, active_readme_streams_from_filter(unblocked, blocked=False), ) ) def readme_tree(): """ Provide a tree configured as the one in the readme. """ p = priority.PriorityTree() p.insert_stream(stream_id=1) p.insert_stream(stream_id=3) p.insert_stream(stream_id=5, depends_on=1) p.insert_stream(stream_id=7, weight=32) p.insert_stream(stream_id=9, depends_on=7, weight=8) p.insert_stream(stream_id=11, depends_on=7, exclusive=True) return p def active_readme_streams_from_filter( filtered: Iterable[int], blocked: bool = True, ) -> List[int]: """ Given a collection of filtered streams, determine which ones are active. This applies only to the readme tree at this time, though in future it should be possible to apply this to an arbitrary tree. If ``blocked`` is ``True``, the filter is a set of blocked streams. If ``False``, it's a collection of unblocked streams. """ tree = { 1: { 5: {}, }, 3: {}, 7: { 11: { 9: {}, }, }, } filtered = set(filtered) def get_expected(tree: Dict[Any, Any]) -> List[int]: expected = [] for stream_id in tree: if stream_id not in filtered and blocked: expected.append(stream_id) elif stream_id in filtered and not blocked: expected.append(stream_id) else: expected.extend(get_expected(tree[stream_id])) return expected return get_expected(tree) class TestStream: def test_stream_repr(self): """ The stream representation renders according to the README. """ s = priority.Stream(stream_id=80, weight=16) assert repr(s) == "Stream" @given(STREAMS_AND_WEIGHTS) def test_streams_are_well_ordered(self, streams_and_weights): """ Streams are ordered by their stream ID. """ stream_list = [ priority.Stream(stream_id=s, weight=w) for s, w in streams_and_weights ] stream_list = sorted(stream_list) streams_by_id = [stream.stream_id for stream in stream_list] assert sorted(streams_by_id) == streams_by_id @given( integers(min_value=1, max_value=2 ** 24), integers(min_value=1, max_value=2 ** 24), ) def test_stream_ordering(self, a, b): """ Two streams are well ordered based on their stream ID. """ s1 = priority.Stream(stream_id=a, weight=16) s2 = priority.Stream(stream_id=b, weight=32) assert (s1 < s2) == (a < b) assert (s1 <= s2) == (a <= b) assert (s1 > s2) == (a > b) assert (s1 >= s2) == (a >= b) assert (s1 == s2) == (a == b) assert (s1 != s2) == (a != b) class TestPriorityTreeManual: """ These tests manually confirm that the PriorityTree output is correct. They use the PriorityTree given in the README and confirm that it outputs data as expected. If possible, I'd like to eventually replace these tests with Hypothesis-based ones for the same data, but getting Hypothesis to generate useful data in this case is going to be quite tricky. """ @given(BLOCKED_AND_ACTIVE) def test_priority_tree_initially_outputs_all_stream_ids(self, blocked_expected): """ The first iterations of the priority tree initially output the active streams, in order of stream ID, regardless of weight. """ tree = readme_tree() blocked = blocked_expected[0] expected = blocked_expected[1] for stream_id in blocked: tree.block(stream_id) result = [next(tree) for _ in range(len(expected))] assert expected == result @given(UNBLOCKED_AND_ACTIVE) def test_priority_tree_blocking_is_isomorphic(self, allowed_expected): """ Blocking everything and then unblocking certain ones has the same effect as blocking specific streams. """ tree = readme_tree() allowed = allowed_expected[0] expected = allowed_expected[1] for stream_id in range(1, 12, 2): tree.block(stream_id) for stream_id in allowed: tree.unblock(stream_id) result = [next(tree) for _ in range(len(expected))] assert expected == result @given(BLOCKED_AND_ACTIVE) def test_removing_items_behaves_similarly_to_blocking(self, blocked_expected): """ From the perspective of iterating over items, removing streams should have the same effect as blocking them, except that the ordering changes. Because the ordering is not important, don't test for it. """ tree = readme_tree() blocked = blocked_expected[0] expected = set(blocked_expected[1]) for stream_id in blocked: tree.remove_stream(stream_id) result = set(next(tree) for _ in range(len(expected))) assert expected == result def test_priority_tree_raises_deadlock_error_if_all_blocked(self): """ Assuming all streams are blocked and none can progress, asking for the one with the next highest priority fires a DeadlockError. """ tree = readme_tree() for stream_id in range(1, 12, 2): tree.block(stream_id) with pytest.raises(priority.DeadlockError): next(tree) @pytest.mark.parametrize( "stream,new_parent,exclusive,weight,blocked,result", [ (1, 3, False, 16, [], [3, 7, 7, 3, 7, 7, 3, 7, 7]), (1, 5, False, 16, [], [3, 5, 7, 7, 3, 5, 7, 7, 3]), (1, 5, False, 16, [5], [3, 1, 7, 7, 3, 1, 7, 7, 3]), (5, 7, False, 16, [7, 1], [3, 5, 11, 3, 5, 11, 3, 5, 11]), (11, None, False, 16, [], [1, 3, 7, 11, 7, 1, 3, 7, 11]), (11, None, False, 16, [11], [1, 3, 7, 9, 7, 1, 3, 7, 9]), (7, 9, False, 16, [], [1, 3, 9, 1, 3, 1, 3, 9, 1]), (7, 1, True, 16, [], [1, 3, 1, 3, 1, 3, 1, 3, 1]), (7, 1, True, 16, [1], [7, 3, 7, 3, 7, 3, 7, 3, 7]), (7, 1, True, 16, [1, 7], [5, 3, 11, 3, 5, 3, 11, 3, 5]), (1, 0, False, 32, [], [1, 3, 7, 1, 7, 1, 3, 7, 1]), (1, 0, True, 32, [], [1, 1, 1, 1, 1, 1, 1, 1, 1]), (1, 0, True, 32, [1], [3, 5, 7, 7, 3, 5, 7, 7, 3]), (1, None, True, 32, [], [1, 1, 1, 1, 1, 1, 1, 1, 1]), (1, None, True, 32, [1], [3, 5, 7, 7, 3, 5, 7, 7, 3]), ], ) def test_can_reprioritize_a_stream( self, stream, new_parent, exclusive, weight, blocked, result ): """ Reprioritizing streams adjusts the outputs of the tree. """ t = readme_tree() for s in blocked: t.block(s) t.reprioritize( stream_id=stream, depends_on=new_parent, weight=weight, exclusive=exclusive, ) actual_result = [next(t) for _ in range(len(result))] assert actual_result == result def test_priority_tree_raises_error_inserting_duplicate(self): """ Attempting to insert a stream that is already in the tree raises a DuplicateStreamError """ p = priority.PriorityTree() p.insert_stream(1) with pytest.raises(priority.DuplicateStreamError): p.insert_stream(1) def test_priority_raises_good_errors_for_missing_streams(self): """ Attempting operations on absent streams raises a MissingStreamError. """ p = priority.PriorityTree() p.insert_stream(1) with pytest.raises(priority.MissingStreamError): p.reprioritize(3) with pytest.raises(priority.MissingStreamError): p.block(3) with pytest.raises(priority.MissingStreamError): p.unblock(3) with pytest.raises(priority.MissingStreamError): p.remove_stream(3) def test_priority_raises_good_errors_for_zero_stream(self): """ Attempting operations on stream 0 raises a PseudoStreamError. """ p = priority.PriorityTree() p.insert_stream(1) with pytest.raises(priority.PseudoStreamError): p.reprioritize(0) with pytest.raises(priority.PseudoStreamError): p.block(0) with pytest.raises(priority.PseudoStreamError): p.unblock(0) with pytest.raises(priority.PseudoStreamError): p.remove_stream(0) @pytest.mark.parametrize("exclusive", [True, False]) def test_priority_allows_inserting_stream_with_absent_parent(self, exclusive): """ Attemping to insert a stream that depends on a stream that is not in the tree automatically inserts the parent with default priority. """ p = priority.PriorityTree() p.insert_stream(stream_id=3, depends_on=1, exclusive=exclusive, weight=32) # Iterate 10 times to prove that the parent stream starts blocked. first_ten_ids = [next(p) for _ in range(0, 10)] assert first_ten_ids == [3] * 10 # Unblock the parent. p.unblock(1) # Iterate 10 times, expecting only the parent. next_ten_ids = [next(p) for _ in range(0, 10)] assert next_ten_ids == [1] * 10 # Insert a new stream into the tree with default priority. p.insert_stream(stream_id=5) # Iterate 10 more times. Expect the parent, and the new stream, in # equal amounts. next_ten_ids = [next(p) for _ in range(0, 10)] assert next_ten_ids == [5, 1] * 5 @pytest.mark.parametrize("exclusive", [True, False]) def test_priority_reprioritizing_stream_with_absent_parent(self, exclusive): """ Attemping to reprioritize a stream to depend on a stream that is not in the tree automatically inserts the parent with default priority. """ p = priority.PriorityTree() p.insert_stream(stream_id=3) p.reprioritize(stream_id=3, depends_on=1, exclusive=exclusive, weight=32) # Iterate 10 times to prove that the parent stream starts blocked. first_ten_ids = [next(p) for _ in range(0, 10)] assert first_ten_ids == [3] * 10 # Unblock the parent. p.unblock(1) # Iterate 10 times, expecting only the parent. next_ten_ids = [next(p) for _ in range(0, 10)] assert next_ten_ids == [1] * 10 # Insert a new stream into the tree with default priority. p.insert_stream(stream_id=5) # Iterate 10 more times. Expect the parent, and the new stream, in # equal amounts. next_ten_ids = [next(p) for _ in range(0, 10)] assert next_ten_ids == [5, 1] * 5 @pytest.mark.parametrize("count", range(2, 10000, 100)) def test_priority_refuses_to_allow_too_many_streams_in_tree(self, count): """ Attempting to insert more streams than maximum_streams into the tree fails. """ p = priority.PriorityTree(maximum_streams=count) # This isn't an off-by-one error: stream 0 is in the tree by default. for x in range(1, count): p.insert_stream(x) with pytest.raises(priority.TooManyStreamsError): p.insert_stream(x + 1) @pytest.mark.parametrize("depends_on", [0, None]) def test_can_insert_stream_with_exclusive_dependency_on_0(self, depends_on): """ It is acceptable to insert a stream with an exclusive dependency on stream 0, both explicitly and implicitly. """ p = priority.PriorityTree() p.insert_stream(stream_id=1) p.insert_stream(stream_id=3) p.insert_stream(stream_id=5, depends_on=depends_on, exclusive=True) next_ten_ids = [next(p) for _ in range(0, 10)] assert next_ten_ids == [5] * 10 @pytest.mark.parametrize("weight", [None, 0.5, float("inf"), "priority", object]) def test_stream_with_non_integer_weight_is_error(self, weight): """ Giving a stream a non-integer weight is rejected. """ p = priority.PriorityTree() with pytest.raises(priority.BadWeightError) as err: p.insert_stream(stream_id=1, weight=weight) assert err.value.args[0] == "Stream weight should be an integer" p.insert_stream(stream_id=2) with pytest.raises(priority.BadWeightError) as err: p.reprioritize(stream_id=2, weight=weight) assert err.value.args[0] == "Stream weight should be an integer" @pytest.mark.parametrize( "weight", [ 0, 257, 1000, -42, ], ) def test_stream_with_out_of_bounds_weight_is_error(self, weight): """ Giving a stream an out-of-bounds integer weight is rejected. """ p = priority.PriorityTree() with pytest.raises(priority.BadWeightError) as err: p.insert_stream(stream_id=1, weight=weight) assert ( err.value.args[0] == "Stream weight must be between 1 and 256 (inclusive)" ) p.insert_stream(stream_id=2) with pytest.raises(priority.BadWeightError) as err: p.reprioritize(stream_id=2, weight=weight) assert ( err.value.args[0] == "Stream weight must be between 1 and 256 (inclusive)" ) @pytest.mark.parametrize("exclusive", (True, False)) @pytest.mark.parametrize("stream_id", (1, 5, 20, 32, 256)) def test_stream_depending_on_self_is_error(self, stream_id, exclusive): """ Inserting a stream that is dependent on itself is rejected. """ p = priority.PriorityTree() with pytest.raises(priority.PriorityLoop): p.insert_stream( stream_id=stream_id, depends_on=stream_id, exclusive=exclusive ) @pytest.mark.parametrize("exclusive", (True, False)) @pytest.mark.parametrize("stream_id", (1, 5, 20, 32, 256)) def test_reprioritize_depend_on_self_is_error(self, stream_id, exclusive): """ Reprioritizing a stream to make it dependent on itself is an error. """ p = priority.PriorityTree() p.insert_stream(stream_id=stream_id) with pytest.raises(priority.PriorityLoop): p.reprioritize( stream_id=stream_id, depends_on=stream_id, exclusive=exclusive ) @pytest.mark.parametrize("maximum_streams", (None, "foo", object(), 2.0)) def test_maximum_streams_with_non_int_is_error(self, maximum_streams): """ Creating a PriorityTree with a non-int argument for maximum_streams is an error. """ with pytest.raises(TypeError) as err: priority.PriorityTree(maximum_streams=maximum_streams) assert err.value.args[0] == "maximum_streams must be an int." @pytest.mark.parametrize("maximum_streams", (0, -1, -50)) def test_maximum_streams_with_bad_int_is_error(self, maximum_streams): """ Creating a PriorityTree with a non-positive integer for maximum_streams is an error. """ with pytest.raises(ValueError) as err: priority.PriorityTree(maximum_streams=maximum_streams) assert err.value.args[0] == "maximum_streams must be a positive integer." class TestPriorityTreeOutput: """ These tests use Hypothesis to attempt to bound the output of iterating over the priority tree. In particular, their goal is to ensure that the output of the tree is "good enough": that it meets certain requirements on fairness and equidistribution. """ @given(STREAMS_AND_WEIGHTS) @settings(deadline=None) def test_period_of_repetition(self, streams_and_weights): """ The period of repetition of a priority sequence is given by the sum of the weights of the streams. Once that many values have been pulled out the sequence repeats identically. """ p = priority.PriorityTree() weights = [] for stream, weight in streams_and_weights: p.insert_stream(stream_id=stream, weight=weight) weights.append(weight) period = sum(weights) # Pop off the first n elements, which will always be evenly # distributed. for _ in weights: next(p) pattern = [next(p) for _ in range(period)] pattern = itertools.cycle(pattern) for i in range(period * 20): assert next(p) == next(pattern), i @given(STREAMS_AND_WEIGHTS) def test_priority_tree_distribution(self, streams_and_weights): """ Once a full period of repetition has been observed, each stream has been emitted a number of times equal to its weight. """ p = priority.PriorityTree() weights = [] for stream, weight in streams_and_weights: p.insert_stream(stream_id=stream, weight=weight) weights.append(weight) period = sum(weights) # Pop off the first n elements, which will always be evenly # distributed. for _ in weights: next(p) count = collections.Counter(next(p) for _ in range(period)) assert len(count) == len(streams_and_weights) for stream, weight in streams_and_weights: count[stream] == weight class PriorityStateMachine(RuleBasedStateMachine): """ This test uses Hypothesis's stateful testing to exercise the priority tree. It randomly inserts, removes, blocks and unblocks streams in the tree, then checks that iterating over priority still selects a sensible stream. """ def __init__(self): super(PriorityStateMachine, self).__init__() self.tree = priority.PriorityTree() self.stream_ids = set([0]) self.blocked_stream_ids = set() @rule(stream_id=integers()) # type: ignore[no-untyped-call] def insert_stream(self, stream_id): try: self.tree.insert_stream(stream_id) except priority.DuplicateStreamError: assert stream_id in self.stream_ids else: assert stream_id not in self.stream_ids self.stream_ids.add(stream_id) def _run_action(self, action, stream_id): try: action(stream_id) except priority.MissingStreamError: assert stream_id not in self.stream_ids except priority.PseudoStreamError: assert stream_id == 0 else: assert stream_id in self.stream_ids @rule(stream_id=integers()) # type: ignore[no-untyped-call] def remove_stream(self, stream_id): self._run_action(self.tree.remove_stream, stream_id) if stream_id != 0: self.stream_ids.discard(stream_id) @rule(stream_id=integers()) # type: ignore[no-untyped-call] def block_stream(self, stream_id): self._run_action(self.tree.block, stream_id) if (stream_id != 0) and (stream_id in self.stream_ids): self.blocked_stream_ids.add(stream_id) @rule(stream_id=integers()) # type: ignore[no-untyped-call] def unblock_stream(self, stream_id): self._run_action(self.tree.unblock, stream_id) self.blocked_stream_ids.discard(stream_id) @invariant() # type: ignore[no-untyped-call] def check_next_stream_consistent(self): """ If we ask priority for the next stream, it always returns a sensible result. """ try: next_stream_id = next(self.tree) except priority.DeadlockError: assert self.blocked_stream_ids ^ {0} == self.stream_ids else: stream = self.tree._streams[next_stream_id] # If a stream is selected, then it isn't blocked assert stream.active # If a stream is selected, then its parent is either the root # stream or blocked parent = stream.parent assert (parent.stream_id == 0) or (not parent.active) TestPriorityTreeStateful = PriorityStateMachine.TestCase ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1624787790.0 priority-2.0.0/tox.ini0000644000076500000240000000246600000000000014217 0ustar00kriechistaff[tox] envlist = py36, py37, py38, py39, pypy3, lint, docs, packaging [gh-actions] python = 3.6: py36 3.7: py37 3.8: py38 3.9: py39, lint, docs, packaging pypy3: pypy3 [testenv] passenv = GITHUB_* deps = pytest>=6.2,<7 pytest-cov>=2.10,<3 pytest-xdist>=2.1,<3 hypothesis>=6.9,<7 commands = pytest --cov-report=xml --cov-report=term --cov=priority {posargs} [testenv:pypy3] # temporarily disable coverage testing on PyPy due to performance problems commands = pytest {posargs} [testenv:lint] deps = flake8>=3.8,<4 black==21.6b0 mypy==0.910 {[testenv]deps} commands = flake8 src/ test/ black --check --diff src/ test/ mypy src/ test/ [testenv:docs] deps = sphinx>=3.5,<5 whitelist_externals = make changedir = {toxinidir}/docs commands = make clean make html [testenv:packaging] basepython = python3.9 deps = check-manifest==0.46 readme-renderer==29.0 twine>=3.4.1,<4 whitelist_externals = rm commands = rm -rf dist/ check-manifest python setup.py sdist bdist_wheel twine check dist/* [testenv:publish] basepython = {[testenv:packaging]basepython} deps = {[testenv:packaging]deps} whitelist_externals = {[testenv:packaging]whitelist_externals} commands = {[testenv:packaging]commands} twine upload dist/*