async_generator-1.10/0000755000175000017500000000000013330225256014225 5ustar njsnjs00000000000000async_generator-1.10/LICENSE.MIT0000644000175000017500000000202613230364720015661 0ustar njsnjs00000000000000The MIT License (MIT) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. async_generator-1.10/LICENSE0000644000175000017500000000027613230364720015236 0ustar njsnjs00000000000000This software is made available under the terms of *either* of the licenses found in LICENSE.APACHE2 or LICENSE.MIT. Contributions to trio are made under the terms of *both* these licenses. async_generator-1.10/LICENSE.APACHE20000644000175000017500000002613613230364720016243 0ustar njsnjs00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. async_generator-1.10/async_generator.egg-info/0000755000175000017500000000000013330225256021102 5ustar njsnjs00000000000000async_generator-1.10/async_generator.egg-info/top_level.txt0000644000175000017500000000002013330225256023624 0ustar njsnjs00000000000000async_generator async_generator-1.10/async_generator.egg-info/dependency_links.txt0000644000175000017500000000000113330225256025150 0ustar njsnjs00000000000000 async_generator-1.10/async_generator.egg-info/SOURCES.txt0000644000175000017500000000127413330225256022772 0ustar njsnjs00000000000000.coveragerc CODE_OF_CONDUCT.md CONTRIBUTING.md LICENSE LICENSE.APACHE2 LICENSE.MIT MANIFEST.in README.rst setup.py test-requirements.txt async_generator/__init__.py async_generator/_impl.py async_generator/_util.py async_generator/_version.py async_generator.egg-info/PKG-INFO async_generator.egg-info/SOURCES.txt async_generator.egg-info/dependency_links.txt async_generator.egg-info/top_level.txt async_generator/_tests/__init__.py async_generator/_tests/conftest.py async_generator/_tests/test_async_generator.py async_generator/_tests/test_util.py docs/Makefile docs/make.bat docs/source/conf.py docs/source/history.rst docs/source/index.rst docs/source/reference.rst docs/source/_static/.gitkeepasync_generator-1.10/async_generator.egg-info/PKG-INFO0000644000175000017500000001333113330225256022200 0ustar njsnjs00000000000000Metadata-Version: 1.2 Name: async-generator Version: 1.10 Summary: Async generators and context managers for Python 3.5+ Home-page: https://github.com/python-trio/async_generator Author: Nathaniel J. Smith Author-email: njs@pobox.com License: MIT -or- Apache License 2.0 Description: .. image:: https://img.shields.io/badge/chat-join%20now-blue.svg :target: https://gitter.im/python-trio/general :alt: Join chatroom .. image:: https://img.shields.io/badge/docs-read%20now-blue.svg :target: https://async-generator.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status .. image:: https://travis-ci.org/python-trio/async_generator.svg?branch=master :target: https://travis-ci.org/python-trio/async_generator :alt: Automated test status .. image:: https://ci.appveyor.com/api/projects/status/af4eyed8o8tc3t0r/branch/master?svg=true :target: https://ci.appveyor.com/project/python-trio/trio/history :alt: Automated test status (Windows) .. image:: https://codecov.io/gh/python-trio/async_generator/branch/master/graph/badge.svg :target: https://codecov.io/gh/python-trio/async_generator :alt: Test coverage The async_generator library =========================== Python 3.6 added `async generators `__. (What's an async generator? `Check out my 5-minute lightning talk demo from PyCon 2016 `__.) Python 3.7 adds some more tools to make them usable, like ``contextlib.asynccontextmanager``. This library gives you all that back to Python 3.5. For example, this code only works in Python 3.6+: .. code-block:: python3 async def load_json_lines(stream_reader): async for line in stream_reader: yield json.loads(line) But this code does the same thing, and works on Python 3.5+: .. code-block:: python3 from async_generator import async_generator, yield_ @async_generator async def load_json_lines(stream_reader): async for line in stream_reader: await yield_(json.loads(line)) Or in Python 3.7, you can write: .. code-block:: python3 from contextlib import asynccontextmanager @asynccontextmanager async def background_server(): async with trio.open_nursery() as nursery: value = await nursery.start(my_server) try: yield value finally: # Kill the server when the scope exits nursery.cancel_scope.cancel() This is the same, but back to 3.5: .. code-block:: python3 from async_generator import async_generator, yield_, asynccontextmanager @asynccontextmanager @async_generator async def background_server(): async with trio.open_nursery() as nursery: value = await nursery.start(my_server) try: await yield_(value) finally: # Kill the server when the scope exits nursery.cancel_scope.cancel() (And if you're on 3.6, you can use ``@asynccontextmanager`` with native generators.) Let's do this ============= * Install: ``python3 -m pip install -U async_generator`` (or on Windows, maybe ``py -3 -m pip install -U async_generator`` * Manual: https://async-generator.readthedocs.io/ * Bug tracker and source code: https://github.com/python-trio/async_generator * Real-time chat: https://gitter.im/python-trio/general * License: MIT or Apache 2, your choice * Contributor guide: https://trio.readthedocs.io/en/latest/contributing.html * Code of conduct: Contributors are requested to follow our `code of conduct `__ in all project spaces. How come some of those links talk about "trio"? =============================================== `Trio `__ is a new async concurrency library for Python that's obsessed with usability and correctness – we want to make it *easy* to get things *right*. The ``async_generator`` library is maintained by the Trio project as part of that mission, and because Trio uses ``async_generator`` internally. You can use ``async_generator`` with any async library. It works great with ``asyncio``, or Twisted, or whatever you like. (But we think Trio is pretty sweet.) Keywords: async Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: License :: OSI Approved :: Apache Software License Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Framework :: AsyncIO Requires-Python: >=3.5 async_generator-1.10/async_generator/0000755000175000017500000000000013330225256017410 5ustar njsnjs00000000000000async_generator-1.10/async_generator/_impl.py0000644000175000017500000003735213267320140021071 0ustar njsnjs00000000000000import sys from functools import wraps from types import coroutine import inspect from inspect import ( getcoroutinestate, CORO_CREATED, CORO_CLOSED, CORO_SUSPENDED ) import collections.abc class YieldWrapper: def __init__(self, payload): self.payload = payload def _wrap(value): return YieldWrapper(value) def _is_wrapped(box): return isinstance(box, YieldWrapper) def _unwrap(box): return box.payload # This is the magic code that lets you use yield_ and yield_from_ with native # generators. # # The old version worked great on Linux and MacOS, but not on Windows, because # it depended on _PyAsyncGenValueWrapperNew. The new version segfaults # everywhere, and I'm not sure why -- probably my lack of understanding # of ctypes and refcounts. # # There are also some commented out tests that should be re-enabled if this is # fixed: # # if sys.version_info >= (3, 6): # # Use the same box type that the interpreter uses internally. This allows # # yield_ and (more importantly!) yield_from_ to work in built-in # # generators. # import ctypes # mua ha ha. # # # We used to call _PyAsyncGenValueWrapperNew to create and set up new # # wrapper objects, but that symbol isn't available on Windows: # # # # https://github.com/python-trio/async_generator/issues/5 # # # # Fortunately, the type object is available, but it means we have to do # # this the hard way. # # # We don't actually need to access this, but we need to make a ctypes # # structure so we can call addressof. # class _ctypes_PyTypeObject(ctypes.Structure): # pass # _PyAsyncGenWrappedValue_Type_ptr = ctypes.addressof( # _ctypes_PyTypeObject.in_dll( # ctypes.pythonapi, "_PyAsyncGenWrappedValue_Type")) # _PyObject_GC_New = ctypes.pythonapi._PyObject_GC_New # _PyObject_GC_New.restype = ctypes.py_object # _PyObject_GC_New.argtypes = (ctypes.c_void_p,) # # _Py_IncRef = ctypes.pythonapi.Py_IncRef # _Py_IncRef.restype = None # _Py_IncRef.argtypes = (ctypes.py_object,) # # class _ctypes_PyAsyncGenWrappedValue(ctypes.Structure): # _fields_ = [ # ('PyObject_HEAD', ctypes.c_byte * object().__sizeof__()), # ('agw_val', ctypes.py_object), # ] # def _wrap(value): # box = _PyObject_GC_New(_PyAsyncGenWrappedValue_Type_ptr) # raw = ctypes.cast(ctypes.c_void_p(id(box)), # ctypes.POINTER(_ctypes_PyAsyncGenWrappedValue)) # raw.contents.agw_val = value # _Py_IncRef(value) # return box # # def _unwrap(box): # assert _is_wrapped(box) # raw = ctypes.cast(ctypes.c_void_p(id(box)), # ctypes.POINTER(_ctypes_PyAsyncGenWrappedValue)) # value = raw.contents.agw_val # _Py_IncRef(value) # return value # # _PyAsyncGenWrappedValue_Type = type(_wrap(1)) # def _is_wrapped(box): # return isinstance(box, _PyAsyncGenWrappedValue_Type) # The magic @coroutine decorator is how you write the bottom level of # coroutine stacks -- 'async def' can only use 'await' = yield from; but # eventually we must bottom out in a @coroutine that calls plain 'yield'. @coroutine def _yield_(value): return (yield _wrap(value)) # But we wrap the bare @coroutine version in an async def, because async def # has the magic feature that users can get warnings messages if they forget to # use 'await'. async def yield_(value=None): return await _yield_(value) async def yield_from_(delegate): # Transcribed with adaptations from: # # https://www.python.org/dev/peps/pep-0380/#formal-semantics # # This takes advantage of a sneaky trick: if an @async_generator-wrapped # function calls another async function (like yield_from_), and that # second async function calls yield_, then because of the hack we use to # implement yield_, the yield_ will actually propagate through yield_from_ # back to the @async_generator wrapper. So even though we're a regular # function, we can directly yield values out of the calling async # generator. def unpack_StopAsyncIteration(e): if e.args: return e.args[0] else: return None _i = type(delegate).__aiter__(delegate) if hasattr(_i, "__await__"): _i = await _i try: _y = await type(_i).__anext__(_i) except StopAsyncIteration as _e: _r = unpack_StopAsyncIteration(_e) else: while 1: try: _s = await yield_(_y) except GeneratorExit as _e: try: _m = _i.aclose except AttributeError: pass else: await _m() raise _e except BaseException as _e: _x = sys.exc_info() try: _m = _i.athrow except AttributeError: raise _e else: try: _y = await _m(*_x) except StopAsyncIteration as _e: _r = unpack_StopAsyncIteration(_e) break else: try: if _s is None: _y = await type(_i).__anext__(_i) else: _y = await _i.asend(_s) except StopAsyncIteration as _e: _r = unpack_StopAsyncIteration(_e) break return _r # This is the awaitable / iterator that implements asynciter.__anext__() and # friends. # # Note: we can be sloppy about the distinction between # # type(self._it).__next__(self._it) # # and # # self._it.__next__() # # because we happen to know that self._it is not a general iterator object, # but specifically a coroutine iterator object where these are equivalent. class ANextIter: def __init__(self, it, first_fn, *first_args): self._it = it self._first_fn = first_fn self._first_args = first_args def __await__(self): return self def __next__(self): if self._first_fn is not None: first_fn = self._first_fn first_args = self._first_args self._first_fn = self._first_args = None return self._invoke(first_fn, *first_args) else: return self._invoke(self._it.__next__) def send(self, value): return self._invoke(self._it.send, value) def throw(self, type, value=None, traceback=None): return self._invoke(self._it.throw, type, value, traceback) def _invoke(self, fn, *args): try: result = fn(*args) except StopIteration as e: # The underlying generator returned, so we should signal the end # of iteration. raise StopAsyncIteration(e.value) except StopAsyncIteration as e: # PEP 479 says: if a generator raises Stop(Async)Iteration, then # it should be wrapped into a RuntimeError. Python automatically # enforces this for StopIteration; for StopAsyncIteration we need # to it ourselves. raise RuntimeError( "async_generator raise StopAsyncIteration" ) from e if _is_wrapped(result): raise StopIteration(_unwrap(result)) else: return result UNSPECIFIED = object() try: from sys import get_asyncgen_hooks, set_asyncgen_hooks except ImportError: import threading asyncgen_hooks = collections.namedtuple( "asyncgen_hooks", ("firstiter", "finalizer") ) class _hooks_storage(threading.local): def __init__(self): self.firstiter = None self.finalizer = None _hooks = _hooks_storage() def get_asyncgen_hooks(): return asyncgen_hooks( firstiter=_hooks.firstiter, finalizer=_hooks.finalizer ) def set_asyncgen_hooks(firstiter=UNSPECIFIED, finalizer=UNSPECIFIED): if firstiter is not UNSPECIFIED: if firstiter is None or callable(firstiter): _hooks.firstiter = firstiter else: raise TypeError( "callable firstiter expected, got {}".format( type(firstiter).__name__ ) ) if finalizer is not UNSPECIFIED: if finalizer is None or callable(finalizer): _hooks.finalizer = finalizer else: raise TypeError( "callable finalizer expected, got {}".format( type(finalizer).__name__ ) ) class AsyncGenerator: # https://bitbucket.org/pypy/pypy/issues/2786: # PyPy implements 'await' in a way that requires the frame object # used to execute a coroutine to keep a weakref to that coroutine. # During a GC pass, weakrefs to all doomed objects are broken # before any of the doomed objects' finalizers are invoked. # If an AsyncGenerator is unreachable, its _coroutine probably # is too, and the weakref from ag._coroutine.cr_frame to # ag._coroutine will be broken before ag.__del__ can do its # one-turn close attempt or can schedule a full aclose() using # the registered finalization hook. It doesn't look like the # underlying issue is likely to be fully fixed anytime soon, # so we work around it by preventing an AsyncGenerator and # its _coroutine from being considered newly unreachable at # the same time if the AsyncGenerator's finalizer might want # to iterate the coroutine some more. _pypy_issue2786_workaround = set() def __init__(self, coroutine): self._coroutine = coroutine self._it = coroutine.__await__() self.ag_running = False self._finalizer = None self._closed = False self._hooks_inited = False # On python 3.5.0 and 3.5.1, __aiter__ must be awaitable. # Starting in 3.5.2, it should not be awaitable, and if it is, then it # raises a PendingDeprecationWarning. # See: # https://www.python.org/dev/peps/pep-0492/#api-design-and-implementation-revisions # https://docs.python.org/3/reference/datamodel.html#async-iterators # https://bugs.python.org/issue27243 if sys.version_info < (3, 5, 2): async def __aiter__(self): return self else: def __aiter__(self): return self ################################################################ # Introspection attributes ################################################################ @property def ag_code(self): return self._coroutine.cr_code @property def ag_frame(self): return self._coroutine.cr_frame ################################################################ # Core functionality ################################################################ # These need to return awaitables, rather than being async functions, # to match the native behavior where the firstiter hook is called # immediately on asend()/etc, even if the coroutine that asend() # produces isn't awaited for a bit. def __anext__(self): return self._do_it(self._it.__next__) def asend(self, value): return self._do_it(self._it.send, value) def athrow(self, type, value=None, traceback=None): return self._do_it(self._it.throw, type, value, traceback) def _do_it(self, start_fn, *args): if not self._hooks_inited: self._hooks_inited = True (firstiter, self._finalizer) = get_asyncgen_hooks() if firstiter is not None: firstiter(self) if sys.implementation.name == "pypy": self._pypy_issue2786_workaround.add(self._coroutine) # On CPython 3.5.2 (but not 3.5.0), coroutines get cranky if you try # to iterate them after they're exhausted. Generators OTOH just raise # StopIteration. We want to convert the one into the other, so we need # to avoid iterating stopped coroutines. if getcoroutinestate(self._coroutine) is CORO_CLOSED: raise StopAsyncIteration() async def step(): if self.ag_running: raise ValueError("async generator already executing") try: self.ag_running = True return await ANextIter(self._it, start_fn, *args) except StopAsyncIteration: self._pypy_issue2786_workaround.discard(self._coroutine) raise finally: self.ag_running = False return step() ################################################################ # Cleanup ################################################################ async def aclose(self): state = getcoroutinestate(self._coroutine) if state is CORO_CLOSED or self._closed: return # Make sure that even if we raise "async_generator ignored # GeneratorExit", and thus fail to exhaust the coroutine, # __del__ doesn't complain again. self._closed = True if state is CORO_CREATED: # Make sure that aclose() on an unstarted generator returns # successfully and prevents future iteration. self._it.close() return try: await self.athrow(GeneratorExit) except (GeneratorExit, StopAsyncIteration): self._pypy_issue2786_workaround.discard(self._coroutine) else: raise RuntimeError("async_generator ignored GeneratorExit") def __del__(self): self._pypy_issue2786_workaround.discard(self._coroutine) if getcoroutinestate(self._coroutine) is CORO_CREATED: # Never started, nothing to clean up, just suppress the "coroutine # never awaited" message. self._coroutine.close() if getcoroutinestate(self._coroutine ) is CORO_SUSPENDED and not self._closed: if self._finalizer is not None: self._finalizer(self) else: # Mimic the behavior of native generators on GC with no finalizer: # throw in GeneratorExit, run for one turn, and complain if it didn't # finish. thrower = self.athrow(GeneratorExit) try: thrower.send(None) except (GeneratorExit, StopAsyncIteration): pass except StopIteration: raise RuntimeError("async_generator ignored GeneratorExit") else: raise RuntimeError( "async_generator {!r} awaited during finalization; install " "a finalization hook to support this, or wrap it in " "'async with aclosing(...):'" .format(self.ag_code.co_name) ) finally: thrower.close() if hasattr(collections.abc, "AsyncGenerator"): collections.abc.AsyncGenerator.register(AsyncGenerator) def async_generator(coroutine_maker): @wraps(coroutine_maker) def async_generator_maker(*args, **kwargs): return AsyncGenerator(coroutine_maker(*args, **kwargs)) async_generator_maker._async_gen_function = id(async_generator_maker) return async_generator_maker def isasyncgen(obj): if hasattr(inspect, "isasyncgen"): if inspect.isasyncgen(obj): return True return isinstance(obj, AsyncGenerator) def isasyncgenfunction(obj): if hasattr(inspect, "isasyncgenfunction"): if inspect.isasyncgenfunction(obj): return True return getattr(obj, "_async_gen_function", -1) == id(obj) async_generator-1.10/async_generator/_version.py0000644000175000017500000000002513330225237021602 0ustar njsnjs00000000000000__version__ = "1.10" async_generator-1.10/async_generator/__init__.py0000644000175000017500000000070613267320140021521 0ustar njsnjs00000000000000from ._version import __version__ from ._impl import ( async_generator, yield_, yield_from_, isasyncgen, isasyncgenfunction, get_asyncgen_hooks, set_asyncgen_hooks, ) from ._util import aclosing, asynccontextmanager __all__ = [ "async_generator", "yield_", "yield_from_", "aclosing", "isasyncgen", "isasyncgenfunction", "asynccontextmanager", "get_asyncgen_hooks", "set_asyncgen_hooks", ] async_generator-1.10/async_generator/_tests/0000755000175000017500000000000013330225256020711 5ustar njsnjs00000000000000async_generator-1.10/async_generator/_tests/conftest.py0000644000175000017500000000227313230364720023113 0ustar njsnjs00000000000000import pytest from functools import wraps, partial import inspect import types @types.coroutine def mock_sleep(): yield "mock_sleep" # Wrap any 'async def' tests so that they get automatically iterated. # We used to use pytest-asyncio as a convenient way to do this, but nowadays # pytest-asyncio uses us! In addition to it being generally bad for our test # infrastructure to depend on the code-under-test, this totally messes up # coverage info because depending on pytest's plugin load order, we might get # imported before pytest-cov can be loaded and start gathering coverage. @pytest.hookimpl(tryfirst=True) def pytest_pyfunc_call(pyfuncitem): if inspect.iscoroutinefunction(pyfuncitem.obj): fn = pyfuncitem.obj @wraps(fn) def wrapper(**kwargs): coro = fn(**kwargs) try: while True: value = coro.send(None) if value != "mock_sleep": # pragma: no cover raise RuntimeError( "coroutine runner confused: {!r}".format(value) ) except StopIteration: pass pyfuncitem.obj = wrapper async_generator-1.10/async_generator/_tests/test_async_generator.py0000644000175000017500000006644013267320140025514 0ustar njsnjs00000000000000import pytest import types import sys import collections.abc from functools import wraps import gc from .conftest import mock_sleep from .. import ( async_generator, yield_, yield_from_, isasyncgen, isasyncgenfunction, get_asyncgen_hooks, set_asyncgen_hooks, ) # like list(it) but works on async iterators async def collect(ait): items = [] async for value in ait: items.append(value) return items ################################################################ # # Basic test # ################################################################ @async_generator async def async_range(count): for i in range(count): print("Calling yield_({})".format(i)) await yield_(i) @async_generator async def double(ait): async for value in ait: await yield_(value * 2) await mock_sleep() class HasAsyncGenMethod: def __init__(self, factor): self._factor = factor @async_generator async def async_multiplied(self, ait): async for value in ait: await yield_(value * self._factor) async def test_async_generator(): assert await collect(async_range(10)) == list(range(10)) assert (await collect(double(async_range(5))) == [0, 2, 4, 6, 8]) tripler = HasAsyncGenMethod(3) assert ( await collect(tripler.async_multiplied(async_range(5))) == [0, 3, 6, 9, 12] ) @async_generator async def agen_yield_no_arg(): await yield_() async def test_yield_no_arg(): assert await collect(agen_yield_no_arg()) == [None] ################################################################ # # async_generators return value # ################################################################ @async_generator async def async_gen_with_non_None_return(): await yield_(1) await yield_(2) return "hi" async def test_bad_return_value(): gen = async_gen_with_non_None_return() async for item in gen: # pragma: no branch assert item == 1 break async for item in gen: # pragma: no branch assert item == 2 break try: await gen.__anext__() except StopAsyncIteration as e: assert e.args[0] == "hi" ################################################################ # # Exhausitve tests of the different ways to re-enter a coroutine. # # It used to be that re-entering via send/__next__ would work, but throw() # immediately followed by an await yield_(...) wouldn't work, and the # YieldWrapper object would propagate back out to the coroutine runner. # # Before I fixed this, the 'assert value is None' check below would fail # (because of the YieldWrapper leaking out), and if you removed that # assertion, then the code would appear to run successfully but the final list # would just be [1, 3] instead of [1, 2, 3]. # ################################################################ class MyTestError(Exception): pass # This unconditionally raises a MyTestError exception, so from the outside # it's equivalent to a simple 'raise MyTestError`. But, for this test to check # the thing we want it to check, the point is that the exception must be # thrown in from the coroutine runner -- this simulates something like an # 'await sock.recv(...) -> TimeoutError'. @types.coroutine def hit_me(): yield "hit me" @types.coroutine def number_me(): assert (yield "number me") == 1 @types.coroutine def next_me(): assert (yield "next me") is None @async_generator async def yield_after_different_entries(): await yield_(1) try: await hit_me() except MyTestError: await yield_(2) await number_me() await yield_(3) await next_me() await yield_(4) def hostile_coroutine_runner(coro): coro_iter = coro.__await__() value = None while True: try: if value == "hit me": value = coro_iter.throw(MyTestError()) elif value == "number me": value = coro_iter.send(1) else: assert value in (None, "next me") value = coro_iter.__next__() except StopIteration as exc: return exc.value def test_yield_different_entries(): coro = collect(yield_after_different_entries()) yielded = hostile_coroutine_runner(coro) assert yielded == [1, 2, 3, 4] async def test_reentrance_forbidden(): @async_generator async def recurse(): async for obj in agen: # pragma: no branch await yield_(obj) # pragma: no cover agen = recurse() with pytest.raises(ValueError): async for _ in agen: # pragma: no branch pass # pragma: no cover async def test_reentrance_forbidden_simultaneous_asends(): @async_generator async def f(): await mock_sleep() ag = f() sender1 = ag.asend(None) sender2 = ag.asend(None) assert sender1.send(None) == "mock_sleep" with pytest.raises(ValueError): sender2.send(None) with pytest.raises(StopAsyncIteration): sender1.send(None) await ag.aclose() # https://bugs.python.org/issue32526 async def test_reentrance_forbidden_while_suspended_in_coroutine_runner(): @async_generator async def f(): await mock_sleep() await yield_("final yield") ag = f() asend_coro = ag.asend(None) fut = asend_coro.send(None) assert fut == "mock_sleep" # Now the async generator's frame is not executing, but a call to asend() # *is* executing. Make sure that in this case, ag_running is True, and we # can't start up another call to asend(). assert ag.ag_running with pytest.raises(ValueError): await ag.asend(None) # Clean up with pytest.raises(StopIteration): asend_coro.send(None) with pytest.raises(StopAsyncIteration): ag.asend(None).send(None) ################################################################ # # asend # ################################################################ @async_generator async def asend_me(): assert (await yield_(1)) == 2 assert (await yield_(3)) == 4 async def test_asend(): aiter = asend_me() assert (await aiter.__anext__()) == 1 assert (await aiter.asend(2)) == 3 with pytest.raises(StopAsyncIteration): await aiter.asend(4) ################################################################ # # athrow # ################################################################ @async_generator async def athrow_me(): with pytest.raises(KeyError): await yield_(1) with pytest.raises(ValueError): await yield_(2) await yield_(3) async def test_athrow(): aiter = athrow_me() assert (await aiter.__anext__()) == 1 assert (await aiter.athrow(KeyError("oops"))) == 2 assert (await aiter.athrow(ValueError("oops"))) == 3 with pytest.raises(OSError): await aiter.athrow(OSError("oops")) ################################################################ # # aclose # ################################################################ @async_generator async def close_me_aiter(track): try: await yield_(1) except GeneratorExit: track[0] = "closed" raise else: # pragma: no cover track[0] = "wtf" async def test_aclose(): track = [None] aiter = close_me_aiter(track) async for obj in aiter: # pragma: no branch assert obj == 1 break assert track[0] is None await aiter.aclose() assert track[0] == "closed" async def test_aclose_on_unstarted_generator(): aiter = close_me_aiter([None]) await aiter.aclose() async for obj in aiter: assert False # pragma: no cover async def test_aclose_on_finished_generator(): aiter = async_range(3) async for obj in aiter: pass # pragma: no cover await aiter.aclose() @async_generator async def sync_yield_during_aclose(): try: await yield_(1) finally: await mock_sleep() @async_generator async def async_yield_during_aclose(): try: await yield_(1) finally: await yield_(2) async def test_aclose_yielding(): aiter = sync_yield_during_aclose() assert (await aiter.__anext__()) == 1 # Doesn't raise: await aiter.aclose() aiter = async_yield_during_aclose() assert (await aiter.__anext__()) == 1 with pytest.raises(RuntimeError): await aiter.aclose() ################################################################ # # yield from # ################################################################ @async_generator async def async_range_twice(count): await yield_from_(async_range(count)) await yield_(None) await yield_from_(async_range(count)) if sys.version_info >= (3, 6): exec( """ async def native_async_range(count): for i in range(count): yield i # XX uncomment if/when we re-enable the ctypes hacks: # async def native_async_range_twice(count): # # make sure yield_from_ works inside a native async generator # await yield_from_(async_range(count)) # yield None # # make sure we can yield_from_ a native async generator # await yield_from_(native_async_range(count)) """ ) async def test_async_yield_from_(): assert await collect(async_range_twice(3)) == [ 0, 1, 2, None, 0, 1, 2, ] if sys.version_info >= (3, 6): # Make sure we can yield_from_ a native generator @async_generator async def yield_from_native(): await yield_from_(native_async_range(3)) assert await collect(yield_from_native()) == [0, 1, 2] # XX uncomment if/when we re-enable the ctypes hacks: # if sys.version_info >= (3, 6): # assert await collect(native_async_range_twice(3)) == [ # 0, 1, 2, None, 0, 1, 2, # ] @async_generator async def doubles_sends(value): while True: value = await yield_(2 * value) @async_generator async def wraps_doubles_sends(value): await yield_from_(doubles_sends(value)) async def test_async_yield_from_asend(): gen = wraps_doubles_sends(10) await gen.__anext__() == 20 assert (await gen.asend(2)) == 4 assert (await gen.asend(5)) == 10 assert (await gen.asend(0)) == 0 await gen.aclose() async def test_async_yield_from_athrow(): gen = async_range_twice(2) assert (await gen.__anext__()) == 0 with pytest.raises(ValueError): await gen.athrow(ValueError) @async_generator async def returns_1(): await yield_(0) return 1 @async_generator async def yields_from_returns_1(): await yield_(await yield_from_(returns_1())) async def test_async_yield_from_return_value(): assert await collect(yields_from_returns_1()) == [0, 1] # Special cases to get coverage async def test_yield_from_empty(): @async_generator async def empty(): return "done" @async_generator async def yield_from_empty(): assert (await yield_from_(empty())) == "done" assert await collect(yield_from_empty()) == [] async def test_yield_from_non_generator(): class Countdown: def __init__(self, count): self.count = count self.closed = False if sys.version_info < (3, 5, 2): async def __aiter__(self): return self else: def __aiter__(self): return self async def __anext__(self): self.count -= 1 if self.count < 0: raise StopAsyncIteration("boom") return self.count async def aclose(self): self.closed = True @async_generator async def yield_from_countdown(count, happenings): try: c = Countdown(count) assert (await yield_from_(c)) == "boom" except BaseException as e: if c.closed: happenings.append("countdown closed") happenings.append("raise") return e h = [] assert await collect(yield_from_countdown(3, h)) == [2, 1, 0] assert h == [] # Throwing into a yield_from_(object with no athrow) just raises the # exception in the generator. h = [] agen = yield_from_countdown(3, h) assert await agen.__anext__() == 2 exc = ValueError("x") try: await agen.athrow(exc) except StopAsyncIteration as e: assert e.args[0] is exc assert h == ["raise"] # Calling aclose on the generator calls aclose on the iterator h = [] agen = yield_from_countdown(3, h) assert await agen.__anext__() == 2 await agen.aclose() assert h == ["countdown closed", "raise"] # Throwing GeneratorExit into the generator calls *aclose* on the iterator # (!) h = [] agen = yield_from_countdown(3, h) assert await agen.__anext__() == 2 exc = GeneratorExit() with pytest.raises(StopAsyncIteration): await agen.athrow(exc) assert h == ["countdown closed", "raise"] async def test_yield_from_non_generator_with_no_aclose(): class Countdown: def __init__(self, count): self.count = count self.closed = False if sys.version_info < (3, 5, 2): async def __aiter__(self): return self else: def __aiter__(self): return self async def __anext__(self): self.count -= 1 if self.count < 0: raise StopAsyncIteration("boom") return self.count @async_generator async def yield_from_countdown(count): return await yield_from_(Countdown(count)) assert await collect(yield_from_countdown(3)) == [2, 1, 0] agen = yield_from_countdown(3) assert await agen.__anext__() == 2 assert await agen.__anext__() == 1 # It's OK that Countdown has no aclose await agen.aclose() async def test_yield_from_with_old_style_aiter(): # old-style 'async def __aiter__' should still work even on newer pythons class Countdown: def __init__(self, count): self.count = count self.closed = False # This is wrong, that's the point async def __aiter__(self): return self async def __anext__(self): self.count -= 1 if self.count < 0: raise StopAsyncIteration("boom") return self.count @async_generator async def yield_from_countdown(count): return await yield_from_(Countdown(count)) assert await collect(yield_from_countdown(3)) == [2, 1, 0] async def test_yield_from_athrow_raises_StopAsyncIteration(): @async_generator async def catch(): try: while True: await yield_("hi") except Exception as exc: return ("bye", exc) @async_generator async def yield_from_catch(): return await yield_from_(catch()) agen = yield_from_catch() assert await agen.__anext__() == "hi" assert await agen.__anext__() == "hi" thrown = ValueError("oops") try: print(await agen.athrow(thrown)) except StopAsyncIteration as caught: assert caught.args == (("bye", thrown),) else: raise AssertionError # pragma: no cover ################################################################ # __del__ ################################################################ async def test___del__(capfd): completions = 0 @async_generator async def awaits_when_unwinding(): await yield_(0) try: await yield_(1) finally: await mock_sleep() try: await yield_(2) finally: nonlocal completions completions += 1 gen = awaits_when_unwinding() # Hasn't started yet, so no problem gen.__del__() gen = awaits_when_unwinding() assert await collect(gen) == [0, 1, 2] # Exhausted, so no problem gen.__del__() for stop_after_turn in (1, 2, 3): gen = awaits_when_unwinding() for turn in range(stop_after_turn): assert await gen.__anext__() == turn await gen.aclose() # Closed, so no problem gen.__del__() for stop_after_turn in (1, 2, 3): gen = awaits_when_unwinding() for turn in range(stop_after_turn): assert await gen.__anext__() == turn if stop_after_turn == 2: # Stopped in the middle of a try/finally that awaits in the finally, # so __del__ can't cleanup. with pytest.raises(RuntimeError) as info: gen.__del__() assert "awaited during finalization; install a finalization hook" in str( info.value ) else: # Can clean up without awaiting, so __del__ is fine gen.__del__() assert completions == 3 @async_generator async def yields_when_unwinding(): try: await yield_(1) finally: await yield_(2) gen = yields_when_unwinding() assert await gen.__anext__() == 1 with pytest.raises(RuntimeError) as info: gen.__del__() ################################################################ # introspection ################################################################ def test_isasyncgen(): assert not isasyncgen(async_range) assert isasyncgen(async_range(10)) if sys.version_info >= (3, 6): assert not isasyncgen(native_async_range) assert isasyncgen(native_async_range(10)) def test_isasyncgenfunction(): assert isasyncgenfunction(async_range) assert not isasyncgenfunction(list) assert not isasyncgenfunction(async_range(10)) if sys.version_info >= (3, 6): assert isasyncgenfunction(native_async_range) assert not isasyncgenfunction(native_async_range(10)) # Very subtle bug: functools.wraps copies across the entire contents of the # wrapped function's __dict__. We used to use a simple _is_async_gen=True # attribute to mark async generators. But if we do that, then simple wrappers # like async_range_wrapper *do* return True for isasyncgenfunction. But that's # not how inspect.isasyncgenfunction works, and it also caused problems for # sphinxcontrib-trio, because given a function like: # # @acontextmanager # @async_generator # async def async_cm(): # ... # # then we end up with async_cm introspecting as both an async context manager # and an async generator, and it doesn't know who to believe. With the # correct, inspect.isasyncgenfunction-compliant behavior, we have async_cm # introspecting as an async context manager, and async_cm.__wrapped__ # introspecting as an async generator. def test_isasyncgenfunction_is_not_inherited_by_wrappers(): @wraps(async_range) def async_range_wrapper(*args, **kwargs): # pragma: no cover return async_range(*args, **kwargs) assert not isasyncgenfunction(async_range_wrapper) assert isasyncgenfunction(async_range_wrapper.__wrapped__) def test_collections_abc_AsyncGenerator(): if hasattr(collections.abc, "AsyncGenerator"): assert isinstance(async_range(10), collections.abc.AsyncGenerator) async def test_ag_attributes(): @async_generator async def f(): x = 1 await yield_() agen = f() assert agen.ag_code.co_name == "f" async for _ in agen: # pragma: no branch assert agen.ag_frame.f_locals["x"] == 1 break ################################################################ # Finicky tests to check that the overly clever ctype stuff has plausible # refcounting from .. import _impl @pytest.mark.skipif(not hasattr(sys, "getrefcount"), reason="CPython only") def test_refcnt(): x = object() print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) base_count = sys.getrefcount(x) l = [_impl._wrap(x) for _ in range(100)] print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) assert sys.getrefcount(x) >= base_count + 100 l2 = [_impl._unwrap(box) for box in l] assert sys.getrefcount(x) >= base_count + 200 print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) del l print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) del l2 print(sys.getrefcount(x)) print(sys.getrefcount(x)) print(sys.getrefcount(x)) assert sys.getrefcount(x) == base_count print(sys.getrefcount(x)) ################################################################ # # Edge cases # ################################################################ # PEP 479: StopIteration or StopAsyncIteration exiting from inside an async # generator should produce a RuntimeError with the __cause__ set to the # original exception. Note that contextlib.asynccontextmanager depends on this # behavior. @async_generator async def lets_exception_out(): await yield_() async def test_throw_StopIteration_or_StopAsyncIteration(): for cls in [StopIteration, StopAsyncIteration]: agen = lets_exception_out() await agen.asend(None) exc = cls() with pytest.raises(RuntimeError) as excinfo: await agen.athrow(exc) assert excinfo.type is RuntimeError assert excinfo.value.__cause__ is exc # No "coroutine was never awaited" warnings for async generators that are not # iterated async def test_no_spurious_unawaited_coroutine_warning(recwarn): agen = async_range(10) del agen # Run collection a few times to make sure any # loops/resurrection/etc. stuff gets fully handled (necessary on pypy) for _ in range(4): gc.collect() # I've seen DeprecationWarnings here triggered by pytest-asyncio, so let's # filter for RuntimeWarning. But if there are no warnings at all, then # that's OK too, so tell coverage not to worry about it. for msg in recwarn: # pragma: no cover print(msg) assert not issubclass(msg.category, RuntimeWarning) ################################################################ # # GC hooks # ################################################################ @pytest.fixture def local_asyncgen_hooks(): old_hooks = get_asyncgen_hooks() yield set_asyncgen_hooks(*old_hooks) def test_gc_hooks_interface(local_asyncgen_hooks): def one(agen): # pragma: no cover pass def two(agen): # pragma: no cover pass set_asyncgen_hooks(None, None) assert get_asyncgen_hooks() == (None, None) set_asyncgen_hooks(finalizer=two) assert get_asyncgen_hooks() == (None, two) set_asyncgen_hooks(firstiter=one) assert get_asyncgen_hooks() == (one, two) set_asyncgen_hooks(finalizer=None, firstiter=two) assert get_asyncgen_hooks() == (two, None) set_asyncgen_hooks(None, one) assert get_asyncgen_hooks() == (None, one) tup = (one, two) set_asyncgen_hooks(*tup) assert get_asyncgen_hooks() == tup with pytest.raises(TypeError): set_asyncgen_hooks(firstiter=42) with pytest.raises(TypeError): set_asyncgen_hooks(finalizer=False) def in_thread(results): results.append(get_asyncgen_hooks()) set_asyncgen_hooks(two, one) results.append(get_asyncgen_hooks()) from threading import Thread results = [] thread = Thread(target=in_thread, args=(results,)) thread.start() thread.join() assert results == [(None, None), (two, one)] assert get_asyncgen_hooks() == (one, two) async def test_gc_hooks_behavior(local_asyncgen_hooks): events = [] to_finalize = [] def firstiter(agen): events.append("firstiter {}".format(agen.ag_frame.f_locals["ident"])) def finalizer(agen): events.append("finalizer {}".format(agen.ag_frame.f_locals["ident"])) to_finalize.append(agen) @async_generator async def agen(ident): events.append("yield 1 {}".format(ident)) await yield_(1) try: events.append("yield 2 {}".format(ident)) await yield_(2) events.append("after yield 2 {}".format(ident)) finally: events.append("mock_sleep {}".format(ident)) await mock_sleep() try: events.append("yield 3 {}".format(ident)) await yield_(3) finally: events.append("unwind 3 {}".format(ident)) # this one is included to make sure we _don't_ execute it events.append("done {}".format(ident)) # pragma: no cover async def anext_verbosely(iter, ident): events.append("before asend {}".format(ident)) sender = iter.asend(None) events.append("before send {}".format(ident)) await sender events.append("after asend {}".format(ident)) # Ensure that firstiter is called immediately on asend(), # before the first turn of the coroutine that asend() returns, # to match the behavior of native generators. # Ensure that the firstiter that gets used is the one in effect # at the time of that first call, rather than at the time of iteration. iterA = agen("A") iterB = agen("B") await anext_verbosely(iterA, "A") set_asyncgen_hooks(firstiter, finalizer) await anext_verbosely(iterB, "B") iterC = agen("C") await anext_verbosely(iterC, "C") assert events == [ "before asend A", "before send A", "yield 1 A", "after asend A", "before asend B", "firstiter B", "before send B", "yield 1 B", "after asend B", "before asend C", "firstiter C", "before send C", "yield 1 C", "after asend C" ] del events[:] # Ensure that firstiter is only called once, even if we create # two asend() coroutines before iterating either of them. iterX = agen("X") sender1 = iterX.asend(None) sender2 = iterX.asend(None) events.append("before close") sender1.close() sender2.close() await iterX.aclose() assert events == ["firstiter X", "before close"] del events[:] from weakref import ref refA, refB, refC = map(ref, (iterA, iterB, iterC)) # iterA uses the finalizer that was in effect when it started, i.e. no finalizer await iterA.__anext__() await iterA.__anext__() del iterA # Do multiple GC passes since we're deliberately shielding the # coroutine objects from the first pass due to PyPy issue 2786. for _ in range(4): gc.collect() assert refA() is None assert events == [ "yield 2 A", "after yield 2 A", "mock_sleep A", "yield 3 A", "unwind 3 A" ] assert not to_finalize del events[:] # iterB and iterC do use our finalizer await iterC.__anext__() await iterB.__anext__() await iterC.__anext__() idB, idC = id(iterB), id(iterC) del iterB for _ in range(4): gc.collect() del iterC for _ in range(4): gc.collect() assert events == [ "yield 2 C", "yield 2 B", "after yield 2 C", "mock_sleep C", "yield 3 C", "finalizer B", "finalizer C" ] del events[:] # finalizer invokes aclose() is not called again once the revived reference drops assert list(map(id, to_finalize)) == [idB, idC] events.append("before aclose B") await to_finalize[0].aclose() events.append("before aclose C") await to_finalize[1].aclose() events.append("after aclose both") del to_finalize[:] for _ in range(4): gc.collect() assert refB() is None and refC() is None assert events == [ "before aclose B", "mock_sleep B", "before aclose C", "unwind 3 C", "after aclose both" ] async_generator-1.10/async_generator/_tests/__init__.py0000644000175000017500000000000013230364720023007 0ustar njsnjs00000000000000async_generator-1.10/async_generator/_tests/test_util.py0000644000175000017500000001434513230364720023305 0ustar njsnjs00000000000000import pytest from .. import aclosing, async_generator, yield_, asynccontextmanager @async_generator async def async_range(count, closed_slot): try: for i in range(count): # pragma: no branch await yield_(i) except GeneratorExit: closed_slot[0] = True async def test_aclosing(): closed_slot = [False] async with aclosing(async_range(10, closed_slot)) as gen: it = iter(range(10)) async for item in gen: # pragma: no branch assert item == next(it) if item == 4: break assert closed_slot[0] closed_slot = [False] try: async with aclosing(async_range(10, closed_slot)) as gen: it = iter(range(10)) async for item in gen: # pragma: no branch assert item == next(it) if item == 4: raise ValueError() except ValueError: pass assert closed_slot[0] async def test_contextmanager_do_not_unchain_non_stopiteration_exceptions(): @asynccontextmanager @async_generator async def manager_issue29692(): try: await yield_() except Exception as exc: raise RuntimeError('issue29692:Chained') from exc with pytest.raises(RuntimeError) as excinfo: async with manager_issue29692(): raise ZeroDivisionError assert excinfo.value.args[0] == 'issue29692:Chained' assert isinstance(excinfo.value.__cause__, ZeroDivisionError) # This is a little funky because of implementation details in # async_generator It can all go away once we stop supporting Python3.5 with pytest.raises(RuntimeError) as excinfo: async with manager_issue29692(): exc = StopIteration('issue29692:Unchained') raise exc assert excinfo.value.args[0] == 'issue29692:Chained' cause = excinfo.value.__cause__ assert cause.args[0] == 'generator raised StopIteration' assert cause.__cause__ is exc with pytest.raises(StopAsyncIteration) as excinfo: async with manager_issue29692(): raise StopAsyncIteration('issue29692:Unchained') assert excinfo.value.args[0] == 'issue29692:Unchained' assert excinfo.value.__cause__ is None @asynccontextmanager @async_generator async def noop_async_context_manager(): await yield_() with pytest.raises(StopIteration): async with noop_async_context_manager(): raise StopIteration # Native async generators are only available from Python 3.6 and onwards nativeasyncgenerators = True try: exec( """ @asynccontextmanager async def manager_issue29692_2(): try: yield except Exception as exc: raise RuntimeError('issue29692:Chained') from exc """ ) except SyntaxError: nativeasyncgenerators = False @pytest.mark.skipif( not nativeasyncgenerators, reason="Python < 3.6 doesn't have native async generators" ) async def test_native_contextmanager_do_not_unchain_non_stopiteration_exceptions( ): with pytest.raises(RuntimeError) as excinfo: async with manager_issue29692_2(): raise ZeroDivisionError assert excinfo.value.args[0] == 'issue29692:Chained' assert isinstance(excinfo.value.__cause__, ZeroDivisionError) for cls in [StopIteration, StopAsyncIteration]: with pytest.raises(cls) as excinfo: async with manager_issue29692_2(): raise cls('issue29692:Unchained') assert excinfo.value.args[0] == 'issue29692:Unchained' assert excinfo.value.__cause__ is None async def test_asynccontextmanager_exception_passthrough(): # This was the cause of annoying coverage flapping, see gh-140 @asynccontextmanager @async_generator async def noop_async_context_manager(): await yield_() for exc_type in [StopAsyncIteration, RuntimeError, ValueError]: with pytest.raises(exc_type): async with noop_async_context_manager(): raise exc_type # And let's also check a boring nothing pass-through while we're at it async with noop_async_context_manager(): pass async def test_asynccontextmanager_catches_exception(): @asynccontextmanager @async_generator async def catch_it(): with pytest.raises(ValueError): await yield_() async with catch_it(): raise ValueError async def test_asynccontextmanager_different_exception(): @asynccontextmanager @async_generator async def switch_it(): try: await yield_() except KeyError: raise ValueError with pytest.raises(ValueError): async with switch_it(): raise KeyError async def test_asynccontextmanager_nice_message_on_sync_enter(): @asynccontextmanager @async_generator async def xxx(): # pragma: no cover await yield_() cm = xxx() with pytest.raises(RuntimeError) as excinfo: with cm: pass # pragma: no cover assert "async with" in str(excinfo.value) async with cm: pass async def test_asynccontextmanager_no_yield(): @asynccontextmanager @async_generator async def yeehaw(): pass with pytest.raises(RuntimeError) as excinfo: async with yeehaw(): assert False # pragma: no cover assert "didn't yield" in str(excinfo.value) async def test_asynccontextmanager_too_many_yields(): closed_count = 0 @asynccontextmanager @async_generator async def doubleyield(): try: await yield_() except Exception: pass try: await yield_() finally: nonlocal closed_count closed_count += 1 with pytest.raises(RuntimeError) as excinfo: async with doubleyield(): pass assert "didn't stop" in str(excinfo.value) assert closed_count == 1 with pytest.raises(RuntimeError) as excinfo: async with doubleyield(): raise ValueError assert "didn't stop after athrow" in str(excinfo.value) assert closed_count == 2 async def test_asynccontextmanager_requires_asyncgenfunction(): with pytest.raises(TypeError): @asynccontextmanager def syncgen(): # pragma: no cover yield async_generator-1.10/async_generator/_util.py0000644000175000017500000001040613230364720021076 0ustar njsnjs00000000000000import sys from functools import wraps from ._impl import isasyncgenfunction class aclosing: def __init__(self, aiter): self._aiter = aiter async def __aenter__(self): return self._aiter async def __aexit__(self, *args): await self._aiter.aclose() # Very much derived from the one in contextlib, by copy/pasting and then # asyncifying everything. (Also I dropped the obscure support for using # context managers as function decorators. It could be re-added; I just # couldn't be bothered.) # So this is a derivative work licensed under the PSF License, which requires # the following notice: # # Copyright © 2001-2017 Python Software Foundation; All Rights Reserved class _AsyncGeneratorContextManager: def __init__(self, func, args, kwds): self._func_name = func.__name__ self._agen = func(*args, **kwds).__aiter__() async def __aenter__(self): if sys.version_info < (3, 5, 2): self._agen = await self._agen try: return await self._agen.asend(None) except StopAsyncIteration: raise RuntimeError("async generator didn't yield") from None async def __aexit__(self, type, value, traceback): async with aclosing(self._agen): if type is None: try: await self._agen.asend(None) except StopAsyncIteration: return False else: raise RuntimeError("async generator didn't stop") else: # It used to be possible to have type != None, value == None: # https://bugs.python.org/issue1705170 # but AFAICT this can't happen anymore. assert value is not None try: await self._agen.athrow(type, value, traceback) raise RuntimeError( "async generator didn't stop after athrow()" ) except StopAsyncIteration as exc: # Suppress StopIteration *unless* it's the same exception # that was passed to throw(). This prevents a # StopIteration raised inside the "with" statement from # being suppressed. return (exc is not value) except RuntimeError as exc: # Don't re-raise the passed in exception. (issue27112) if exc is value: return False # Likewise, avoid suppressing if a StopIteration exception # was passed to throw() and later wrapped into a # RuntimeError (see PEP 479). if (isinstance(value, (StopIteration, StopAsyncIteration)) and exc.__cause__ is value): return False raise except: # only re-raise if it's *not* the exception that was # passed to throw(), because __exit__() must not raise an # exception unless __exit__() itself failed. But throw() # has to raise the exception to signal propagation, so # this fixes the impedance mismatch between the throw() # protocol and the __exit__() protocol. # if sys.exc_info()[1] is value: return False raise def __enter__(self): raise RuntimeError( "use 'async with {func_name}(...)', not 'with {func_name}(...)'". format(func_name=self._func_name) ) def __exit__(self): # pragma: no cover assert False, """Never called, but should be defined""" def asynccontextmanager(func): """Like @contextmanager, but async.""" if not isasyncgenfunction(func): raise TypeError( "must be an async generator (native or from async_generator; " "if using @async_generator then @acontextmanager must be on top." ) @wraps(func) def helper(*args, **kwds): return _AsyncGeneratorContextManager(func, args, kwds) # A hint for sphinxcontrib-trio: helper.__returns_acontextmanager__ = True return helper async_generator-1.10/README.rst0000644000175000017500000000763313267320140015722 0ustar njsnjs00000000000000.. image:: https://img.shields.io/badge/chat-join%20now-blue.svg :target: https://gitter.im/python-trio/general :alt: Join chatroom .. image:: https://img.shields.io/badge/docs-read%20now-blue.svg :target: https://async-generator.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status .. image:: https://travis-ci.org/python-trio/async_generator.svg?branch=master :target: https://travis-ci.org/python-trio/async_generator :alt: Automated test status .. image:: https://ci.appveyor.com/api/projects/status/af4eyed8o8tc3t0r/branch/master?svg=true :target: https://ci.appveyor.com/project/python-trio/trio/history :alt: Automated test status (Windows) .. image:: https://codecov.io/gh/python-trio/async_generator/branch/master/graph/badge.svg :target: https://codecov.io/gh/python-trio/async_generator :alt: Test coverage The async_generator library =========================== Python 3.6 added `async generators `__. (What's an async generator? `Check out my 5-minute lightning talk demo from PyCon 2016 `__.) Python 3.7 adds some more tools to make them usable, like ``contextlib.asynccontextmanager``. This library gives you all that back to Python 3.5. For example, this code only works in Python 3.6+: .. code-block:: python3 async def load_json_lines(stream_reader): async for line in stream_reader: yield json.loads(line) But this code does the same thing, and works on Python 3.5+: .. code-block:: python3 from async_generator import async_generator, yield_ @async_generator async def load_json_lines(stream_reader): async for line in stream_reader: await yield_(json.loads(line)) Or in Python 3.7, you can write: .. code-block:: python3 from contextlib import asynccontextmanager @asynccontextmanager async def background_server(): async with trio.open_nursery() as nursery: value = await nursery.start(my_server) try: yield value finally: # Kill the server when the scope exits nursery.cancel_scope.cancel() This is the same, but back to 3.5: .. code-block:: python3 from async_generator import async_generator, yield_, asynccontextmanager @asynccontextmanager @async_generator async def background_server(): async with trio.open_nursery() as nursery: value = await nursery.start(my_server) try: await yield_(value) finally: # Kill the server when the scope exits nursery.cancel_scope.cancel() (And if you're on 3.6, you can use ``@asynccontextmanager`` with native generators.) Let's do this ============= * Install: ``python3 -m pip install -U async_generator`` (or on Windows, maybe ``py -3 -m pip install -U async_generator`` * Manual: https://async-generator.readthedocs.io/ * Bug tracker and source code: https://github.com/python-trio/async_generator * Real-time chat: https://gitter.im/python-trio/general * License: MIT or Apache 2, your choice * Contributor guide: https://trio.readthedocs.io/en/latest/contributing.html * Code of conduct: Contributors are requested to follow our `code of conduct `__ in all project spaces. How come some of those links talk about "trio"? =============================================== `Trio `__ is a new async concurrency library for Python that's obsessed with usability and correctness – we want to make it *easy* to get things *right*. The ``async_generator`` library is maintained by the Trio project as part of that mission, and because Trio uses ``async_generator`` internally. You can use ``async_generator`` with any async library. It works great with ``asyncio``, or Twisted, or whatever you like. (But we think Trio is pretty sweet.) async_generator-1.10/setup.cfg0000644000175000017500000000004613330225256016046 0ustar njsnjs00000000000000[egg_info] tag_build = tag_date = 0 async_generator-1.10/MANIFEST.in0000644000175000017500000000027613330223362015764 0ustar njsnjs00000000000000include LICENSE LICENSE.MIT LICENSE.APACHE2 include README.rst include CODE_OF_CONDUCT.md CONTRIBUTING.md include test-requirements.txt .coveragerc recursive-include docs * prune docs/build async_generator-1.10/CODE_OF_CONDUCT.md0000644000175000017500000000014213230364720017020 0ustar njsnjs00000000000000For the Trio code of conduct, see: https://trio.readthedocs.io/en/latest/code-of-conduct.html async_generator-1.10/test-requirements.txt0000644000175000017500000000002213230364720020457 0ustar njsnjs00000000000000pytest pytest-cov async_generator-1.10/setup.py0000644000175000017500000000237613330223362015743 0ustar njsnjs00000000000000from pathlib import Path from setuptools import setup, find_packages exec(open("async_generator/_version.py", encoding="utf-8").read()) setup( name="async_generator", version=__version__, description="Async generators and context managers for Python 3.5+", # Just in case the cwd is not the root of the source tree, or python is # not set to use utf-8 by default: long_description=Path(__file__).with_name("README.rst").read_text('utf-8'), author="Nathaniel J. Smith", author_email="njs@pobox.com", license="MIT -or- Apache License 2.0", packages=find_packages(), url="https://github.com/python-trio/async_generator", python_requires=">=3.5", keywords=["async"], classifiers=[ 'Development Status :: 5 - Production/Stable', "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Framework :: AsyncIO", ] ) async_generator-1.10/PKG-INFO0000644000175000017500000001333113330225256015323 0ustar njsnjs00000000000000Metadata-Version: 1.2 Name: async_generator Version: 1.10 Summary: Async generators and context managers for Python 3.5+ Home-page: https://github.com/python-trio/async_generator Author: Nathaniel J. Smith Author-email: njs@pobox.com License: MIT -or- Apache License 2.0 Description: .. image:: https://img.shields.io/badge/chat-join%20now-blue.svg :target: https://gitter.im/python-trio/general :alt: Join chatroom .. image:: https://img.shields.io/badge/docs-read%20now-blue.svg :target: https://async-generator.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status .. image:: https://travis-ci.org/python-trio/async_generator.svg?branch=master :target: https://travis-ci.org/python-trio/async_generator :alt: Automated test status .. image:: https://ci.appveyor.com/api/projects/status/af4eyed8o8tc3t0r/branch/master?svg=true :target: https://ci.appveyor.com/project/python-trio/trio/history :alt: Automated test status (Windows) .. image:: https://codecov.io/gh/python-trio/async_generator/branch/master/graph/badge.svg :target: https://codecov.io/gh/python-trio/async_generator :alt: Test coverage The async_generator library =========================== Python 3.6 added `async generators `__. (What's an async generator? `Check out my 5-minute lightning talk demo from PyCon 2016 `__.) Python 3.7 adds some more tools to make them usable, like ``contextlib.asynccontextmanager``. This library gives you all that back to Python 3.5. For example, this code only works in Python 3.6+: .. code-block:: python3 async def load_json_lines(stream_reader): async for line in stream_reader: yield json.loads(line) But this code does the same thing, and works on Python 3.5+: .. code-block:: python3 from async_generator import async_generator, yield_ @async_generator async def load_json_lines(stream_reader): async for line in stream_reader: await yield_(json.loads(line)) Or in Python 3.7, you can write: .. code-block:: python3 from contextlib import asynccontextmanager @asynccontextmanager async def background_server(): async with trio.open_nursery() as nursery: value = await nursery.start(my_server) try: yield value finally: # Kill the server when the scope exits nursery.cancel_scope.cancel() This is the same, but back to 3.5: .. code-block:: python3 from async_generator import async_generator, yield_, asynccontextmanager @asynccontextmanager @async_generator async def background_server(): async with trio.open_nursery() as nursery: value = await nursery.start(my_server) try: await yield_(value) finally: # Kill the server when the scope exits nursery.cancel_scope.cancel() (And if you're on 3.6, you can use ``@asynccontextmanager`` with native generators.) Let's do this ============= * Install: ``python3 -m pip install -U async_generator`` (or on Windows, maybe ``py -3 -m pip install -U async_generator`` * Manual: https://async-generator.readthedocs.io/ * Bug tracker and source code: https://github.com/python-trio/async_generator * Real-time chat: https://gitter.im/python-trio/general * License: MIT or Apache 2, your choice * Contributor guide: https://trio.readthedocs.io/en/latest/contributing.html * Code of conduct: Contributors are requested to follow our `code of conduct `__ in all project spaces. How come some of those links talk about "trio"? =============================================== `Trio `__ is a new async concurrency library for Python that's obsessed with usability and correctness – we want to make it *easy* to get things *right*. The ``async_generator`` library is maintained by the Trio project as part of that mission, and because Trio uses ``async_generator`` internally. You can use ``async_generator`` with any async library. It works great with ``asyncio``, or Twisted, or whatever you like. (But we think Trio is pretty sweet.) Keywords: async Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: License :: OSI Approved :: Apache Software License Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Classifier: Programming Language :: Python :: 3 :: Only Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Framework :: AsyncIO Requires-Python: >=3.5 async_generator-1.10/CONTRIBUTING.md0000644000175000017500000000014213230364720016452 0ustar njsnjs00000000000000For the Trio contributing guide, see: https://trio.readthedocs.io/en/latest/contributing.html async_generator-1.10/docs/0000755000175000017500000000000013330225256015155 5ustar njsnjs00000000000000async_generator-1.10/docs/source/0000755000175000017500000000000013330225256016455 5ustar njsnjs00000000000000async_generator-1.10/docs/source/history.rst0000644000175000017500000001120213330224465020705 0ustar njsnjs00000000000000Release history =============== .. currentmodule:: async_generator .. towncrier release notes start Async_Generator 1.10 (2018-07-31) --------------------------------- Features ~~~~~~~~ - Add support for PEP 525-style finalization hooks via ``set_asyncgen_hooks()`` and ``get_asyncgen_hooks()`` functions. On Python 3.6+, these are aliases for the versions in ``sys``; on Python 3.5, they're work-alike implementations. And, ``@async_generator`` generators now call these hooks at the appropriate times. (`#15 `__) Fixes ~~~~~ - Package now properly includes license files. (`#11 `__) 1.9 (2018-01-19) ---------------- * Add :func:`asynccontextmanager` * When a partially-exhausted ``async_generator`` is garbage collected, the warning printed now includes the generator's name to help you track it down. * Move under the auspices of the Trio project. This includes a license change from MIT → dual MIT+Apache2, and various changes to internal organization to match Trio project standard. 1.8 (2017-06-17) ---------------- * Implement PEP 479: if a ``StopAsyncIteration`` leaks out of an async generator body, wrap it into a ``RuntimeError``. * If an async generator was instantiated but never iterated, then we used to issue a spurious "RuntimeWarning: coroutine '...' was never awaited" warning. This is now fixed. * Add PyPy3 to our test matrix. * 100% test coverage. 1.7 (2017-05-13) ---------------- * Fix a subtle bug where if you wrapped an async generator using ``functools.wraps``, then ``isasyncgenfunction`` would return True for the wrapper. This isn't how ``inspect.isasyncgenfunction`` works, and it broke ``sphinxcontrib_trio``. 1.6 (2017-02-17) ---------------- * Add support for async generator introspection attributes ``ag_running``, ``ag_code``, ``ag_frame``. * Attempting to re-enter a running async_generator now raises ``ValueError``, just like for native async generators. * 100% test coverage. 1.5 (2017-01-15) ---------------- * Remove (temporarily?) the hacks that let ``yield_`` and ``yield_from_`` work with native async generators. It turns out that due to obscure linking issues this was causing the library to be entirely broken on Python 3.6 on Windows (but not Linux or MacOS). It's probably fixable, but needs some fiddling with ctypes to get the refcounting right, and I couldn't figure it out in the time I had available to spend. So in this version, everything that worked before still works with ``@async_generator``-style generators, but uniformly, on all platforms, ``yield_`` and ``yield_from_`` now do *not* work inside native-style async generators. * Now running CI testing on Windows as well as Linux. * 100% test coverage. 1.4 (2016-12-05) ---------------- * Allow ``await yield_()`` as an shorthand for ``await yield_(None)`` (thanks to Alex Grönholm for the suggestion+patch). * Small cleanups to setup.py and test infrastructure. * 100% test coverage (now including branch coverage!) 1.3 (2016-11-24) ---------------- * Added ``isasyncgen`` and ``isasyncgenfunction``. * On 3.6+, register our async generators with ``collections.abc.AsyncGenerator``. * 100% test coverage. 1.2 (2016-11-14) ---------------- * Rewrote ``yield from`` support; now has much more accurate handling of edge cases. * ``yield_from_`` now works inside CPython 3.6's native async generators. * Added ``aclosing`` context manager; it's pretty trivial, but if we're going to recommend it be used everywhere then it seems polite to include it. * 100% test coverage. 1.1 (2016-11-06) ---------------- * Support for ``asend``\/``athrow``\/``aclose`` * Support for ``yield from`` * Add a ``__del__`` method that complains about improperly cleaned up async generators. * Adapt to `the change in Python 3.5.2 `_ where ``__aiter__`` should now be a regular method instead of an async method. * Adapt to Python 3.5.2's pickiness about iterating over already-exhausted coroutines. * 100% test coverage. 1.0 (2016-07-03) ---------------- * Fixes a very nasty and hard-to-hit bug where ``await yield_(...)`` calls could escape out to the top-level coroutine runner and get lost, if the last trap out to the coroutine runner before the ``await yield_(...)`` caused an exception to be injected. * Infinitesimally more efficient due to re-using internal ``ANextIter`` objects instead of recreating them on each call to ``__anext__``. * 100% test coverage. 0.0.1 (2016-05-31) ------------------ Initial release. async_generator-1.10/docs/source/conf.py0000644000175000017500000001401013230364720017747 0ustar njsnjs00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- # # Documentation build configuration file, created by # sphinx-quickstart on Sat Jan 21 19:11:14 2017. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # import os import sys # So autodoc can import our package sys.path.insert(0, os.path.abspath('../..')) # Warn about all references to unknown targets nitpicky = True # Except for these ones, which we expect to point to unknown targets: nitpick_ignore = [ # Format is ("sphinx reference type", "string"), e.g.: ("py:obj", "bytes-like"), ] # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.coverage', 'sphinx.ext.napoleon', 'sphinxcontrib_trio', ] intersphinx_mapping = { # 3.7 so that we can link to contextlib.asynccontextmanager # This URL can switch back to /3 after cpython 3.7 is released "python": ('https://docs.python.org/3.7', None), "trio": ('https://trio.readthedocs.io/en/stable', None), } autodoc_member_order = "bysource" # Add any paths that contain templates here, relative to this directory. templates_path = [] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # # source_suffix = ['.rst', '.md'] source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = 'async_generator' copyright = 'The async_generator authors' author = 'The async_generator authors' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. import async_generator version = async_generator.__version__ # The full version, including alpha/beta/rc tags. release = version # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path exclude_patterns = [] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # The default language for :: blocks highlight_language = 'python3' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # #html_theme = 'alabaster' # We have to set this ourselves, not only because it's useful for local # testing, but also because if we don't then RTD will throw away our # html_theme_options. import sphinx_rtd_theme html_theme = 'sphinx_rtd_theme' html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # html_theme_options = { # default is 2 # show deeper nesting in the RTD theme's sidebar TOC # https://stackoverflow.com/questions/27669376/ # I'm not 100% sure this actually does anything with our current # versions/settings... "navigation_depth": 4, "logo_only": True, } # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # -- Options for HTMLHelp output ------------------------------------------ # Output file base name for HTML help builder. htmlhelp_basename = 'async_generatordoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # # 'preamble': '', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'async_generator.tex', 'Trio Documentation', author, 'manual'), ] # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'async_generator', 'async_generator Documentation', [author], 1) ] # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'async_generator', 'async_generator Documentation', author, 'async_generator', 'A short description of the project (for setup.py metadata)', 'Miscellaneous'), ] async_generator-1.10/docs/source/reference.rst0000644000175000017500000001371313267320140021147 0ustar njsnjs00000000000000API documentation ================= Async generators ---------------- In Python 3.6+, you can write a native async generator like this:: async def load_json_lines(stream_reader): async for line in stream_reader: yield json.loads(line) Here's the same thing written with this library, which works on Python 3.5+:: from async_generator import async_generator, yield @async_generator async def load_json_lines(stream_reader): async for line in stream_reader: await yield_(json.loads(line)) Basically: * decorate your function with ``@async_generator`` * replace ``yield`` with ``await yield_()`` * replace ``yield X`` with ``await yield_(X)`` That's it! Yield from ~~~~~~~~~~ Native async generators don't support ``yield from``:: # Doesn't work! async def wrap_load_json_lines(stream_reader): # This is a SyntaxError yield from load_json_lines(stream_reader) But we do:: from async_generator import async_generator, yield_from_ # This works! @async_generator async def wrap_load_json_lines(stream_reader): await yield_from_(load_json_lines(stream_reader)) You can only use ``yield_from_`` inside an ``@async_generator`` function, BUT the thing you PASS to ``yield_from_`` can be any kind of async iterator, including native async generators. Our ``yield_from_`` fully supports the classic ``yield from`` semantics, including forwarding ``asend`` and ``athrow`` calls into the delegated async generator, and returning values:: from async_generator import async_generator, yield_, yield_from_ @async_generator async def agen1(): await yield_(1) await yield_(2) return "great!" @async_generator async def agen2(): value = await yield_from_(agen1()) assert value == "great!" Introspection ~~~~~~~~~~~~~ For introspection purposes, we also export the following functions: .. function:: isasyncgen(agen_obj) Returns true if passed either an async generator object created by this library, or a native Python 3.6+ async generator object. Analogous to :func:`inspect.isasyncgen` in 3.6+. .. function:: isasyncgenfunction(agen_func) Returns true if passed either an async generator function created by this library, or a native Python 3.6+ async generator function. Analogous to :func:`inspect.isasyncgenfunction` in 3.6+. Example:: >>> isasyncgenfunction(load_json_lines) True >>> gen_object = load_json_lines(asyncio_stream_reader) >>> isasyncgen(gen_object) True In addition, this library's async generator objects are registered with the ``collections.abc.AsyncGenerator`` abstract base class (if available):: >>> isinstance(gen_object, collections.abc.AsyncGenerator) True Semantics ~~~~~~~~~ This library generally tries hard to match the semantics of Python 3.6's native async generators in every detail (`PEP 525 `__), with additional support for ``yield from`` and for returning non-None values from an async generator (under the theory that these may well be added to native async generators one day). Garbage collection hooks ~~~~~~~~~~~~~~~~~~~~~~~~ This library fully supports the native async generator `finalization semantics `__, including the per-thread ``firstiter`` and ``finalizer`` hooks. You can use ``async_generator.set_asyncgen_hooks()`` exactly like you would use ``sys.set_asyncgen_hooks()`` with native generators. On Python 3.6+, the former is an alias for the latter, so libraries that use the native mechanism should work seamlessly with ``@async_generator`` functions. On Python 3.5, where there is no ``sys.set_asyncgen_hooks()``, most libraries probably *won't* know about ``async_generator.set_asyncgen_hooks()``, so you'll need to exercise more care with explicit cleanup, or install appropriate hooks yourself. While finishing cleanup of an async generator is better than dropping it on the floor at the first ``await``, it's still not a perfect solution; in addition to the unpredictability of GC timing, the ``finalizer`` hook has no practical way to determine the context in which the generator was being iterated, so an exception thrown from the generator during ``aclose()`` must either crash the program or get discarded. It's much better to close your generators explicitly when you're done with them, perhaps using the :ref:`aclosing context manager `. See `this discussion `__ and `PEP 533 `__ for more details. .. _contextmanagers: Context managers ---------------- As discussed above, you should always explicitly call ``aclose`` on async generators. To make this more convenient, this library also includes an ``aclosing`` async context manager. It acts just like the ``closing`` context manager included in the stdlib ``contextlib`` module, but does ``await obj.aclose()`` instead of ``obj.close()``. Use it like this:: from async_generator import aclosing async with aclosing(load_json_lines(asyncio_stream_reader)) as agen: async for json_obj in agen: ... Or if you want to write your own async context managers, we've got you covered: .. function:: asynccontextmanager :decorator: This is a backport of :func:`contextlib.asynccontextmanager`, which wasn't added to the standard library until Python 3.7. You can use ``@asynccontextmanager`` with either native async generators, or the ones from this package. If you use it with the ones from this package, remember that ``@asynccontextmanager`` goes *on top* of ``@async_generator``:: # Correct! @asynccontextmanager @async_generator async def my_async_context_manager(): ... # This won't work :-( @async_generator @asynccontextmanager async def my_async_context_manager(): ... async_generator-1.10/docs/source/_static/0000755000175000017500000000000013330225256020103 5ustar njsnjs00000000000000async_generator-1.10/docs/source/_static/.gitkeep0000644000175000017500000000000013230364720021521 0ustar njsnjs00000000000000async_generator-1.10/docs/source/index.rst0000644000175000017500000000115213230364720020314 0ustar njsnjs00000000000000.. documentation master file, created by sphinx-quickstart on Sat Jan 21 19:11:14 2017. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. ===================================================================== async_generator: Async generators and related tools for older Pythons ===================================================================== .. toctree:: :maxdepth: 2 reference.rst history.rst ==================== Indices and tables ==================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` * :ref:`glossary` async_generator-1.10/docs/make.bat0000644000175000017500000000142313230364720016561 0ustar njsnjs00000000000000@ECHO OFF pushd %~dp0 REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set SOURCEDIR=source set BUILDDIR=build set SPHINXPROJ=async_generator if "%1" == "" goto help %SPHINXBUILD% >NUL 2>NUL if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% goto end :help %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% :end popd async_generator-1.10/docs/Makefile0000644000175000017500000000115113230364720016612 0ustar njsnjs00000000000000# Minimal makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build SPHINXPROJ = async_generator SOURCEDIR = source BUILDDIR = build # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) async_generator-1.10/.coveragerc0000644000175000017500000000012213021445742016342 0ustar njsnjs00000000000000[run] branch=True source=async_generator omit= setup.py [report] precision = 1