pax_global_header00006660000000000000000000000064140732707160014521gustar00rootroot0000000000000052 comment=1378c9fc07a0031943a97a58f2b99b706fd57b05 asyncio-mqtt-0.10.0/000077500000000000000000000000001407327071600142275ustar00rootroot00000000000000asyncio-mqtt-0.10.0/.gitignore000066400000000000000000000000701407327071600162140ustar00rootroot00000000000000__pycache__ build dist *.egg-info local_test.py .idea/ asyncio-mqtt-0.10.0/CHANGELOG.md000066400000000000000000000203021407327071600160350ustar00rootroot00000000000000# Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] Nothing so far. ## [0.10.0] - 2021-07-13 ### Added - Add new parameter `socket_options` to `Client.__init__`. Contributed by [@xydan83](https://github.com/xydan83) in [#71](https://github.com/sbtinstruments/asyncio-mqtt/pull/71) - Add new parameter `message_retry_set` to `Client.__init__`. Contributed by [@xydan83](https://github.com/xydan83) in [#69](https://github.com/sbtinstruments/asyncio-mqtt/pull/69) - Export `ProtocolVersion` from the main module. Contributed by [Andrรฉ (@tropxy)](https://github.com/tropxy) in [#65](https://github.com/sbtinstruments/asyncio-mqtt/pull/65) (1/2) - Add documentation about publishers, client arguments, etc. Contributed by [Andrรฉ (@tropxy)](https://github.com/tropxy) in [#65](https://github.com/sbtinstruments/asyncio-mqtt/pull/65) (2/2) ### Fixed - Fix race condition that caused `InvalidStateError` in `force_disconnect()`. Contributed by [@functionpointer](https://github.com/functionpointer) in [#67](https://github.com/sbtinstruments/asyncio-mqtt/pull/67) ## [0.9.1] - 2021-05-13 ### Fixed - Fix handling of MQTTv5 reason codes. Contributed by [Jakob Schlyter (@jschlyter)](https://github.com/jschlyter) in [#59](https://github.com/sbtinstruments/asyncio-mqtt/pull/59) - Account for `-1` socket handles in the close callback. Contributed by [@wrobell](https://github.com/wrobell) in [#60](https://github.com/sbtinstruments/asyncio-mqtt/pull/60) ## [0.9.0] - 2021-05-03 ### Added - Add type hints. Contributed by [Ellis Percival (@flyte)](https://github.com/flyte) in [#37](https://github.com/sbtinstruments/asyncio-mqtt/pull/37) - Add the `keepalive`, `bind_address`, `bind_port`, `clean_start`, `properties` arguments. Contributed by [Marcin Jaworski (@yawor)](https://github.com/yawor) in [#56](https://github.com/sbtinstruments/asyncio-mqtt/pull/56) ### Fixed - Fix Python 3.6 compatibility. Contributed by [(@fipwmaqzufheoxq92ebc)](https://github.com/fipwmaqzufheoxq92ebc) in [#57](https://github.com/sbtinstruments/asyncio-mqtt/pull/57). Note that asyncio-mqtt officially targets Python 3.7. Compatibility with 3.6 is community-driven. - Fix "Broken pipe" error. Contributed by [Gilbert (@gilbertsmink)](https://github.com/gilbertsmink) in [#55](https://github.com/sbtinstruments/asyncio-mqtt/pull/55) - Fix socket check when you select the WebSocket transport. Contributed by [Robert Chmielowiec (@chmielowiec)](https://github.com/chmielowiec) in [#54](https://github.com/sbtinstruments/asyncio-mqtt/pull/54) - Fix `TypeError` on invalid username and password combination - Check that _misc_task is not None before trying to cancel it. Contributed by [Ellis Percival (@flyte)](https://github.com/flyte) in [#41](https://github.com/sbtinstruments/asyncio-mqtt/pull/41) - Fix exception in `on_socket_open`: Non-thread-safe operation invoked on an event loop other than the current one. Contributed by [Ellis Percival (@flyte)](https://github.com/flyte) in [#40](https://github.com/sbtinstruments/asyncio-mqtt/pull/40) ## [0.8.1] - 2021-02-23 ### Fixed - Fix `AttributeError` when you use WebSockets. Contributed by [Robert Chmielowiec (@chmielowiec)](https://github.com/chmielowiec) in [#36](https://github.com/sbtinstruments/asyncio-mqtt/pull/36) - Fix `asyncio.InvalidStateError` in the `_on_connect` callback. Contributed by [Maxim Shmalovsky (@vitalalerter)](https://github.com/vitalalerter) in [#31](https://github.com/sbtinstruments/asyncio-mqtt/pull/31) - Fix "Future exception was never retrieved" on disconnect. Contributed by [Martin Hjelmare (@martinhjelmare)](https://github.com/martinhjelmare) in [#25](https://github.com/sbtinstruments/asyncio-mqtt/pull/25) - Fix `connect` so it no longer blocks the event loop. Contributed by [ร˜ystein Haug Olsen (@oholsen)](https://github.com/oholsen) in [#23](https://github.com/sbtinstruments/asyncio-mqtt/pull/23) ## [0.8.0] - 2020-11-09 ### Added - Add `transport` argument to `Client` Contributed by [@opengs](https://github.com/opengs) in [#21](https://github.com/sbtinstruments/asyncio-mqtt/pull/21) - Add `clean_session` argument to `Client` Contributed by [@nadyka](https://github.com/madnadyka) in [#17](https://github.com/sbtinstruments/asyncio-mqtt/pull/17) ## [0.7.0] - 2020-08-04 I've tested the library for production use at SBT Instruments. This uncovered a bunch of bugs and missing features that I've adressed in this release. We are approaching a 1.0.0 release. Let me know if you want something changed before that via the issue tracker on GitHub. ### Added - Add support for MQTTv5. - Add `will` keyword argument to `Client`. - Add `MqttConnectError` with specific error messages for connection failures. - Add `Client.id` property that returns the client ID (or `None` if the client ID was not specified during construction). ### Fixed - Fix unhandled exception error. - Fix "Task was destroyed but it is pending" error. - Fix compatibility with `asyncqt`'s event loop. - Fix race condition in `Client.connect` that raised an `AttributeError`. - Fix "[asyncio] Future exception was never retrieved" debug message. - Fix support for python 3.6. Contributed by [Derrick Lyndon Pallas (@pallas)](https://github.com/pallas) in [#12](https://github.com/sbtinstruments/asyncio-mqtt/pull/12) ## [0.6.0] - 2020-06-26 ### Changed - No longer logs exception in `Client.__aexit__`. It's perfectly valid to exit due to, e.g., `asyncio.CancelledError` so let's not treat it as an error. ## [0.5.0] - 2020-06-08 ### Added - Add support for python 3.6. Contributed by [Derrick Lyndon Pallas (@pallas)](https://github.com/pallas) in [#7](https://github.com/sbtinstruments/asyncio-mqtt/pull/7) (1/2). - Add `client_id` and `tls_context` keyword arguments to the `Client` constructor. Contributed by [Derrick Lyndon Pallas (@pallas)](https://github.com/pallas) in [#7](https://github.com/sbtinstruments/asyncio-mqtt/pull/7) (2/2). - Add `timeout` keyword argument to both `Client.connect` and `Client.disconnect`. Default value of `10` seconds (like the other functions). ### Changed - Propagate disconnection errors to subscribers. This enables user code to detect if a disconnect occurs. E.g., due to network errors. ## [0.4.0] - 2020-05-06 ### Changed - **BREAKING CHANGE:** Forward the [MQTTMessage](https://github.com/eclipse/paho.mqtt.python/blob/1eec03edf39128e461e6729694cf5d7c1959e5e4/src/paho/mqtt/client.py#L355) class from paho-mqtt instead of just the `payload`. This applies to both `Client.filtered_messages` and `Client.unfiltered_messages`. This way, user code not only gets the message `payload` but also the `topic`, `qos` level, `retain` flag, etc. Contributed by [Matthew Bradbury (@MBradbury)](https://github.com/MBradbury) in [#3](https://github.com/sbtinstruments/asyncio-mqtt/pull/3). ## [0.3.0] - 2020-04-13 ### Added - Add `username` and `password` keyword arguments to the `Client` constructor. Contributed by [@gluap](https://github.com/gluap) in [#1](https://github.com/sbtinstruments/asyncio-mqtt/pull/1). ### Fixed - Fix log message context for `Client.filtered_messages`. ## [0.2.1] - 2020-04-07 ### Fixed - Fix regression with the `Client._wait_for` helper method introduced in the latest release. ## [0.2.0] - 2020-04-07 ### Changed - **BREAKING CHANGE:** Replace all uses of `asyncio.TimeoutError` with `MqttError`. Calls to `Client.subscribe`/`unsubscribe`/`publish` will no longer raise `asyncio.TimeoutError.` The new behaviour makes it easier to reason about, which exceptions the library throws: - Wrong input parameters? Raise `ValueError`. - Network or protocol failures? `MqttError`. - Broken library state? `RuntimeError`. ## [0.1.3] - 2020-04-07 ### Fixed - Fix how keyword arguments are forwarded in `Client.publish` and `Client.subscribe`. ## [0.1.2] - 2020-04-06 ### Fixed - Remove log call that was erroneously put in while debugging the latest release. ## [0.1.1] - 2020-04-06 ### Fixed - Add missing parameters to `Client.publish`. - Fix error in `Client.publish` when paho-mqtt publishes immediately. ## [0.1.0] - 2020-04-06 Initial release. asyncio-mqtt-0.10.0/LICENSE000066400000000000000000000026671407327071600152470ustar00rootroot00000000000000Copyright 2020 (c) SBT Instruments Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.asyncio-mqtt-0.10.0/README.md000066400000000000000000000203041407327071600155050ustar00rootroot00000000000000![license](https://img.shields.io/github/license/sbtinstruments/asyncio-mqtt) ![semver](https://img.shields.io/github/v/tag/sbtinstruments/asyncio-mqtt?sort=semver) # MQTT client with idiomatic asyncio interface ๐Ÿ™Œ Write code like this: ##### Subscriber ```python async with Client("test.mosquitto.org") as client: async with client.filtered_messages("floors/+/humidity") as messages: await client.subscribe("floors/#") async for message in messages: print(message.payload.decode()) ``` ##### Publisher ```python async with Client("test.mosquitto.org") as client: message = "10%" await client.publish( "floors/bed_room/humidity", payload=message.encode() ) ``` asyncio-mqtt combines the stability of the time-proven [paho-mqtt](https://github.com/eclipse/paho.mqtt.python) library with a modern, asyncio-based interface. * No more callbacks! ๐Ÿ‘ * No more return codes (welcome to the `MqttError`) * Graceful disconnection (forget about `on_unsubscribe`, `on_disconnect`, etc.) * Compatible with `async` code * Fully type-hinted * Did we mention no more callbacks? The whole thing is less than [600 lines of code](https://github.com/sbtinstruments/asyncio-mqtt/blob/master/asyncio_mqtt/client.py). ## Installation ๐Ÿ“š `pip install asyncio-mqtt` ## Advanced use โšก Let's make the example from before more interesting: ```python import asyncio from contextlib import AsyncExitStack, asynccontextmanager from random import randrange from asyncio_mqtt import Client, MqttError async def advanced_example(): # We ๐Ÿ’› context managers. Let's create a stack to help # us manage them. async with AsyncExitStack() as stack: # Keep track of the asyncio tasks that we create, so that # we can cancel them on exit tasks = set() stack.push_async_callback(cancel_tasks, tasks) # Connect to the MQTT broker client = Client("test.mosquitto.org") await stack.enter_async_context(client) # You can create any number of topic filters topic_filters = ( "floors/+/humidity", "floors/rooftop/#" # ๐Ÿ‘‰ Try to add more filters! ) for topic_filter in topic_filters: # Log all messages that matches the filter manager = client.filtered_messages(topic_filter) messages = await stack.enter_async_context(manager) template = f'[topic_filter="{topic_filter}"] {{}}' task = asyncio.create_task(log_messages(messages, template)) tasks.add(task) # Messages that doesn't match a filter will get logged here messages = await stack.enter_async_context(client.unfiltered_messages()) task = asyncio.create_task(log_messages(messages, "[unfiltered] {}")) tasks.add(task) # Subscribe to topic(s) # ๐Ÿค” Note that we subscribe *after* starting the message # loggers. Otherwise, we may miss retained messages. await client.subscribe("floors/#") # Publish a random value to each of these topics topics = ( "floors/basement/humidity", "floors/rooftop/humidity", "floors/rooftop/illuminance", # ๐Ÿ‘‰ Try to add more topics! ) task = asyncio.create_task(post_to_topics(client, topics)) tasks.add(task) # Wait for everything to complete (or fail due to, e.g., network # errors) await asyncio.gather(*tasks) async def post_to_topics(client, topics): while True: for topic in topics: message = randrange(100) print(f'[topic="{topic}"] Publishing message={message}') await client.publish(topic, message, qos=1) await asyncio.sleep(2) async def log_messages(messages, template): async for message in messages: # ๐Ÿค” Note that we assume that the message paylod is an # UTF8-encoded string (hence the `bytes.decode` call). print(template.format(message.payload.decode())) async def cancel_tasks(tasks): for task in tasks: if task.done(): continue task.cancel() try: await task except asyncio.CancelledError: pass async def main(): # Run the advanced_example indefinitely. Reconnect automatically # if the connection is lost. reconnect_interval = 3 # [seconds] while True: try: await advanced_example() except MqttError as error: print(f'Error "{error}". Reconnecting in {reconnect_interval} seconds.') finally: await asyncio.sleep(reconnect_interval) asyncio.run(main()) ``` ## Alternative asyncio-based MQTT clients Is asyncio-mqtt not what you are looking for? Try another client: * [hbmqtt](https://github.com/beerfactory/hbmqtt) - Own protocol implementation. Includes a broker. ![GitHub stars](https://img.shields.io/github/stars/beerfactory/hbmqtt) ![license](https://img.shields.io/github/license/beerfactory/hbmqtt) * [gmqtt](https://github.com/wialon/gmqtt) - Own protocol implementation. No dependencies. ![GitHub stars](https://img.shields.io/github/stars/wialon/gmqtt) ![license](https://img.shields.io/github/license/wialon/gmqtt) * [aiomqtt](https://github.com/mossblaser/aiomqtt) - Wrapper around paho-mqtt. ![GitHub stars](https://img.shields.io/github/stars/mossblaser/aiomqtt) ![license](https://img.shields.io/github/license/mossblaser/aiomqtt) * [mqttools](https://github.com/eerimoq/mqttools) - Own protocol implementation. No dependencies. ![GitHub stars](https://img.shields.io/github/stars/eerimoq/mqttools) ![license](https://img.shields.io/github/license/eerimoq/mqttools) * [aio-mqtt](https://github.com/NotJustAToy/aio-mqtt) - Own protocol implementation. No dependencies. ![GitHub stars](https://img.shields.io/github/stars/NotJustAToy/aio-mqtt) ![license](https://img.shields.io/github/license/NotJustAToy/aio-mqtt) This is not an exhaustive list. ### Honorable mentions * [trio-paho-mqtt](https://github.com/bkanuka/trio-paho-mqtt) - Trio-based. Wrapper around paho-mqtt. ![GitHub stars](https://img.shields.io/github/stars/bkanuka/trio-paho-mqtt) ![license](https://img.shields.io/github/license/bkanuka/trio-paho-mqtt) ## Requirements Python 3.7 or later. There is only a single dependency: * [paho-mqtt](https://github.com/eclipse/paho.mqtt.python) ![GitHub stars](https://img.shields.io/github/stars/eclipse/paho.mqtt.python) ![license](https://img.shields.io/github/license/eclipse/paho.mqtt.python) ## Note for Windows Users Since Python 3.8, the default asyncio event loop is the `ProactorEventLoop`. Said loop [doesn't support the `add_reader` method](https://docs.python.org/3/library/asyncio-platforms.html#windows) that is required by asyncio-mqtt. To use asyncio-mqtt, please switch to an event loop that supports the `add_reader` method such as the built-in `SelectorEventLoop`. E.g: ``` # Change to the "Selector" event loop asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) # Run your async application as usual asyncio.run(main()) ``` ## Changelog Please refer to the [CHANGELOG](https://github.com/sbtinstruments/asyncio-mqtt/blob/master/CHANGELOG.md) document. It adheres to the principles of [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## Versioning ![semver](https://img.shields.io/github/v/tag/sbtinstruments/asyncio-mqtt?sort=semver) This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Expect API changes until we reach version `1.0.0`. After `1.0.0`, breaking changes will only occur in major release (e.g., `2.0.0`, `3.0.0`, etc.). ## License ![license](https://img.shields.io/github/license/sbtinstruments/asyncio-mqtt) Note that the underlying paho-mqtt library is dual-licensed. One of the licenses is the so-called [Eclipse Distribution License v1.0](https://www.eclipse.org/org/documents/edl-v10.php). It is almost word-for-word identical to the [BSD 3-clause License](https://opensource.org/licenses/BSD-3-Clause). The only differences are: * One use of "COPYRIGHT OWNER" (EDL) instead of "COPYRIGHT HOLDER" (BSD) * One use of "Eclipse Foundation, Inc." (EDL) instead of "copyright holder" (BSD) asyncio-mqtt-0.10.0/asyncio_mqtt/000077500000000000000000000000001407327071600167415ustar00rootroot00000000000000asyncio-mqtt-0.10.0/asyncio_mqtt/__init__.py000066400000000000000000000004741407327071600210570ustar00rootroot00000000000000# SPDX-License-Identifier: BSD-3-Clause from .error import MqttError, MqttCodeError from .client import Client, Will, ProtocolVersion from .version import __version__ __all__ = ["MqttError", "MqttCodeError", "Client", "Will", "ProtocolVersion", "__version__"] asyncio-mqtt-0.10.0/asyncio_mqtt/client.py000066400000000000000000000526651407327071600206070ustar00rootroot00000000000000# SPDX-License-Identifier: BSD-3-Clause import asyncio import logging import socket import ssl from contextlib import contextmanager, suppress from enum import IntEnum from types import TracebackType from typing import ( Any, AsyncGenerator, AsyncIterator, Awaitable, Callable, Dict, Generator, Iterator, List, Optional, Tuple, Type, Union, cast, Iterable, ) try: from contextlib import asynccontextmanager except ImportError: from async_generator import asynccontextmanager # type: ignore import paho.mqtt.client as mqtt # type: ignore from paho.mqtt.properties import Properties from .error import MqttCodeError, MqttConnectError, MqttError from .types import PayloadType, T MQTT_LOGGER = logging.getLogger("mqtt") MQTT_LOGGER.setLevel(logging.WARNING) class ProtocolVersion(IntEnum): """ A mapping of Paho MQTT protocol version constants to an Enum for use in type hints. """ V31 = mqtt.MQTTv31 V311 = mqtt.MQTTv311 V5 = mqtt.MQTTv5 # TODO: This should be a (frozen) dataclass (from Python 3.7) # when we drop Python 3.6 support class Will: def __init__( self, topic: str, payload: Optional[PayloadType] = None, qos: int = 0, retain: bool = False, properties: Optional[mqtt.Properties] = None, ): self.topic = topic self.payload = payload self.qos = qos self.retain = retain self.properties = properties # See the overloads of `socket.setsockopt` for details. SocketOption = Union[ Tuple[int, int, Union[int, bytes]], Tuple[int, int, None, int], ] class Client: def __init__( self, hostname: str, port: int = 1883, *, username: Optional[str] = None, password: Optional[str] = None, logger: Optional[logging.Logger] = None, client_id: Optional[str] = None, tls_context: Optional[ssl.SSLContext] = None, protocol: Optional[ProtocolVersion] = None, will: Optional[Will] = None, clean_session: Optional[bool] = None, transport: str = "tcp", keepalive: int = 60, bind_address: str = "", bind_port: int = 0, clean_start: bool = mqtt.MQTT_CLEAN_START_FIRST_ONLY, properties: Optional[Properties] = None, message_retry_set: int = 20, socket_options: Optional[Iterable[SocketOption]] = (), ): self._hostname = hostname self._port = port self._keepalive = keepalive self._bind_address = bind_address self._bind_port = bind_port self._clean_start = clean_start self._properties = properties self._loop = asyncio.get_event_loop() self._connected: "asyncio.Future[int]" = asyncio.Future() self._disconnected: "asyncio.Future[Optional[int]]" = asyncio.Future() # Pending subscribe, unsubscribe, and publish calls self._pending_subscribes: Dict[int, "asyncio.Future[int]"] = {} self._pending_unsubscribes: Dict[int, asyncio.Event] = {} self._pending_publishes: Dict[int, asyncio.Event] = {} self._pending_calls_threshold: int = 10 self._misc_task: Optional["asyncio.Task[None]"] = None if protocol is None: protocol = ProtocolVersion.V311 self._client: mqtt.Client = mqtt.Client( client_id=client_id, protocol=protocol, clean_session=clean_session, transport=transport, ) self._client.on_connect = self._on_connect self._client.on_disconnect = self._on_disconnect self._client.on_subscribe = self._on_subscribe self._client.on_unsubscribe = self._on_unsubscribe self._client.on_message = None self._client.on_publish = self._on_publish # Callbacks for custom event loop self._client.on_socket_open = self._on_socket_open self._client.on_socket_close = self._on_socket_close self._client.on_socket_register_write = self._on_socket_register_write self._client.on_socket_unregister_write = self._on_socket_unregister_write if logger is None: logger = MQTT_LOGGER self._client.enable_logger(logger) if username is not None and password is not None: self._client.username_pw_set(username=username, password=password) if tls_context is not None: self._client.tls_set_context(tls_context) if will is not None: self._client.will_set( will.topic, will.payload, will.qos, will.retain, will.properties ) self._client.message_retry_set(message_retry_set) self._socket_options: Tuple[SocketOption] = tuple(socket_options) @property def id(self) -> str: """Return the client ID. Note that paho-mqtt stores the client ID as `bytes` internally. We assume that the client ID is a UTF8-encoded string and decode it first. """ return cast(bytes, self._client._client_id).decode() @property def _pending_calls(self) -> Generator[int, None, None]: """ Yield all message IDs with pending calls. """ yield from self._pending_subscribes.keys() yield from self._pending_unsubscribes.keys() yield from self._pending_publishes.keys() async def connect(self, *, timeout: int = 10) -> None: try: # get_running_loop is preferred, but only available in python>=3.7 try: loop = asyncio.get_running_loop() except AttributeError: loop = asyncio.get_event_loop() # [3] Run connect() within an executor thread, since it blocks on socket # connection for up to `keepalive` seconds: https://git.io/Jt5Yc await loop.run_in_executor( None, self._client.connect, self._hostname, self._port, self._keepalive, self._bind_address, self._bind_port, self._clean_start, self._properties ) client_socket = self._client.socket() _set_client_socket_defaults(client_socket, self._socket_options) # paho.mqtt.Client.connect may raise one of several exceptions. # We convert all of them to the common MqttError for user convenience. # See: https://github.com/eclipse/paho.mqtt.python/blob/v1.5.0/src/paho/mqtt/client.py#L1770 except (socket.error, OSError, mqtt.WebsocketConnectionError) as error: raise MqttError(str(error)) await self._wait_for(self._connected, timeout=timeout) async def disconnect(self, *, timeout: int = 10) -> None: rc = self._client.disconnect() # Early out on error if rc != mqtt.MQTT_ERR_SUCCESS: raise MqttCodeError(rc, "Could not disconnect") # Wait for acknowledgement await self._wait_for(self._disconnected, timeout=timeout) async def force_disconnect(self) -> None: if not self._disconnected.done(): self._disconnected.set_result(None) async def subscribe(self, *args: Any, timeout: int = 10, **kwargs: Any) -> int: result, mid = self._client.subscribe(*args, **kwargs) # Early out on error if result != mqtt.MQTT_ERR_SUCCESS: raise MqttCodeError(result, "Could not subscribe to topic") # Create future for when the on_subscribe callback is called cb_result: "asyncio.Future[int]" = asyncio.Future() with self._pending_call(mid, cb_result, self._pending_subscribes): # Wait for cb_result return await self._wait_for(cb_result, timeout=timeout) async def unsubscribe(self, *args: Any, timeout: int = 10) -> None: result, mid = self._client.unsubscribe(*args) # Early out on error if result != mqtt.MQTT_ERR_SUCCESS: raise MqttCodeError(result, "Could not unsubscribe from topic") # Create event for when the on_unsubscribe callback is called confirmation = asyncio.Event() with self._pending_call(mid, confirmation, self._pending_unsubscribes): # Wait for confirmation await self._wait_for(confirmation.wait(), timeout=timeout) async def publish(self, *args: Any, timeout: int = 10, **kwargs: Any) -> None: info = self._client.publish(*args, **kwargs) # [2] # Early out on error if info.rc != mqtt.MQTT_ERR_SUCCESS: raise MqttCodeError(info.rc, "Could not publish message") # Early out on immediate success if info.is_published(): return # Create event for when the on_publish callback is called confirmation = asyncio.Event() with self._pending_call(info.mid, confirmation, self._pending_publishes): # Wait for confirmation await self._wait_for(confirmation.wait(), timeout=timeout) @asynccontextmanager async def filtered_messages( self, topic_filter: str, *, queue_maxsize: int = 0 ) -> AsyncIterator[AsyncGenerator[mqtt.MQTTMessage, None]]: """Return async generator of messages that match the given filter. Use queue_maxsize to restrict the queue size. If the queue is full, incoming messages will be discarded (and a warning is logged). If queue_maxsize is less than or equal to zero, the queue size is infinite. Example use: async with client.filtered_messages('floors/+/humidity') as messages: async for message in messages: print(f'Humidity reading: {message.payload.decode()}') """ cb, generator = self._cb_and_generator( log_context=f'topic_filter="{topic_filter}"', queue_maxsize=queue_maxsize ) try: self._client.message_callback_add(topic_filter, cb) # Back to the caller (run whatever is inside the with statement) yield generator finally: # We are exitting the with statement. Remove the topic filter. self._client.message_callback_remove(topic_filter) @asynccontextmanager async def unfiltered_messages( self, *, queue_maxsize: int = 0 ) -> AsyncIterator[AsyncGenerator[mqtt.MQTTMessage, None]]: """Return async generator of all messages that are not caught in filters.""" # Early out if self._client.on_message is not None: # TODO: This restriction can easily be removed. raise RuntimeError( "Only a single unfiltered_messages generator can be used at a time." ) cb, generator = self._cb_and_generator( log_context="unfiltered", queue_maxsize=queue_maxsize ) try: self._client.on_message = cb # Back to the caller (run whatever is inside the with statement) yield generator finally: # We are exitting the with statement. Unset the callback. self._client.on_message = None def _cb_and_generator( self, *, log_context: str, queue_maxsize: int = 0 ) -> Tuple[ Callable[[mqtt.Client, Any, mqtt.MQTTMessage], None], AsyncGenerator[mqtt.MQTTMessage, None], ]: # Queue to hold the incoming messages messages: "asyncio.Queue[mqtt.MQTTMessage]" = asyncio.Queue( maxsize=queue_maxsize ) # Callback for the underlying API def _put_in_queue( client: mqtt.Client, userdata: Any, msg: mqtt.MQTTMessage ) -> None: try: messages.put_nowait(msg) except asyncio.QueueFull: MQTT_LOGGER.warning( f"[{log_context}] Message queue is full. Discarding message." ) # The generator that we give to the caller async def _message_generator() -> AsyncGenerator[mqtt.MQTTMessage, None]: # Forward all messages from the queue while True: # Wait until we either: # 1. Receive a message # 2. Disconnect from the broker get: "asyncio.Task[mqtt.MQTTMessage]" = self._loop.create_task( messages.get() ) try: done, _ = await asyncio.wait( (get, self._disconnected), return_when=asyncio.FIRST_COMPLETED ) except asyncio.CancelledError: # If the asyncio.wait is cancelled, we must make sure # to also cancel the underlying tasks. get.cancel() raise if get in done: # We received a message. Return the result. yield get.result() else: # We got disconnected from the broker. Cancel the "get" task. get.cancel() # Stop the generator with the following exception raise MqttError("Disconnected during message iteration") return _put_in_queue, _message_generator() async def _wait_for( self, fut: Awaitable[T], timeout: Optional[float], **kwargs: Any ) -> T: try: return await asyncio.wait_for(fut, timeout=timeout, **kwargs) except asyncio.TimeoutError: raise MqttError("Operation timed out") @contextmanager def _pending_call( self, mid: int, value: T, pending_dict: Dict[int, T] ) -> Iterator[None]: if mid in self._pending_calls: raise RuntimeError( f'There already exists a pending call for message ID "{mid}"' ) pending_dict[mid] = value # [1] try: # Log a warning if there is a concerning number of pending calls pending = len(list(self._pending_calls)) if pending > self._pending_calls_threshold: MQTT_LOGGER.warning(f"There are {pending} pending publish calls.") # Back to the caller (run whatever is inside the with statement) yield finally: # The normal procedure is: # * We add the item at [1] # * A callback will remove the item # # However, if the callback doesn't get called (e.g., due to a # network error) we still need to remove the item from the dict. try: del pending_dict[mid] except KeyError: pass def _on_connect( self, client: mqtt.Client, userdata: Any, flags: Dict[str, int], rc: int, properties: Optional[mqtt.Properties] = None, ) -> None: # Return early if already connected. Sometimes, paho-mqtt calls _on_connect # multiple times. Maybe because we receive multiple CONNACK messages # from the server. In any case, we return early so that we don't set # self._connected twice (as it raises an asyncio.InvalidStateError). if self._connected.done(): return if rc == mqtt.CONNACK_ACCEPTED: self._connected.set_result(rc) else: self._connected.set_exception(MqttConnectError(rc)) def _on_disconnect( self, client: mqtt.Client, userdata: Any, rc: int, properties: Optional[mqtt.Properties] = None, ) -> None: # Return early if the disconnect is already acknowledged. # Sometimes (e.g., due to timeouts), paho-mqtt calls _on_disconnect # twice. We return early to avoid setting self._disconnected twice # (as it raises an asyncio.InvalidStateError). if self._disconnected.done(): return # Return early if we are not connected yet. This avoids calling # `_disconnected.set_exception` with an exception that will never # be retrieved (since `__aexit__` won't get called if `__aenter__` # fails). In turn, this avoids asyncio debug messages like the # following: # # "[asyncio] Future exception was never retrieved" # # See also: https://docs.python.org/3/library/asyncio-dev.html#detect-never-retrieved-exceptions if not self._connected.done() or self._connected.exception() is not None: return if rc == mqtt.MQTT_ERR_SUCCESS: self._disconnected.set_result(rc) else: self._disconnected.set_exception(MqttCodeError(rc, "Unexpected disconnect")) def _on_subscribe( self, client: mqtt.Client, userdata: Any, mid: int, granted_qos: int, properties: Optional[mqtt.Properties] = None, ) -> None: try: self._pending_subscribes.pop(mid).set_result(granted_qos) except KeyError: MQTT_LOGGER.error(f'Unexpected message ID "{mid}" in on_subscribe callback') def _on_unsubscribe( self, client: mqtt.Client, userdata: Any, mid: int, properties: Optional[mqtt.Properties] = None, reasonCodes: Optional[List[mqtt.ReasonCodes]] = None, ) -> None: try: self._pending_unsubscribes.pop(mid).set() except KeyError: MQTT_LOGGER.error( f'Unexpected message ID "{mid}" in on_unsubscribe callback' ) def _on_publish(self, client: mqtt.Client, userdata: Any, mid: int) -> None: try: self._pending_publishes.pop(mid).set() except KeyError: # Do nothing since [2] may call on_publish before it even returns. # That is, the message may already be published before we even get a # chance to set up the 'pending_call' logic. pass def _on_socket_open( self, client: mqtt.Client, userdata: Any, sock: socket.socket ) -> None: def cb() -> None: # client.loop_read() may raise an exception, such as BadPipe. It's # usually a sign that the underlaying connection broke, therefore we # disconnect straight away try: client.loop_read() except Exception as exc: self._disconnected.set_exception(exc) self._loop.add_reader(sock.fileno(), cb) # paho-mqtt calls this function from the executor thread on which we've called # `self._client.connect()` (see [3]), so we create a callback function to schedule # `_misc_loop()` and run it on the loop thread-safely. def create_task_cb() -> None: self._misc_task = self._loop.create_task(self._misc_loop()) self._loop.call_soon_threadsafe(create_task_cb) def _on_socket_close( self, client: mqtt.Client, userdata: Any, sock: socket.socket ) -> None: fileno = sock.fileno() if fileno > -1: self._loop.remove_reader(fileno) if self._misc_task is not None and not self._misc_task.done(): with suppress(asyncio.CancelledError): self._misc_task.cancel() def _on_socket_register_write( self, client: mqtt.Client, userdata: Any, sock: socket.socket ) -> None: def cb() -> None: # client.loop_write() may raise an exception, such as BadPipe. It's # usually a sign that the underlaying connection broke, therefore we # disconnect straight away try: client.loop_write() except Exception as exc: self._disconnected.set_exception(exc) self._loop.add_writer(sock, cb) def _on_socket_unregister_write( self, client: mqtt.Client, userdata: Any, sock: socket.socket ) -> None: self._loop.remove_writer(sock) async def _misc_loop(self) -> None: while self._client.loop_misc() == mqtt.MQTT_ERR_SUCCESS: await asyncio.sleep(1) async def __aenter__(self) -> "Client": """Connect to the broker.""" await self.connect() return self async def __aexit__( self, exc_type: Type[Exception], exc: Exception, tb: TracebackType ) -> None: """Disconnect from the broker.""" # Early out if already disconnected... if self._disconnected.done(): disc_exc = self._disconnected.exception() if disc_exc is not None: # ...by raising the error that caused the disconnect raise disc_exc # ...by returning since the disconnect was intentional return # Try to gracefully disconnect from the broker try: await self.disconnect() except MqttError as error: # We tried to be graceful. Now there is no mercy. MQTT_LOGGER.warning( f'Could not gracefully disconnect due to "{error}". Forcing disconnection.' ) await self.force_disconnect() _PahoSocket = Union[socket.socket, mqtt.WebsocketWrapper] def _set_client_socket_defaults(client_socket: Optional[_PahoSocket], socket_options: Iterable[SocketOption]) -> None: # Note that socket may be None if, e.g., the username and # password combination didn't work. In this case, we return early. if client_socket is None: return # Furthermore, paho sometimes gives us a socket wrapper instead of # the raw socket. E.g., for WebSocket-based connections. if not isinstance(client_socket, socket.socket): return # At this point, we know that we got an actual socket. We change # some of the default options. for socket_option in socket_options: client_socket.setsockopt(*socket_option) asyncio-mqtt-0.10.0/asyncio_mqtt/error.py000066400000000000000000000030621407327071600204450ustar00rootroot00000000000000# SPDX-License-Identifier: BSD-3-Clause from typing import Any, Dict, Union import paho.mqtt.client as mqtt # type: ignore class MqttError(Exception): """Base exception for all asyncio-mqtt exceptions.""" pass class MqttCodeError(MqttError): def __init__(self, rc: Union[int, mqtt.ReasonCodes], *args: Any): super().__init__(*args) self.rc = rc def __str__(self) -> str: if isinstance(self.rc, mqtt.ReasonCodes): return f"[code:{self.rc.value}] {str(self.rc)}" else: return f"[code:{self.rc}] {super().__str__()}" class MqttConnectError(MqttCodeError): def __init__(self, rc: Union[int, mqtt.ReasonCodes]): if isinstance(rc, mqtt.ReasonCodes): return super().__init__(rc) msg = "Connection refused" try: msg += f": {_CONNECT_RC_STRINGS[rc]}" except KeyError: pass super().__init__(rc, msg) _CONNECT_RC_STRINGS: Dict[int, str] = { # Reference: https://github.com/eclipse/paho.mqtt.python/blob/v1.5.0/src/paho/mqtt/client.py#L1898 # 0: Connection successful # 1: Connection refused - incorrect protocol version 1: "Incorrect protocol version", # 2: Connection refused - invalid client identifier 2: "Invalid client identifier", # 3: Connection refused - server unavailable 3: "Server unavailable", # 4: Connection refused - bad username or password 4: "Bad username or password", # 5: Connection refused - not authorised 5: "Not authorised" # 6-255: Currently unused. } asyncio-mqtt-0.10.0/asyncio_mqtt/py.typed000066400000000000000000000000001407327071600204260ustar00rootroot00000000000000asyncio-mqtt-0.10.0/asyncio_mqtt/types.py000066400000000000000000000002001407327071600204470ustar00rootroot00000000000000from typing import Optional, TypeVar, Union T = TypeVar("T") PayloadType = Optional[Union[str, bytes, bytearray, int, float]] asyncio-mqtt-0.10.0/asyncio_mqtt/version.py000066400000000000000000000000771407327071600210040ustar00rootroot00000000000000# SPDX-License-Identifier: BSD-3-Clause __version__ = "0.10.0" asyncio-mqtt-0.10.0/examples/000077500000000000000000000000001407327071600160455ustar00rootroot00000000000000asyncio-mqtt-0.10.0/examples/EXAMPLES.md000066400000000000000000000052231407327071600176070ustar00rootroot00000000000000![license](https://img.shields.io/github/license/sbtinstruments/asyncio-mqtt) ![semver](https://img.shields.io/github/v/tag/sbtinstruments/asyncio-mqtt?sort=semver) ### Sending a JSON payload ##### Subscriber The following example describes a subscriber expecting to receive a JSON payload, which, also, connects to the client using Basic Auth and specifying the MQTT protocol to be used. Please beware that some MQTT brokers requires you to specify the protocol to be used. For more arguments that the Client may accept, please check [client.py](https://github.com/sbtinstruments/asyncio-mqtt/blob/f4736adf0d3c5b87a39ea27afd025ed58c7bb54c/asyncio_mqtt/client.py#L70) Please observe that the content which is decoded is not the message received but the content (payload) which is of the type [MQTTMessage](https://github.com/eclipse/paho.mqtt.python/blob/c339cea2652a957d47de68eafb2a76736c1514e6/src/paho/mqtt/client.py#L355) ```python import json from asyncio_mqtt import Client, ProtocolVersion async with Client( "test.mosquitto.org", username="username", password="password", protocol=ProtocolVersion.V31 ) as client: async with client.filtered_messages("floors/+/humidity") as messages: # subscribe is done afterwards so that we just start receiving messages # from this point on await client.subscribe("floors/#") async for message in messages: print(message.topic) print(json.loads(message.payload)) ``` ##### Publisher The publisher, besides specifying the Protocol and using also Basic Auth to acess the broker, it also specifies the [QoS](https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/) desired ```python import json from asyncio_mqtt import Client, ProtocolVersion async with Client( "test.mosquitto.org", username="username", password="password", protocol=ProtocolVersion.V31 ) as client: message = {"state": 3} await client.publish("floors/bed_room/humidity", payload=json.dumps(message), qos=2, retain=False) ``` ## License ![license](https://img.shields.io/github/license/sbtinstruments/asyncio-mqtt) Note that the underlying paho-mqtt library is dual-licensed. One of the licenses is the so-called [Eclipse Distribution License v1.0](https://www.eclipse.org/org/documents/edl-v10.php). It is almost word-for-word identical to the [BSD 3-clause License](https://opensource.org/licenses/BSD-3-Clause). The only differences are: * One use of "COPYRIGHT OWNER" (EDL) instead of "COPYRIGHT HOLDER" (BSD) * One use of "Eclipse Foundation, Inc." (EDL) instead of "copyright holder" (BSD) asyncio-mqtt-0.10.0/setup.py000066400000000000000000000023751407327071600157500ustar00rootroot00000000000000# SPDX-License-Identifier: BSD-3-Clause from setuptools import setup, find_packages with open("asyncio_mqtt/version.py", "r") as f: exec(f.read()) with open("README.md", "r") as readme_file: readme = readme_file.read() setup( name="asyncio_mqtt", version=__version__, packages=find_packages(), package_data={ "asyncio_mqtt": ["py.typed"], }, url="https://github.com/sbtinstruments/asyncio-mqtt", author="Frederik Aalund", author_email="fpa@sbtinstruments.com", description="Idomatic asyncio wrapper around paho-mqtt.", long_description=readme, long_description_content_type="text/markdown", license="BSD 3-clause License", classifiers=[ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved", "Operating System :: POSIX :: Linux", "Operating System :: Microsoft :: Windows", "Operating System :: MacOS", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", ], keywords="mqtt async asyncio paho-mqtt wrapper", install_requires=[ "paho-mqtt>=1.5.0", "async_generator;python_version<'3.7'", ], ) asyncio-mqtt-0.10.0/tests/000077500000000000000000000000001407327071600153715ustar00rootroot00000000000000asyncio-mqtt-0.10.0/tests/reconnect.py000066400000000000000000000023621407327071600177260ustar00rootroot00000000000000import asyncio import logging from asyncio_mqtt import Client, MqttError logger = logging.getLogger(__name__) async def tick(): while True: logger.info("Tick") await asyncio.sleep(1) async def test(): while True: try: logger.info("Connecting to MQTT") async with Client("localhost") as client: logger.info("Connection to MQTT open") async with client.unfiltered_messages() as messages: await client.subscribe("#") async for message in messages: logger.info( "Message %s %s", message.topic, message.payload.decode() ) # await asyncio.sleep(2) except MqttError as e: logger.error("Connection to MQTT closed: " + str(e)) except Exception: logger.exception("Connection to MQTT closed") await asyncio.sleep(3) def main(): logging.basicConfig( level=logging.DEBUG, format="%(asctime)s %(levelname)s %(message)s", datefmt="%Y-%m-%d %H:%M:%S", ) asyncio.get_event_loop().run_until_complete(asyncio.wait([test(), tick()])) if __name__ == "__main__": main()