pax_global_header00006660000000000000000000000064143776024700014524gustar00rootroot0000000000000052 comment=e38557217daa66d350dc009a74629d06d25c665d python-netfilterqueue-1.1.0/000077500000000000000000000000001437760247000161035ustar00rootroot00000000000000python-netfilterqueue-1.1.0/.github/000077500000000000000000000000001437760247000174435ustar00rootroot00000000000000python-netfilterqueue-1.1.0/.github/workflows/000077500000000000000000000000001437760247000215005ustar00rootroot00000000000000python-netfilterqueue-1.1.0/.github/workflows/ci.yml000066400000000000000000000021501437760247000226140ustar00rootroot00000000000000name: CI on: push: branches: - master pull_request: jobs: Ubuntu: name: 'Ubuntu (${{ matrix.python }}${{ matrix.extra_name }})' timeout-minutes: 10 runs-on: 'ubuntu-latest' strategy: fail-fast: false matrix: python: - '3.7' - '3.8' - '3.9' - '3.10' - '3.11' - 'pypy-3.7' - 'pypy-3.8' - 'pypy-3.9' check_lint: ['0'] extra_name: [''] include: - python: '3.9' check_lint: '1' extra_name: ', check lint' steps: - name: Checkout uses: actions/checkout@v2 - name: Setup python uses: actions/setup-python@v2 with: python-version: ${{ fromJSON(format('["{0}", "{1}"]', format('{0}.0-alpha - {0}.X', matrix.python), matrix.python))[startsWith(matrix.python, 'pypy')] }} - name: Run tests run: ./ci.sh env: CHECK_LINT: '${{ matrix.check_lint }}' # Should match 'name:' up above JOB_NAME: 'Ubuntu (${{ matrix.python }}${{ matrix.extra_name }})' python-netfilterqueue-1.1.0/.gitignore000066400000000000000000000000331437760247000200670ustar00rootroot00000000000000*.so build/ dist/ MANIFEST python-netfilterqueue-1.1.0/CHANGES.txt000066400000000000000000000035631437760247000177230ustar00rootroot00000000000000v1.1.0, 1 Mar 2023 Add Packet accessors for {indev, outdev, physindev, physoutdev} interface indices v1.0.0, 14 Jan 2022 Propagate exceptions raised by the user's packet callback Avoid calls to the packet callback during queue unbinding Raise an error if a packet verdict is set after its parent queue is closed set_payload() now affects the result of later get_payload() Handle signals received when run() is blocked in recv() Accept packets in COPY_META mode, only failing on an attempt to access the payload Add a parameter NetfilterQueue(sockfd=N) that uses an already-opened Netlink socket Add type hints Remove the Packet.payload attribute; it was never safe (treated as a char* but not NUL-terminated) nor documented, but was exposed in the API (perhaps inadvertently). v0.9.0, 12 Jan 2022 Improve usability when Packet objects are retained past the callback Add Packet.retain() to save the packet contents in such cases Eliminate warnings during build on py3 Add CI and basic test suite Raise a warning, not an error, if we don't get the bufsize we want Don't allow bind() more than once on the same NetfilterQueue, since that would leak the old queue handle ** This will be the last version with support for Python 2.7. ** v0.8.1, 30 Jan 2017 Fix bug #25- crashing when used in OUTPUT or POSTROUTING chains v0.8, 15 Dec 2016 Add get_hw() Fix byte order bug in set_mark v0.7, 28 June 2016 Add Python 3 compatibility. Add sock_len argument to bind() Add block argument to run() Add run_socket() Fix COPY* constants Don't crash on double unlink() v0.6, 15 Apr 2013 Add get_mark. v0.5, 3 Apr 2013 Add repeat. v0.4, 24 Dec 2012 Add set_payload. v0.2, 13 May 2011 Rename NetfilterQueue to QueueHandler. Add API section to README.rst. v0.1, 12 May 2011 Initial release. python-netfilterqueue-1.1.0/LICENSE.txt000066400000000000000000000020571437760247000177320ustar00rootroot00000000000000Copyright (c) 2011, Kerkhoff Technologies Inc. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. python-netfilterqueue-1.1.0/MANIFEST.in000066400000000000000000000002151437760247000176370ustar00rootroot00000000000000include LICENSE.txt README.rst CHANGES.txt recursive-include netfilterqueue *.py *.pyx *.pxd *.c *.pyi py.typed recursive-include tests *.py python-netfilterqueue-1.1.0/README.rst000066400000000000000000000331361437760247000176000ustar00rootroot00000000000000.. image:: https://img.shields.io/pypi/v/netfilterqueue.svg :target: https://pypi.org/project/netfilterqueue :alt: Latest PyPI version .. image:: https://github.com/oremanj/python-netfilterqueue/actions/workflows/ci.yml/badge.svg?branch=master :target: https://github.com/oremanj/python-netfilterqueue/actions?query=branch%3Amaster :alt: Automated test status ============== NetfilterQueue ============== NetfilterQueue provides access to packets matched by an iptables rule in Linux. Packets so matched can be accepted, dropped, altered, reordered, or given a mark. libnetfilter_queue (the netfilter library, not this module) is part of the `Netfilter project `_. The current version of NetfilterQueue requires Python 3.6 or later. The last version with support for Python 2.7 was 0.9.0. Example ======= The following script prints a short description of each packet before accepting it. :: from netfilterqueue import NetfilterQueue def print_and_accept(pkt): print(pkt) pkt.accept() nfqueue = NetfilterQueue() nfqueue.bind(1, print_and_accept) try: nfqueue.run() except KeyboardInterrupt: print('') nfqueue.unbind() You can also make your own socket so that it can be used with gevent, for example. :: from netfilterqueue import NetfilterQueue import socket def print_and_accept(pkt): print(pkt) pkt.accept() nfqueue = NetfilterQueue() nfqueue.bind(1, print_and_accept) s = socket.fromfd(nfqueue.get_fd(), socket.AF_UNIX, socket.SOCK_STREAM) try: nfqueue.run_socket(s) except KeyboardInterrupt: print('') s.close() nfqueue.unbind() To send packets destined for your LAN to the script, type something like:: iptables -I INPUT -d 192.168.0.0/24 -j NFQUEUE --queue-num 1 Installation ============ NetfilterQueue is a C extention module that links against libnetfilter_queue. Before installing, ensure you have: 1. A C compiler 2. Python development files 3. Libnetfilter_queue development files and associated dependencies On Debian or Ubuntu, install these files with:: apt-get install build-essential python3-dev libnetfilter-queue-dev From PyPI --------- To install from PyPI by pip:: pip install NetfilterQueue From source ----------- To install from source:: pip install cython git clone https://github.com/oremanj/python-netfilterqueue cd python-netfilterqueue pip install . API === ``NetfilterQueue.COPY_NONE``, ``NetfilterQueue.COPY_META``, ``NetfilterQueue.COPY_PACKET`` These constants specify how much of the packet should be given to the script: nothing, metadata, or the whole packet. NetfilterQueue objects ---------------------- A NetfilterQueue object represents a single queue. Configure your queue with a call to ``bind``, then start receiving packets with a call to ``run``. ``NetfilterQueue.bind(queue_num, callback, max_len=1024, mode=COPY_PACKET, range=65535, sock_len=...)`` Create and bind to the queue. ``queue_num`` uniquely identifies this queue for the kernel. It must match the ``--queue-num`` in your iptables rule, but there is no ordering requirement: it's fine to either ``bind()`` first or set up the iptables rule first. ``callback`` is a function or method that takes one argument, a Packet object (see below). ``max_len`` sets the largest number of packets that can be in the queue; new packets are dropped if the size of the queue reaches this number. ``mode`` determines how much of the packet data is provided to your script. Use the constants above. ``range`` defines how many bytes of the packet you want to get. For example, if you only want the source and destination IPs of a IPv4 packet, ``range`` could be 20. ``sock_len`` sets the receive socket buffer size. ``NetfilterQueue.unbind()`` Remove the queue. Packets matched by your iptables rule will be dropped. ``NetfilterQueue.get_fd()`` Get the file descriptor of the socket used to receive queued packets and send verdicts. If you're using an async event loop, you can poll this FD for readability and call ``run(False)`` every time data appears on it. ``NetfilterQueue.run(block=True)`` Send packets to your callback. By default, this method blocks, running until an exception is raised (such as by Ctrl+C). Set ``block=False`` to process the pending messages without waiting for more; in conjunction with the ``get_fd`` method, you can use this to integrate with async event loops. ``NetfilterQueue.run_socket(socket)`` Send packets to your callback, but use the supplied socket instead of recv, so that, for example, gevent can monkeypatch it. You can make a socket with ``socket.fromfd(nfqueue.get_fd(), socket.AF_NETLINK, socket.SOCK_RAW)`` and optionally make it non-blocking with ``socket.setblocking(False)``. Packet objects -------------- Objects of this type are passed to your callback. ``Packet.get_payload()`` Return the packet's payload as a bytes object. The returned value starts with the IP header. You must call ``retain()`` if you want to be able to ``get_payload()`` after your callback has returned. If you have already called ``set_payload()``, then ``get_payload()`` returns what you passed to ``set_payload()``. ``Packet.set_payload(payload)`` Set the packet payload. Call this before ``accept()`` if you want to change the contents of the packet before allowing it to be released. Don't forget to update the transport-layer checksum (or clear it, if you're using UDP), or else the recipient is likely to drop the packet. If you're changing the length of the packet, you'll also need to update the IP length, IP header checksum, and probably some transport-level fields (such as UDP length for UDP). ``Packet.get_payload_len()`` Return the size of the payload. ``Packet.set_mark(mark)`` Give the packet a kernel mark, which can be used in future iptables rules. ``mark`` is a 32-bit number. ``Packet.get_mark()`` Get the mark on the packet (either the one you set using ``set_mark()``, or the one it arrived with if you haven't called ``set_mark()``). ``Packet.get_hw()`` Return the source hardware address of the packet as a Python bytestring, or ``None`` if the source hardware address was not captured (packets captured by the ``OUTPUT`` or ``PREROUTING`` hooks). For example, on Ethernet the result will be a six-byte MAC address. The destination hardware address is not available because it is determined in the kernel only after packet filtering is complete. ``Packet.get_timestamp()`` Return the time at which this packet was received by the kernel, as a floating-point Unix timestamp with microsecond precision (comparable to the result of ``time.time()``, for example). Packets captured by the ``OUTPUT`` or ``POSTROUTING`` hooks do not have a timestamp, and ``get_timestamp()`` will return 0.0 for them. ``Packet.id`` The identifier assigned to this packet by the kernel. Typically the first packet received by your queue starts at 1 and later ones count up from there. ``Packet.hw_protocol`` The link-layer protocol for this packet. For example, IPv4 packets on Ethernet would have this set to the EtherType for IPv4, which is ``0x0800``. ``Packet.mark`` The mark that had been assigned to this packet when it was enqueued. Unlike the result of ``get_mark()``, this does not change if you call ``set_mark()``. ``Packet.hook`` The netfilter hook (iptables chain, roughly) that diverted this packet into our queue. Values 0 through 4 correspond to PREROUTING, INPUT, FORWARD, OUTPUT, and POSTROUTING respectively. ``Packet.indev``, ``Packet.outdev``, ``Packet.physindev``, ``Packet.physoutdev`` The interface indices on which the packet arrived (``indev``) or is slated to depart (``outdev``). These are integers, which can be converted to names like "eth0" by using ``socket.if_indextoname()``. Zero means no interface is applicable, either because the packet was locally generated or locally received, or because the interface information wasn't available when the packet was queued (for example, ``PREROUTING`` rules don't yet know the ``outdev``). If the ``indev`` or ``outdev`` refers to a bridge device, then the corresponding ``physindev`` or ``physoutdev`` will name the bridge member on which the actual traffic occurred; otherwise ``physindev`` and ``physoutdev`` will be zero. ``Packet.retain()`` Allocate a copy of the packet payload for use after the callback has returned. ``get_payload()`` will raise an exception at that point if you didn't call ``retain()``. ``Packet.accept()`` Accept the packet. You can reorder packets by accepting them in a different order than the order in which they were passed to your callback. ``Packet.drop()`` Drop the packet. ``Packet.repeat()`` Restart processing of this packet from the beginning of its Netfilter hook (iptables chain, roughly). Any changes made using ``set_payload()`` or ``set_mark()`` are preserved; in the absence of such changes, the packet will probably come right back to the same queue. Callback objects ---------------- Your callback can be any one-argument callable and will be invoked with a ``Packet`` object as argument. You must call ``retain()`` within the callback if you want to be able to ``get_payload()`` after the callback has returned. You can hang onto ``Packet`` objects and resolve them later, but note that packets continue to count against the queue size limit until they've been given a verdict (accept, drop, or repeat). Also, the kernel stores the enqueued packets in a linked list, so keeping lots of packets outstanding is likely to adversely impact performance. Monitoring a different network namespace ---------------------------------------- If you are using Linux network namespaces (``man 7 network_namespaces``) in some kind of containerization system, all of the Netfilter queue state is kept per-namespace; queue 1 in namespace X is not the same as queue 1 in namespace Y. NetfilterQueue will ordinarily pass you the traffic for the network namespace you're a part of. If you want to monitor a different one, you can do so with a bit of trickery and cooperation from a process in that namespace; this section describes how. You'll need to arrange for a process in the network namespace you want to monitor to call ``socket(AF_NETLINK, SOCK_RAW, 12)`` and pass you the resulting file descriptor using something like ``socket.send_fds()`` over a Unix domain socket. (12 is ``NETLINK_NETFILTER``, a constant which is not exposed by the Python ``socket`` module.) Once you've received that file descriptor in your process, you can create a NetfilterQueue object using the special constructor ``NetfilterQueue(sockfd=N)`` where N is the file descriptor you received. Because the socket was originally created in the other network namespace, the kernel treats it as part of that namespace, and you can use it to access that namespace even though it's not the namespace you're in yourself. Usage ===== To send packets to the queue:: iptables -I -j NFQUEUE --queue-num For example:: iptables -I INPUT -d 192.168.0.0/24 -j NFQUEUE --queue-num 1 The only special part of the rule is the target. Rules can have any match and can be added to any table or chain. Valid queue numbers are integers from 0 to 65,535 inclusive. To view libnetfilter_queue stats, refer to /proc/net/netfilter/nfnetlink_queue:: cat /proc/net/netfilter/nfnetlink_queue 1 31621 0 2 4016 0 0 2 1 The fields are: 1. Queue ID 2. Bound process ID 3. Number of currently queued packets 4. Copy mode 5. Copy size 6. Number of packets dropped due to reaching max queue size 7. Number of packets dropped due to netlink socket failure 8. Total number of packets sent to queue 9. Something for libnetfilter_queue's internal use Limitations =========== * We use a fixed-size 4096-byte buffer for packets, so you are likely to see truncation on loopback and on Ethernet with jumbo packets. If this is a problem, either lower the MTU on your loopback, disable jumbo packets, or get Cython, change ``DEF BufferSize = 4096`` in ``netfilterqueue.pyx``, and rebuild. * Not all information available from libnetfilter_queue is exposed: missing pieces include packet input/output network interface names, checksum offload flags, UID/GID and security context data associated with the packet (if any). * Not all information available from the kernel is even processed by libnetfilter_queue: missing pieces include additional link-layer header data for some packets (including VLAN tags), connection-tracking state, and incoming packet length (if truncated for queueing). * We do not expose the libnetfilter_queue interface for changing queue flags. Most of these pertain to other features we don't support (listed above), but there's one that could set the queue to accept (rather than dropping) packets received when it's full. Source ====== https://github.com/oremanj/python-netfilterqueue Authorship ========== python-netfilterqueue was originally written by Matthew Fox of Kerkhoff Technologies, Inc. Since 2022 it has been maintained by Joshua Oreman of Hudson River Trading LLC. Both authors wish to thank their employers for their support of open source. License ======= Copyright (c) 2011, Kerkhoff Technologies, Inc, and contributors. `MIT licensed `_ python-netfilterqueue-1.1.0/ci.sh000077500000000000000000000025201437760247000170340ustar00rootroot00000000000000#!/bin/bash set -ex -o pipefail pip install -U pip setuptools wheel sudo apt-get install libnetfilter-queue-dev # Cython is required to build the sdist... pip install cython python setup.py sdist --formats=zip # ... but not to install it pip uninstall -y cython python setup.py build_ext pip install dist/*.zip pip install -Ur test-requirements.txt if [ "$CHECK_LINT" = "1" ]; then error=0 black_files="setup.py tests netfilterqueue" if ! black --check $black_files; then error=$? black --diff $black_files fi mypy --strict -p netfilterqueue || error=$? ( mkdir empty; cd empty; python -m mypy.stubtest netfilterqueue ) || error=$? if [ $error -ne 0 ]; then cat <": ctypedef unsigned char u_int8_t ctypedef unsigned short int u_int16_t ctypedef unsigned int u_int32_t cdef extern from "": int dup2(int oldfd, int newfd) cdef extern from "": int errno # dummy defines from asm-generic/errno.h: cdef enum: EINTR = 4 EAGAIN = 11 # Try again EWOULDBLOCK = EAGAIN ENOBUFS = 105 # No buffer space available cdef extern from "": struct iphdr: u_int8_t tos u_int16_t tot_len u_int16_t id u_int16_t frag_off u_int8_t ttl u_int8_t protocol u_int16_t check u_int32_t saddr u_int32_t daddr # Dummy defines from netinet/in.h: cdef enum: IPPROTO_IP = 0 # Dummy protocol for TCP. IPPROTO_HOPOPTS = 0 # IPv6 Hop-by-Hop options. IPPROTO_ICMP = 1 # Internet Control Message Protocol. IPPROTO_IGMP = 2 # Internet Group Management Protocol. */ IPPROTO_IPIP = 4 # IPIP tunnels (older KA9Q tunnels use 94). IPPROTO_TCP = 6 # Transmission Control Protocol. IPPROTO_EGP = 8 # Exterior Gateway Protocol. IPPROTO_PUP = 12 # PUP protocol. IPPROTO_UDP = 17 # User Datagram Protocol. IPPROTO_IDP = 22 # XNS IDP protocol. IPPROTO_TP = 29 # SO Transport Protocol Class 4. IPPROTO_IPV6 = 41 # IPv6 header. IPPROTO_ROUTING = 43 # IPv6 routing header. IPPROTO_FRAGMENT = 44 # IPv6 fragmentation header. IPPROTO_RSVP = 46 # Reservation Protocol. IPPROTO_GRE = 47 # General Routing Encapsulation. IPPROTO_ESP = 50 # encapsulating security payload. IPPROTO_AH = 51 # authentication header. IPPROTO_ICMPV6 = 58 # ICMPv6. IPPROTO_NONE = 59 # IPv6 no next header. IPPROTO_DSTOPTS = 60 # IPv6 destination options. IPPROTO_MTP = 92 # Multicast Transport Protocol. IPPROTO_ENCAP = 98 # Encapsulation Header. IPPROTO_PIM = 103 # Protocol Independent Multicast. IPPROTO_COMP = 108 # Compression Header Protocol. IPPROTO_SCTP = 132 # Stream Control Transmission Protocol. IPPROTO_RAW = 255 # Raw IP packets. IPPROTO_MAX cdef extern from "Python.h": object PyBytes_FromStringAndSize(char *s, Py_ssize_t len) object PyString_FromStringAndSize(char *s, Py_ssize_t len) cdef extern from "": ctypedef long time_t struct timeval: time_t tv_sec time_t tv_usec struct timezone: pass cdef extern from "": u_int32_t ntohl (u_int32_t __netlong) nogil u_int16_t ntohs (u_int16_t __netshort) nogil u_int32_t htonl (u_int32_t __hostlong) nogil u_int16_t htons (u_int16_t __hostshort) nogil cdef extern from "libnfnetlink/linux_nfnetlink.h": struct nfgenmsg: u_int8_t nfgen_family u_int8_t version u_int16_t res_id cdef extern from "libnfnetlink/libnfnetlink.h": struct nfnl_handle: pass nfnl_handle *nfnl_open() void nfnl_close(nfnl_handle *h) int nfnl_fd(nfnl_handle *h) unsigned int nfnl_rcvbufsiz(nfnl_handle *h, unsigned int size) cdef extern from "libnetfilter_queue/linux_nfnetlink_queue.h": enum nfqnl_config_mode: NFQNL_COPY_NONE NFQNL_COPY_META NFQNL_COPY_PACKET struct nfqnl_msg_packet_hdr: u_int32_t packet_id u_int16_t hw_protocol u_int8_t hook cdef extern from "libnetfilter_queue/libnetfilter_queue.h": struct nfq_handle: pass struct nfq_q_handle: pass struct nfq_data: pass struct nfqnl_msg_packet_hw: u_int8_t hw_addr[8] nfq_handle *nfq_open() nfq_handle *nfq_open_nfnl(nfnl_handle *h) int nfq_close(nfq_handle *h) int nfq_bind_pf(nfq_handle *h, u_int16_t pf) int nfq_unbind_pf(nfq_handle *h, u_int16_t pf) ctypedef int *nfq_callback(nfq_q_handle *gh, nfgenmsg *nfmsg, nfq_data *nfad, void *data) nfq_q_handle *nfq_create_queue(nfq_handle *h, u_int16_t num, nfq_callback *cb, void *data) # Any function that parses Netlink replies might invoke the user # callback and thus might need to propagate a Python exception. # This includes nfq_handle_packet but is not limited to that -- # other functions might send a query, read until they get the reply, # and find a packet notification before the reply which they then # must deal with. int nfq_destroy_queue(nfq_q_handle *qh) except? -1 int nfq_handle_packet(nfq_handle *h, char *buf, int len) except? -1 int nfq_set_mode(nfq_q_handle *qh, u_int8_t mode, unsigned int len) except? -1 int nfq_set_queue_maxlen(nfq_q_handle *qh, u_int32_t queuelen) except? -1 int nfq_set_verdict(nfq_q_handle *qh, u_int32_t id, u_int32_t verdict, u_int32_t data_len, unsigned char *buf) nogil int nfq_set_verdict2(nfq_q_handle *qh, u_int32_t id, u_int32_t verdict, u_int32_t mark, u_int32_t datalen, unsigned char *buf) nogil int nfq_fd(nfq_handle *h) nfqnl_msg_packet_hdr *nfq_get_msg_packet_hdr(nfq_data *nfad) int nfq_get_payload(nfq_data *nfad, unsigned char **data) int nfq_get_timestamp(nfq_data *nfad, timeval *tv) nfqnl_msg_packet_hw *nfq_get_packet_hw(nfq_data *nfad) int nfq_get_nfmark(nfq_data *nfad) u_int32_t nfq_get_indev(nfq_data *nfad) u_int32_t nfq_get_outdev(nfq_data *nfad) u_int32_t nfq_get_physindev(nfq_data *nfad) u_int32_t nfq_get_physoutdev(nfq_data *nfad) nfnl_handle *nfq_nfnlh(nfq_handle *h) # Dummy defines from linux/socket.h: cdef enum: # Protocol families, same as address families. PF_INET = 2 PF_INET6 = 10 PF_NETLINK = 16 cdef extern from "": ssize_t recv(int __fd, void *__buf, size_t __n, int __flags) nogil int MSG_DONTWAIT # Dummy defines from linux/netfilter.h cdef enum: NF_DROP NF_ACCEPT NF_STOLEN NF_QUEUE NF_REPEAT NF_STOP NF_MAX_VERDICT = NF_STOP cdef class NetfilterQueue: cdef object __weakref__ cdef object user_callback # User callback cdef nfq_handle *h # Handle to NFQueue library cdef nfq_q_handle *qh # A handle to the queue cdef class Packet: cdef NetfilterQueue _queue cdef bint _verdict_is_set # True if verdict has been issued, false otherwise cdef bint _mark_is_set # True if a mark has been given, false otherwise cdef bint _hwaddr_is_set cdef bint _timestamp_is_set cdef u_int32_t _given_mark # Mark given to packet cdef bytes _given_payload # New payload of packet, or null cdef bytes _owned_payload # From NFQ packet header: cdef readonly u_int32_t id cdef readonly u_int16_t hw_protocol cdef readonly u_int8_t hook cdef readonly u_int32_t mark # Packet details: cdef Py_ssize_t payload_len cdef unsigned char *payload cdef timeval timestamp cdef u_int8_t hw_addr[8] cdef readonly u_int32_t indev cdef readonly u_int32_t physindev cdef readonly u_int32_t outdev cdef readonly u_int32_t physoutdev cdef set_nfq_data(self, NetfilterQueue queue, nfq_data *nfa) cdef drop_refs(self) cdef int verdict(self, u_int8_t verdict) except -1 cpdef Py_ssize_t get_payload_len(self) cpdef double get_timestamp(self) cpdef bytes get_payload(self) cpdef set_payload(self, bytes payload) cpdef set_mark(self, u_int32_t mark) cpdef get_mark(self) cpdef retain(self) cpdef accept(self) cpdef drop(self) cpdef repeat(self) python-netfilterqueue-1.1.0/netfilterqueue/_impl.pyi000066400000000000000000000024721437760247000227740ustar00rootroot00000000000000import socket from enum import IntEnum from typing import Callable, Dict, Optional, Tuple COPY_NONE: int COPY_META: int COPY_PACKET: int class Packet: hook: int hw_protocol: int id: int mark: int # These are ifindexes, pass to socket.if_indextoname() to get names: indev: int outdev: int physindev: int physoutdev: int def get_hw(self) -> Optional[bytes]: ... def get_payload(self) -> bytes: ... def get_payload_len(self) -> int: ... def get_timestamp(self) -> float: ... def get_mark(self) -> int: ... def set_payload(self, payload: bytes) -> None: ... def set_mark(self, mark: int) -> None: ... def retain(self) -> None: ... def accept(self) -> None: ... def drop(self) -> None: ... def repeat(self) -> None: ... class NetfilterQueue: def __new__(self, *, af: int = ..., sockfd: int = ...) -> NetfilterQueue: ... def bind( self, queue_num: int, user_callback: Callable[[Packet], None], max_len: int = ..., mode: int = COPY_PACKET, range: int = ..., sock_len: int = ..., ) -> None: ... def unbind(self) -> None: ... def get_fd(self) -> int: ... def run(self, block: bool = ...) -> None: ... def run_socket(self, s: socket.socket) -> None: ... PROTOCOLS: Dict[int, str] python-netfilterqueue-1.1.0/netfilterqueue/_impl.pyx000066400000000000000000000370141437760247000230130ustar00rootroot00000000000000""" Bind to a Linux netfilter queue. Send packets to a user-specified callback function. Copyright: (c) 2011, Kerkhoff Technologies Inc. License: MIT; see LICENSE.txt """ # Constants for module users COPY_NONE = 0 COPY_META = 1 COPY_PACKET = 2 # Packet copying defaults DEF DEFAULT_MAX_QUEUELEN = 1024 DEF MaxPacketSize = 0xFFFF DEF BufferSize = 4096 DEF MetadataSize = 80 DEF MaxCopySize = BufferSize - MetadataSize # Experimentally determined overhead DEF SockOverhead = 760+20 DEF SockCopySize = MaxCopySize + SockOverhead # Socket queue should hold max number of packets of copysize bytes DEF SockRcvSize = DEFAULT_MAX_QUEUELEN * SockCopySize // 2 from cpython.exc cimport PyErr_CheckSignals cdef extern from "Python.h": ctypedef struct PyTypeObject: const char* tp_name # A negative return value from this callback will stop processing and # make nfq_handle_packet return -1, so we use that as the error flag. cdef int global_callback(nfq_q_handle *qh, nfgenmsg *nfmsg, nfq_data *nfa, void *data) except -1 with gil: """Create a Packet and pass it to appropriate callback.""" cdef NetfilterQueue nfqueue = data cdef object user_callback = nfqueue.user_callback if user_callback is None: # Queue is being unbound; we can't send a verdict at this point # so just ignore the packet. The kernel will drop it once we # unbind. return 1 packet = Packet() packet.set_nfq_data(nfqueue, nfa) try: user_callback(packet) finally: packet.drop_refs() return 1 cdef class Packet: """A packet received from NetfilterQueue.""" def __cinit__(self): self._verdict_is_set = False self._mark_is_set = False self._given_payload = None def __str__(self): cdef unsigned char *payload = NULL if self._owned_payload: payload = self._owned_payload elif self.payload != NULL: payload = self.payload else: return "%d byte packet, contents unretained" % (self.payload_len,) cdef iphdr *hdr = payload protocol = PROTOCOLS.get(hdr.protocol, "Unknown protocol") return "%s packet, %s bytes" % (protocol, self.payload_len) cdef set_nfq_data(self, NetfilterQueue queue, nfq_data *nfa): """ Assign a packet from NFQ to this object. Parse the header and load local values. """ cdef nfqnl_msg_packet_hw *hw cdef nfqnl_msg_packet_hdr *hdr hdr = nfq_get_msg_packet_hdr(nfa) self._queue = queue self.id = ntohl(hdr.packet_id) self.hw_protocol = ntohs(hdr.hw_protocol) self.hook = hdr.hook hw = nfq_get_packet_hw(nfa) if hw == NULL: # nfq_get_packet_hw doesn't work on OUTPUT and PREROUTING chains self._hwaddr_is_set = False else: self.hw_addr = hw.hw_addr self._hwaddr_is_set = True self.payload_len = nfq_get_payload(nfa, &self.payload) if self.payload_len < 0: # Probably using a mode that doesn't provide the payload self.payload = NULL self.payload_len = 0 nfq_get_timestamp(nfa, &self.timestamp) self.mark = nfq_get_nfmark(nfa) self.indev = nfq_get_indev(nfa) self.outdev = nfq_get_outdev(nfa) self.physindev = nfq_get_physindev(nfa) self.physoutdev = nfq_get_physoutdev(nfa) cdef drop_refs(self): """ Called at the end of the user_callback, when the storage passed to set_nfq_data() is about to be reused. """ self.payload = NULL cdef int verdict(self, u_int8_t verdict) except -1: """Call appropriate set_verdict... function on packet.""" if self._verdict_is_set: raise RuntimeError("Verdict already given for this packet") if self._queue.qh == NULL: raise RuntimeError("Parent queue has already been unbound") cdef u_int32_t modified_payload_len = 0 cdef unsigned char *modified_payload = NULL if self._given_payload: modified_payload_len = len(self._given_payload) modified_payload = self._given_payload if self._mark_is_set: nfq_set_verdict2( self._queue.qh, self.id, verdict, self._given_mark, modified_payload_len, modified_payload) else: nfq_set_verdict( self._queue.qh, self.id, verdict, modified_payload_len, modified_payload) self._verdict_is_set = True def get_hw(self): """Return the packet's source MAC address as a Python bytestring, or None if it's not available. """ if not self._hwaddr_is_set: return None cdef object py_string py_string = PyBytes_FromStringAndSize(self.hw_addr, 8) return py_string cpdef bytes get_payload(self): """Return payload as Python string.""" if self._given_payload: return self._given_payload elif self._owned_payload: return self._owned_payload elif self.payload != NULL: return self.payload[:self.payload_len] elif self.payload_len == 0: raise RuntimeError( "Packet has no payload -- perhaps you're using COPY_META mode?" ) else: raise RuntimeError( "Payload data is no longer available. You must call " "retain() within the user_callback in order to copy " "the payload if you need to expect it after your " "callback has returned." ) cpdef Py_ssize_t get_payload_len(self): return self.payload_len cpdef double get_timestamp(self): return self.timestamp.tv_sec + (self.timestamp.tv_usec/1000000.0) cpdef set_payload(self, bytes payload): """Set the new payload of this packet.""" self._given_payload = payload cpdef set_mark(self, u_int32_t mark): self._given_mark = mark self._mark_is_set = True cpdef get_mark(self): if self._mark_is_set: return self._given_mark return self.mark cpdef retain(self): self._owned_payload = self.get_payload() cpdef accept(self): """Accept the packet.""" self.verdict(NF_ACCEPT) cpdef drop(self): """Drop the packet.""" self.verdict(NF_DROP) cpdef repeat(self): """Repeat the packet.""" self.verdict(NF_REPEAT) cdef class NetfilterQueue: """Handle a single numbered queue.""" def __cinit__(self, *, u_int16_t af = PF_INET, int sockfd = -1): cdef nfnl_handle *nlh = NULL try: if sockfd >= 0: # This is a hack to use the given Netlink socket instead # of the one allocated by nfq_open(). Intended use case: # the given socket was opened in a different network # namespace, and you want to monitor traffic in that # namespace from this process running outside of it. # Call socket(AF_NETLINK, SOCK_RAW, /*NETLINK_NETFILTER*/ 12) # in the other namespace and pass that fd here (via Unix # domain socket or similar). nlh = nfnl_open() if nlh == NULL: raise OSError(errno, "Failed to open nfnetlink handle") # At this point nfnl_get_fd(nlh) is a new netlink socket # and has been bound to an automatically chosen port id. # This dup2 will close it, freeing up that address. if dup2(sockfd, nfnl_fd(nlh)) < 0: raise OSError(errno, "dup2 failed") # Opening the netfilterqueue subsystem will rebind # the socket, using the same portid from the old socket, # which is hopefully now free. An alternative approach, # theoretically more robust against concurrent binds, # would be to autobind the new socket and write the chosen # address to nlh->local. nlh is an opaque type so this # would need to be done using memcpy (local starts # 4 bytes into the structure); let's avoid that unless # we really need it. self.h = nfq_open_nfnl(nlh) else: self.h = nfq_open() if self.h == NULL: raise OSError(errno, "Failed to open NFQueue.") except: if nlh != NULL: nfnl_close(nlh) raise nfq_unbind_pf(self.h, af) # This does NOT kick out previous queues if nfq_bind_pf(self.h, af) < 0: raise OSError("Failed to bind family %s. Are you root?" % af) def __del__(self): # unbind() can result in invocations of global_callback, so we # must do it in __del__ (when this is still a valid # NetfilterQueue object) rather than __dealloc__ self.unbind() def __dealloc__(self): # Don't call nfq_unbind_pf unless you want to disconnect any other # processes using this libnetfilter_queue on this protocol family! if self.h != NULL: nfq_close(self.h) def bind(self, int queue_num, object user_callback, u_int32_t max_len=DEFAULT_MAX_QUEUELEN, u_int8_t mode=NFQNL_COPY_PACKET, u_int32_t range=MaxPacketSize, u_int32_t sock_len=SockRcvSize): """Create and bind to a new queue.""" if self.qh != NULL: raise RuntimeError("A queue is already bound; use unbind() first") cdef unsigned int newsiz self.user_callback = user_callback self.qh = nfq_create_queue(self.h, queue_num, global_callback, self) if self.qh == NULL: raise OSError("Failed to create queue %s." % queue_num) if range > MaxCopySize: range = MaxCopySize if nfq_set_mode(self.qh, mode, range) < 0: self.unbind() raise OSError("Failed to set packet copy mode.") nfq_set_queue_maxlen(self.qh, max_len) newsiz = nfnl_rcvbufsiz(nfq_nfnlh(self.h), sock_len) if newsiz != sock_len * 2: try: import warnings warnings.warn( "Socket rcvbuf limit is now %d, requested %d." % (newsiz, sock_len), category=RuntimeWarning, ) except: # if warnings are being treated as errors self.unbind() raise def unbind(self): """Destroy the queue.""" self.user_callback = None if self.qh != NULL: nfq_destroy_queue(self.qh) self.qh = NULL # See warning about nfq_unbind_pf in __dealloc__ above. def get_fd(self): """Get the file descriptor of the queue handler.""" return nfq_fd(self.h) def run(self, block=True): """Accept packets using recv.""" cdef int fd = self.get_fd() cdef char buf[BufferSize] cdef int rv cdef int recv_flags recv_flags = 0 if block else MSG_DONTWAIT while True: with nogil: rv = recv(fd, buf, sizeof(buf), recv_flags) if rv < 0: if errno == EAGAIN: break if errno == ENOBUFS: # Kernel is letting us know we dropped a packet continue if errno == EINTR: PyErr_CheckSignals() continue raise OSError(errno, "recv failed") nfq_handle_packet(self.h, buf, rv) def run_socket(self, s): """Accept packets using socket.recv so that, for example, gevent can monkeypatch it.""" import socket while True: try: buf = s.recv(BufferSize) except socket.error as e: err = e.args[0] if err == ENOBUFS: continue elif err == EAGAIN or err == EWOULDBLOCK: # This should only happen with a non-blocking socket, and the # app should call run_socket again when more data is available. break else: # This is bad. Let the caller handle it. raise e else: nfq_handle_packet(self.h, buf, len(buf)) cdef void _fix_names(): # Avoid ._impl showing up in reprs. This doesn't work on PyPy; there we would # need to modify the name before PyType_Ready(), but I can't find any way to # write Cython code that would execute at that time. cdef PyTypeObject* tp = Packet tp.tp_name = "netfilterqueue.Packet" tp = NetfilterQueue tp.tp_name = "netfilterqueue.NetfilterQueue" _fix_names() PROTOCOLS = { 0: "HOPOPT", 1: "ICMP", 2: "IGMP", 3: "GGP", 4: "IP", 5: "ST", 6: "TCP", 7: "CBT", 8: "EGP", 9: "IGP", 10: "BBN-RCC-MON", 11: "NVP-II", 12: "PUP", 13: "ARGUS", 14: "EMCON", 15: "XNET", 16: "CHAOS", 17: "UDP", 18: "MUX", 19: "DCN-MEAS", 20: "HMP", 21: "PRM", 22: "XNS-IDP", 23: "TRUNK-1", 24: "TRUNK-2", 25: "LEAF-1", 26: "LEAF-2", 27: "RDP", 28: "IRTP", 29: "ISO-TP4", 30: "NETBLT", 31: "MFE-NSP", 32: "MERIT-INP", 33: "DCCP", 34: "3PC", 35: "IDPR", 36: "XTP", 37: "DDP", 38: "IDPR-CMTP", 39: "TP++", 40: "IL", 41: "IPv6", 42: "SDRP", 43: "IPv6-Route", 44: "IPv6-Frag", 45: "IDRP", 46: "RSVP", 47: "GRE", 48: "DSR", 49: "BNA", 50: "ESP", 51: "AH", 52: "I-NLSP", 53: "SWIPE", 54: "NARP", 55: "MOBILE", 56: "TLSP", 57: "SKIP", 58: "IPv6-ICMP", 59: "IPv6-NoNxt", 60: "IPv6-Opts", 61: "any host internal protocol", 62: "CFTP", 63: "any local network", 64: "SAT-EXPAK", 65: "KRYPTOLAN", 66: "RVD", 67: "IPPC", 68: "any distributed file system", 69: "SAT-MON", 70: "VISA", 71: "IPCV", 72: "CPNX", 73: "CPHB", 74: "WSN", 75: "PVP", 76: "BR-SAT-MON", 77: "SUN-ND", 78: "WB-MON", 79: "WB-EXPAK", 80: "ISO-IP", 81: "VMTP", 82: "SECURE-VMTP", 83: "VINES", 84: "TTP", 85: "NSFNET-IGP", 86: "DGP", 87: "TCF", 88: "EIGRP", 89: "OSPFIGP", 90: "Sprite-RPC", 91: "LARP", 92: "MTP", 93: "AX.25", 94: "IPIP", 95: "MICP", 96: "SCC-SP", 97: "ETHERIP", 98: "ENCAP", 99: "any private encryption scheme", 100: "GMTP", 101: "IFMP", 102: "PNNI", 103: "PIM", 104: "ARIS", 105: "SCPS", 106: "QNX", 107: "A/N", 108: "IPComp", 109: "SNP", 110: "Compaq-Peer", 111: "IPX-in-IP", 112: "VRRP", 113: "PGM", 114: "any 0-hop protocol", 115: "L2TP", 116: "DDX", 117: "IATP", 118: "STP", 119: "SRP", 120: "UTI", 121: "SMP", 122: "SM", 123: "PTP", 124: "ISIS", 125: "FIRE", 126: "CRTP", 127: "CRUDP", 128: "SSCOPMCE", 129: "IPLT", 130: "SPS", 131: "PIPE", 132: "SCTP", 133: "FC", 134: "RSVP-E2E-IGNORE", 135: "Mobility", 136: "UDPLite", 137: "MPLS-in-IP", 138: "manet", 139: "HIP", 140: "Shim6", 255: "Reserved", } python-netfilterqueue-1.1.0/netfilterqueue/_version.py000066400000000000000000000001551437760247000233430ustar00rootroot00000000000000# This file is imported from __init__.py and exec'd from setup.py __version__ = "1.1.0" VERSION = (1, 1, 0) python-netfilterqueue-1.1.0/netfilterqueue/py.typed000066400000000000000000000000001437760247000226310ustar00rootroot00000000000000python-netfilterqueue-1.1.0/pyproject.toml000066400000000000000000000001321437760247000210130ustar00rootroot00000000000000[build-system] requires = ["setuptools", "wheel"] build-backend = "setuptools.build_meta" python-netfilterqueue-1.1.0/setup.py000066400000000000000000000045171437760247000176240ustar00rootroot00000000000000import os, sys from setuptools import setup, Extension exec(open("netfilterqueue/_version.py", encoding="utf-8").read()) setup_requires = ["wheel"] try: # Use Cython from Cython.Build import cythonize ext_modules = cythonize( Extension( "netfilterqueue._impl", ["netfilterqueue/_impl.pyx"], libraries=["netfilter_queue"], ), compiler_directives={"language_level": "3str"}, ) except ImportError: # No Cython if "egg_info" in sys.argv: # We're being run by pip to figure out what we need. Request cython in # setup_requires below. setup_requires += ["cython"] elif not os.path.exists( os.path.join(os.path.dirname(__file__), "netfilterqueue/_impl.c") ): sys.stderr.write( "You must have Cython installed (`pip install cython`) to build this " "package from source.\nIf you're receiving this error when installing from " "PyPI, please file a bug report at " "https://github.com/oremanj/python-netfilterqueue/issues/new\n" ) sys.exit(1) ext_modules = [ Extension( "netfilterqueue._impl", ["netfilterqueue/_impl.c"], libraries=["netfilter_queue"], ) ] setup( name="NetfilterQueue", version=__version__, license="MIT", author="Matthew Fox , Joshua Oreman ", author_email="oremanj@gmail.com", url="https://github.com/oremanj/python-netfilterqueue", description="Python bindings for libnetfilter_queue", long_description=open("README.rst", encoding="utf-8").read(), packages=["netfilterqueue"], ext_modules=ext_modules, include_package_data=True, exclude_package_data={"netfilterqueue": ["*.c"]}, setup_requires=setup_requires, python_requires=">=3.6", classifiers=[ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License", "Operating System :: POSIX :: Linux", "Topic :: System :: Networking", "Topic :: Security", "Intended Audience :: Developers", "Intended Audience :: Telecommunications Industry", "Programming Language :: Cython", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", ], ) python-netfilterqueue-1.1.0/test-requirements.in000066400000000000000000000003721437760247000221350ustar00rootroot00000000000000git+https://github.com/NightTsarina/python-unshare.git@4e98c177bdeb24c5dcfcd66c457845a776bbb75c pytest trio pytest-trio async_generator black platformdirs <= 2.4.0 # needed by black; 2.4.1+ don't support py3.6 mypy; implementation_name == "cpython" python-netfilterqueue-1.1.0/test-requirements.txt000066400000000000000000000026161437760247000223510ustar00rootroot00000000000000# # This file is autogenerated by pip-compile with python 3.9 # To update, run: # # pip-compile test-requirements.in # async-generator==1.10 # via # -r test-requirements.in # pytest-trio # trio attrs==21.4.0 # via # outcome # pytest # trio black==21.12b0 # via -r test-requirements.in click==8.0.3 # via black idna==3.3 # via trio iniconfig==1.1.1 # via pytest mypy==0.931 ; implementation_name == "cpython" # via -r test-requirements.in mypy-extensions==0.4.3 # via # black # mypy outcome==1.1.0 # via # pytest-trio # trio packaging==21.3 # via pytest pathspec==0.9.0 # via black platformdirs==2.4.0 # via # -r test-requirements.in # black pluggy==1.0.0 # via pytest py==1.11.0 # via pytest pyparsing==3.0.6 # via packaging pytest==6.2.5 # via # -r test-requirements.in # pytest-trio pytest-trio==0.7.0 # via -r test-requirements.in python-unshare @ git+https://github.com/NightTsarina/python-unshare.git@4e98c177bdeb24c5dcfcd66c457845a776bbb75c # via -r test-requirements.in sniffio==1.2.0 # via trio sortedcontainers==2.4.0 # via trio toml==0.10.2 # via pytest tomli==1.2.3 # via # black # mypy trio==0.19.0 # via # -r test-requirements.in # pytest-trio typing-extensions==4.0.1 # via # black # mypy python-netfilterqueue-1.1.0/tests/000077500000000000000000000000001437760247000172455ustar00rootroot00000000000000python-netfilterqueue-1.1.0/tests/conftest.py000066400000000000000000000261471437760247000214560ustar00rootroot00000000000000import math import os import pytest import socket import subprocess import sys import trio import unshare # type: ignore import netfilterqueue from functools import partial from typing import Any, AsyncIterator, Callable, Dict, Optional, Tuple from async_generator import asynccontextmanager from pytest_trio.enable_trio_mode import * # type: ignore # We'll create three network namespaces, representing a router (which # has interfaces on ROUTER_IP[1, 2]) and two hosts connected to it # (PEER_IP[1, 2] respectively). The router (in the parent pytest # process) will configure netfilterqueue iptables rules and use them # to intercept and modify traffic between the two hosts (each of which # is implemented in a subprocess). # # The 'peer' subprocesses communicate with each other over UDP, and # with the router parent over a UNIX domain SOCK_SEQPACKET socketpair. # Each packet sent from the parent to one peer over the UNIX domain # socket will be forwarded to the other peer over UDP. Each packet # received over UDP by either of the peers will be forwarded to its # parent. ROUTER_IP = {1: "172.16.101.1", 2: "172.16.102.1"} PEER_IP = {1: "172.16.101.2", 2: "172.16.102.2"} def enter_netns() -> None: # Create new namespaces of the other types we need unshare.unshare(unshare.CLONE_NEWNS | unshare.CLONE_NEWNET) # Mount /sys so network tools work subprocess.run("/bin/mount -t sysfs sys /sys".split(), check=True) # Bind-mount /run so iptables can get its lock subprocess.run("/bin/mount -t tmpfs tmpfs /run".split(), check=True) # Set up loopback interface subprocess.run("/sbin/ip link set lo up".split(), check=True) @pytest.hookimpl(tryfirst=True) # type: ignore def pytest_runtestloop() -> None: if os.getuid() != 0: # Create a new user namespace for the whole test session outer = {"uid": os.getuid(), "gid": os.getgid()} unshare.unshare(unshare.CLONE_NEWUSER) with open("/proc/self/setgroups", "wb") as fp: # This is required since we're unprivileged outside the namespace fp.write(b"deny") for idtype in ("uid", "gid"): with open(f"/proc/self/{idtype}_map", "wb") as fp: fp.write(b"0 %d 1" % (outer[idtype],)) assert os.getuid() == os.getgid() == 0 # Create a new network namespace for this pytest process enter_netns() with open("/proc/sys/net/ipv4/ip_forward", "wb") as fp: fp.write(b"1\n") async def peer_main(idx: int, parent_fd: int) -> None: parent = trio.socket.fromfd(parent_fd, socket.AF_UNIX, socket.SOCK_SEQPACKET) # Tell parent we've set up our netns, wait for it to confirm it's # created our veth interface await parent.send(b"ok") assert b"ok" == await parent.recv(4096) my_ip = PEER_IP[idx] router_ip = ROUTER_IP[idx] peer_ip = PEER_IP[3 - idx] for cmd in ( f"ip link set veth0 up", f"ip addr add {my_ip}/24 dev veth0", f"ip route add default via {router_ip} dev veth0", ): await trio.run_process(cmd.split(), capture_stdout=True, capture_stderr=True) peer = trio.socket.socket(socket.AF_INET, socket.SOCK_DGRAM) await peer.bind((my_ip, 0)) # Tell the parent our port and get our peer's port await parent.send(b"%d" % peer.getsockname()[1]) peer_port = int(await parent.recv(4096)) await peer.connect((peer_ip, peer_port)) # Enter the message-forwarding loop async def proxy_one_way( src: trio.socket.SocketType, dest: trio.socket.SocketType ) -> None: while src.fileno() >= 0: try: msg = await src.recv(4096) except trio.ClosedResourceError: return if not msg: dest.close() return try: await dest.send(msg) except BrokenPipeError: return async with trio.open_nursery() as nursery: nursery.start_soon(proxy_one_way, parent, peer) nursery.start_soon(proxy_one_way, peer, parent) def _default_capture_cb( target: "trio.MemorySendChannel[netfilterqueue.Packet]", packet: netfilterqueue.Packet, ) -> None: packet.retain() target.send_nowait(packet) class Harness: def __init__(self) -> None: self._received: Dict[int, trio.MemoryReceiveChannel[bytes]] = {} self._conn: Dict[int, trio.socket.SocketType] = {} self.dest_addr: Dict[int, Tuple[str, int]] = {} self.failed = False async def _run_peer(self, idx: int, *, task_status: Any) -> None: their_ip = PEER_IP[idx] my_ip = ROUTER_IP[idx] conn, child_conn = trio.socket.socketpair(socket.AF_UNIX, socket.SOCK_SEQPACKET) with conn: try: process = await trio.open_process( [sys.executable, __file__, str(idx), str(child_conn.fileno())], stdin=subprocess.DEVNULL, pass_fds=[child_conn.fileno()], preexec_fn=enter_netns, ) finally: child_conn.close() assert b"ok" == await conn.recv(4096) for cmd in ( f"ip link add veth{idx} type veth peer netns {process.pid} name veth0", f"ip link set veth{idx} up", f"ip addr add {my_ip}/24 dev veth{idx}", ): await trio.run_process(cmd.split()) try: await conn.send(b"ok") self._conn[idx] = conn task_status.started() retval = await process.wait() except BaseException: process.kill() with trio.CancelScope(shield=True): await process.wait() raise else: if retval != 0: raise RuntimeError( "peer subprocess exited with code {}".format(retval) ) finally: # On some kernels the veth device is removed when the subprocess exits # and its netns goes away. check=False to suppress that error. await trio.run_process(f"ip link delete veth{idx}".split(), check=False) async def _manage_peer(self, idx: int, *, task_status: Any) -> None: async with trio.open_nursery() as nursery: await nursery.start(self._run_peer, idx) packets_w, packets_r = trio.open_memory_channel[bytes](math.inf) self._received[idx] = packets_r task_status.started() async with packets_w: while True: msg = await self._conn[idx].recv(4096) if not msg: break await packets_w.send(msg) @asynccontextmanager async def run(self) -> AsyncIterator[None]: async with trio.open_nursery() as nursery: async with trio.open_nursery() as start_nursery: start_nursery.start_soon(nursery.start, self._manage_peer, 1) start_nursery.start_soon(nursery.start, self._manage_peer, 2) # Tell each peer about the other one's port for idx in (1, 2): self.dest_addr[idx] = ( PEER_IP[idx], int(await self._received[idx].receive()), ) await self._conn[3 - idx].send(b"%d" % self.dest_addr[idx][1]) yield self._conn[1].shutdown(socket.SHUT_WR) self._conn[2].shutdown(socket.SHUT_WR) if not self.failed: for idx in (1, 2): async for remainder in self._received[idx]: raise AssertionError( f"Peer {idx} received unexepcted packet {remainder!r}" ) def bind_queue( self, cb: Callable[[netfilterqueue.Packet], None], *, queue_num: int = -1, **options: int, ) -> Tuple[int, netfilterqueue.NetfilterQueue]: nfq = netfilterqueue.NetfilterQueue() # Use a smaller socket buffer to avoid a warning in CI options.setdefault("sock_len", 131072) if queue_num >= 0: nfq.bind(queue_num, cb, **options) else: for queue_num in range(16): try: nfq.bind(queue_num, cb, **options) break except Exception as ex: last_error = ex else: raise RuntimeError( "Couldn't bind any netfilter queue number between 0-15" ) from last_error return queue_num, nfq @asynccontextmanager async def enqueue_packets_to( self, idx: int, queue_num: int, *, forwarded: bool = True ) -> AsyncIterator[None]: if forwarded: chain = "FORWARD" else: chain = "OUTPUT" rule = f"{chain} -d {PEER_IP[idx]} -j NFQUEUE --queue-num {queue_num}" await trio.run_process(f"/sbin/iptables -A {rule}".split()) try: yield finally: await trio.run_process(f"/sbin/iptables -D {rule}".split()) @asynccontextmanager async def capture_packets_to( self, idx: int, cb: Callable[ ["trio.MemorySendChannel[netfilterqueue.Packet]", netfilterqueue.Packet], None, ] = _default_capture_cb, **options: int, ) -> AsyncIterator["trio.MemoryReceiveChannel[netfilterqueue.Packet]"]: packets_w, packets_r = trio.open_memory_channel[netfilterqueue.Packet](math.inf) queue_num, nfq = self.bind_queue(partial(cb, packets_w), **options) try: async with self.enqueue_packets_to(idx, queue_num): async with packets_w, trio.open_nursery() as nursery: @nursery.start_soon async def listen_for_packets() -> None: while True: await trio.lowlevel.wait_readable(nfq.get_fd()) nfq.run(block=False) yield packets_r nursery.cancel_scope.cancel() finally: nfq.unbind() async def expect(self, idx: int, *packets: bytes) -> None: for expected in packets: with trio.move_on_after(5) as scope: received = await self._received[idx].receive() if scope.cancelled_caught: self.failed = True raise AssertionError( f"Timeout waiting for peer {idx} to receive {expected!r}" ) if received != expected: self.failed = True raise AssertionError( f"Expected peer {idx} to receive {expected!r} but it " f"received {received!r}" ) async def send(self, idx: int, *packets: bytes) -> None: for packet in packets: await self._conn[3 - idx].send(packet) @pytest.fixture async def harness() -> AsyncIterator[Harness]: h = Harness() async with h.run(): yield h if __name__ == "__main__": trio.run(peer_main, int(sys.argv[1]), int(sys.argv[2])) python-netfilterqueue-1.1.0/tests/test_basic.py000066400000000000000000000263641437760247000217520ustar00rootroot00000000000000import gc import struct import os import pytest import signal import socket import sys import time import trio import trio.testing import weakref from netfilterqueue import NetfilterQueue, COPY_META async def test_comms_without_queue(harness): await harness.send(2, b"hello", b"world") await harness.expect(2, b"hello", b"world") await harness.send(1, b"it works?") await harness.expect(1, b"it works?") async def test_queue_dropping(harness): async def drop(packets, msg): async for packet in packets: assert "UDP packet" in str(packet) if packet.get_payload()[28:] == msg: packet.drop() else: packet.accept() async with trio.open_nursery() as nursery: async with harness.capture_packets_to(2) as p2, harness.capture_packets_to( 1 ) as p1: nursery.start_soon(drop, p2, b"two") nursery.start_soon(drop, p1, b"one") await harness.send(2, b"one", b"two", b"three") await harness.send(1, b"one", b"two", b"three") await harness.expect(2, b"one", b"three") await harness.expect(1, b"two", b"three") # Once we stop capturing, everything gets through again: await harness.send(2, b"one", b"two", b"three") await harness.send(1, b"one", b"two", b"three") await harness.expect(2, b"one", b"two", b"three") await harness.expect(1, b"one", b"two", b"three") async def test_rewrite_reorder(harness): async def munge(packets): def set_udp_payload(p, msg): data = bytearray(p.get_payload()) old_len = len(data) - 28 if len(msg) != old_len: data[2:4] = struct.pack(">H", len(msg) + 28) data[24:26] = struct.pack(">H", len(msg) + 8) # Recompute checksum too data[10:12] = b"\x00\x00" words = struct.unpack(">10H", data[:20]) cksum = sum(words) while cksum >> 16: cksum = (cksum & 0xFFFF) + (cksum >> 16) data[10:12] = struct.pack(">H", cksum ^ 0xFFFF) # Clear UDP checksum and set payload data[28:] = msg data[26:28] = b"\x00\x00" p.set_payload(bytes(data)) async for packet in packets: payload = packet.get_payload()[28:] if payload == b"one": set_udp_payload(packet, b"numero uno") assert b"numero uno" == packet.get_payload()[28:] packet.accept() elif payload == b"two": two = packet elif payload == b"three": set_udp_payload(two, b"TWO") packet.accept() two.accept() else: packet.accept() async with trio.open_nursery() as nursery: async with harness.capture_packets_to(2) as p2: nursery.start_soon(munge, p2) await harness.send(2, b"one", b"two", b"three", b"four") await harness.expect(2, b"numero uno", b"three", b"TWO", b"four") async def test_mark_repeat(harness): counter = 0 timestamps = [] def cb(chan, pkt): nonlocal counter with pytest.raises(RuntimeError, match="Packet has no payload"): pkt.get_payload() assert pkt.get_mark() == counter timestamps.append(pkt.get_timestamp()) if counter < 5: counter += 1 pkt.set_mark(counter) pkt.repeat() assert pkt.get_mark() == counter else: pkt.accept() async with harness.capture_packets_to(2, cb, mode=COPY_META): t0 = time.time() await harness.send(2, b"testing") await harness.expect(2, b"testing") t1 = time.time() assert counter == 5 # All iterations of the packet have the same timestamps assert all(t == timestamps[0] for t in timestamps[1:]) assert t0 < timestamps[0] < t1 async def test_hwaddr_and_inoutdev(harness): hwaddrs = [] inoutdevs = [] def cb(pkt): hwaddrs.append((pkt.get_hw(), pkt.hook, pkt.get_payload()[28:])) inoutdevs.append((pkt.indev, pkt.outdev)) pkt.accept() queue_num, nfq = harness.bind_queue(cb) try: async with trio.open_nursery() as nursery: @nursery.start_soon async def listen_for_packets(): while True: await trio.lowlevel.wait_readable(nfq.get_fd()) nfq.run(block=False) async with harness.enqueue_packets_to(2, queue_num, forwarded=True): await harness.send(2, b"one", b"two") await harness.expect(2, b"one", b"two") async with harness.enqueue_packets_to(2, queue_num, forwarded=False): with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: for payload in (b"three", b"four"): sock.sendto(payload, harness.dest_addr[2]) with trio.fail_after(1): while len(hwaddrs) < 4: await trio.sleep(0.1) nursery.cancel_scope.cancel() finally: nfq.unbind() # Forwarded packets capture a hwaddr, but OUTPUT don't FORWARD = 2 OUTPUT = 3 mac1 = hwaddrs[0][0] assert mac1 is not None assert hwaddrs == [ (mac1, FORWARD, b"one"), (mac1, FORWARD, b"two"), (None, OUTPUT, b"three"), (None, OUTPUT, b"four"), ] if sys.implementation.name != "pypy": # pypy doesn't appear to provide if_nametoindex() iface1 = socket.if_nametoindex("veth1") iface2 = socket.if_nametoindex("veth2") else: iface1, iface2 = inoutdevs[0] assert 0 != iface1 != iface2 != 0 assert inoutdevs == [ (iface1, iface2), (iface1, iface2), (0, iface2), (0, iface2), ] async def test_errors(harness): with pytest.warns(RuntimeWarning, match="rcvbuf limit is") as record: async with harness.capture_packets_to(2, sock_len=2 ** 30): pass assert record[0].filename.endswith("conftest.py") async with harness.capture_packets_to(2, queue_num=0): with pytest.raises(OSError, match="Failed to create queue"): async with harness.capture_packets_to(2, queue_num=0): pass _, nfq = harness.bind_queue(lambda: None, queue_num=1) with pytest.raises(RuntimeError, match="A queue is already bound"): nfq.bind(2, lambda p: None) # Test unbinding via __del__ nfq = weakref.ref(nfq) for _ in range(4): gc.collect() if nfq() is None: break else: raise RuntimeError("Couldn't trigger garbage collection of NFQ") async def test_unretained(harness): def cb(chan, pkt): # Can access payload within callback assert "UDP packet" in str(pkt) assert pkt.get_payload()[-3:] in (b"one", b"two") chan.send_nowait(pkt) # Capture packets without retaining -> can't access payload after cb returns async with harness.capture_packets_to(2, cb) as chan: await harness.send(2, b"one", b"two") accept = True async for p in chan: with pytest.raises( RuntimeError, match="Payload data is no longer available" ): p.get_payload() assert "contents unretained" in str(p) # Can still issue verdicts though if accept: p.accept() accept = False else: break with pytest.raises(RuntimeError, match="Parent queue has already been unbound"): p.drop() await harness.expect(2, b"one") async def test_cb_exception(harness): pkt = None def cb(channel, p): nonlocal pkt pkt = p raise ValueError("test") # Error raised within run(): with pytest.raises(ValueError, match="test"): async with harness.capture_packets_to(2, cb): await harness.send(2, b"boom") with trio.fail_after(1): try: await trio.sleep_forever() finally: # At this point the error has been raised (since we were # cancelled) but the queue is still open. We shouldn't # be able to access the payload, since we didn't retain(), # but verdicts should otherwise work. with pytest.raises(RuntimeError, match="Payload data is no longer"): pkt.get_payload() pkt.accept() await harness.expect(2, b"boom") with pytest.raises(RuntimeError, match="Verdict already given for this packet"): pkt.drop() @pytest.mark.skipif( sys.implementation.name == "pypy", reason="pypy does not support PyErr_CheckSignals", ) def test_signal(): nfq = NetfilterQueue() nfq.bind(1, lambda p: None, sock_len=131072) def raise_alarm(sig, frame): raise KeyboardInterrupt("brrrrrring!") old_handler = signal.signal(signal.SIGALRM, raise_alarm) old_timer = signal.setitimer(signal.ITIMER_REAL, 0.5, 0) try: with pytest.raises(KeyboardInterrupt, match="brrrrrring!") as exc_info: nfq.run() assert any("NetfilterQueue.run" in line.name for line in exc_info.traceback) finally: nfq.unbind() signal.setitimer(signal.ITIMER_REAL, *old_timer) signal.signal(signal.SIGALRM, old_handler) async def test_external_fd(harness): child_prog = """ import os, sys, unshare from netfilterqueue import NetfilterQueue unshare.unshare(unshare.CLONE_NEWNET) nfq = NetfilterQueue(sockfd=int(sys.argv[1])) def cb(pkt): pkt.accept() sys.exit(pkt.get_payload()[28:].decode("ascii")) nfq.bind(1, cb, sock_len=131072) os.write(1, b"ok\\n") try: nfq.run() finally: nfq.unbind() """ async with trio.open_nursery() as nursery: async def monitor_in_child(task_status): with trio.fail_after(5): r, w = os.pipe() # 12 is NETLINK_NETFILTER family nlsock = socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, 12) @nursery.start_soon async def wait_started(): await trio.lowlevel.wait_readable(r) assert b"ok\n" == os.read(r, 16) nlsock.close() os.close(w) os.close(r) task_status.started() result = await trio.run_process( [sys.executable, "-c", child_prog, str(nlsock.fileno())], stdout=w, capture_stderr=True, check=False, pass_fds=(nlsock.fileno(),), ) assert result.stderr == b"this is a test\n" await nursery.start(monitor_in_child) async with harness.enqueue_packets_to(2, queue_num=1): await harness.send(2, b"this is a test") await harness.expect(2, b"this is a test") with pytest.raises(OSError, match="dup2 failed"): NetfilterQueue(sockfd=1000) with pytest.raises(OSError, match="Failed to open NFQueue"): with open("/dev/null") as fp: NetfilterQueue(sockfd=fp.fileno())