py_zipkin-0.15.0/0000755000174700017470000000000013363457445013071 5ustar debiandebianpy_zipkin-0.15.0/setup.cfg0000644000174700017470000000014013363457445014705 0ustar debiandebian[wheel] universal = True [pep8] ignore = E265,E309,E501 [egg_info] tag_build = tag_date = 0 py_zipkin-0.15.0/setup.py0000644000174700017470000000254013363457355014604 0ustar debiandebian#!/usr/bin/python # -*- coding: utf-8 -*- import os from setuptools import find_packages from setuptools import setup __version__ = '0.15.0' def read(f): return open(os.path.join(os.path.dirname(__file__), f)).read().strip() setup( name='py_zipkin', version=__version__, provides=["py_zipkin"], author='Yelp, Inc.', author_email='opensource+py-zipkin@yelp.com', license='Copyright Yelp 2018', url="https://github.com/Yelp/py_zipkin", description='Library for using Zipkin in Python.', long_description='\n\n'.join((read('README.md'), read('CHANGELOG.rst'))), long_description_content_type="text/markdown", packages=find_packages(exclude=('tests*', 'testing*', 'tools*')), package_data={'': ['*.thrift']}, install_requires=[ 'six', 'thriftpy', ], extras_require={':python_version=="2.7"': ['enum34']}, classifiers=[ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Topic :: Software Development :: Libraries :: Python Modules", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", ], ) py_zipkin-0.15.0/PKG-INFO0000644000174700017470000004040413363457445014170 0ustar debiandebianMetadata-Version: 2.1 Name: py_zipkin Version: 0.15.0 Summary: Library for using Zipkin in Python. Home-page: https://github.com/Yelp/py_zipkin Author: Yelp, Inc. Author-email: opensource+py-zipkin@yelp.com License: Copyright Yelp 2018 Description: [![Build Status](https://travis-ci.org/Yelp/py_zipkin.svg?branch=master)](https://travis-ci.org/Yelp/py_zipkin) [![Coverage Status](https://img.shields.io/coveralls/Yelp/py_zipkin.svg)](https://coveralls.io/r/Yelp/py_zipkin) [![PyPi version](https://img.shields.io/pypi/v/py_zipkin.svg)](https://pypi.python.org/pypi/py_zipkin/) [![Supported Python versions](https://img.shields.io/pypi/pyversions/py_zipkin.svg)](https://pypi.python.org/pypi/py_zipkin/) py_zipkin --------- py_zipkin provides a context manager/decorator along with some utilities to facilitate the usage of Zipkin in Python applications. Install ------- ``` pip install py_zipkin ``` Usage ----- py_zipkin requires a `transport_handler` object that handles logging zipkin messages to a central logging service such as kafka or scribe. `py_zipkin.zipkin.zipkin_span` is the main tool for starting zipkin traces or logging spans inside an ongoing trace. zipkin_span can be used as a context manager or a decorator. #### Usage #1: Start a trace with a given sampling rate ```python from py_zipkin.zipkin import zipkin_span def some_function(a, b): with zipkin_span( service_name='my_service', span_name='my_span_name', transport_handler=some_handler, port=42, sample_rate=0.05, # Value between 0.0 and 100.0 ): do_stuff(a, b) ``` #### Usage #2: Trace a service call The difference between this and Usage #1 is that the zipkin_attrs are calculated separately and passed in, thus negating the need of the sample_rate param. ```python # Define a pyramid tween def tween(request): zipkin_attrs = some_zipkin_attr_creator(request) with zipkin_span( service_name='my_service', span_name='my_span_name', zipkin_attrs=zipkin_attrs, transport_handler=some_handler, port=22, ) as zipkin_context: response = handler(request) zipkin_context.update_binary_annotations( some_binary_annotations) return response ``` #### Usage #3: Log a span inside an ongoing trace This can be also be used inside itself to produce continuously nested spans. ```python @zipkin_span(service_name='my_service', span_name='some_function') def some_function(a, b): return do_stuff(a, b) ``` #### Other utilities `zipkin_span.update_binary_annotations()` can be used inside a zipkin trace to add to the existing set of binary annotations. ```python def some_function(a, b): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, ) as zipkin_context: result = do_stuff(a, b) zipkin_context.update_binary_annotations({'result': result}) ``` `zipkin_span.add_sa_binary_annotation()` can be used to add a binary annotation to the current span with the key 'sa'. This function allows the user to specify the destination address of the service being called (useful if the destination doesn't support zipkin). See http://zipkin.io/pages/data_model.html for more information on the 'sa' binary annotation. > NOTE: the V2 span format only support 1 "sa" endpoint (represented by remoteEndpoint) > so `add_sa_binary_annotation` now raises `ValueError` if you try to set multiple "sa" > annotations for the same span. ```python def some_function(): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, ) as zipkin_context: make_call_to_non_instrumented_service() zipkin_context.add_sa_binary_annotation( port=123, service_name='non_instrumented_service', host='12.34.56.78', ) ``` `create_http_headers_for_new_span()` creates a set of HTTP headers that can be forwarded in a request to another service. ```python headers = {} headers.update(create_http_headers_for_new_span()) http_client.get( path='some_url', headers=headers, ) ``` Transport --------- py_zipkin (for the moment) thrift-encodes spans. The actual transport layer is pluggable, though. The recommended way to implement a new transport handler is to subclass `py_zipkin.transport.BaseTransportHandler` and implement the `send` and `get_max_payload_bytes` methods. `send` receives an already encoded thrift list as argument. `get_max_payload_bytes` should return the maximum payload size supported by your transport, or `None` if you can send arbitrarily big messages. The simplest way to get spans to the collector is via HTTP POST. Here's an example of a simple HTTP transport using the `requests` library. This assumes your Zipkin collector is running at localhost:9411. > NOTE: older versions of py_zipkin suggested implementing the transport handler > as a function with a single argument. That's still supported and should work > with the current py_zipkin version, but it's deprecated. ```python import requests from py_zipkin.transport import BaseTransportHandler class HttpTransport(BaseTransportHandler): def get_max_payload_bytes(self): return None def send(self, encoded_span): # The collector expects a thrift-encoded list of spans. requests.post( 'http://localhost:9411/api/v1/spans', data=encoded_span, headers={'Content-Type': 'application/x-thrift'}, ) ``` If you have the ability to send spans over Kafka (more like what you might do in production), you'd do something like the following, using the [kafka-python](https://pypi.python.org/pypi/kafka-python) package: ```python from kafka import SimpleProducer, KafkaClient from py_zipkin.transport import BaseTransportHandler class KafkaTransport(BaseTransportHandler): def get_max_payload_bytes(self): # By default Kafka rejects messages bigger than 1000012 bytes. return 1000012 def send(self, message): kafka_client = KafkaClient('{}:{}'.format('localhost', 9092)) producer = SimpleProducer(kafka_client) producer.send_messages('kafka_topic_name', message) ``` Using in multithreading evironments ----------------------------------- If you want to use py_zipkin in a cooperative multithreading environment, e.g. asyncio, you need to explicitly pass an instance of `py_zipkin.storage.Stack` as parameter `context_stack` for `zipkin_span` and `create_http_headers_for_new_span`. By default, py_zipkin uses a thread local storage for the attributes, which is defined in `py_zipkin.storage.ThreadLocalStack`. Additionally, you'll also need to explicitly pass an instance of `py_zipkin.storage.SpanStorage` as parameter `span_storage` to `zipkin_span`. ```python from py_zipkin.zipkin import zipkin_span from py_zipkin.storage import Stack from py_zipkin.storage import SpanStorage def my_function(): context_stack = Stack() span_storage = SpanStorage() await my_function(context_stack, span_storage) async def my_function(context_stack, span_storage): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, context_stack=context_stack, span_storage=span_storage, ): result = do_stuff(a, b) ``` Firehose mode [EXPERIMENTAL] ---------------------------- "Firehose mode" records 100% of the spans, regardless of sampling rate. This is useful if you want to treat these spans differently, e.g. send them to a different backend that has limited retention. It works in tandem with normal operation, however there may be additional overhead. In order to use this, you add a `firehose_handler` just like you add a `transport_handler`. This feature should be considered experimental and may be removed at any time without warning. If you do use this, be sure to send asynchronously to avoid excess overhead for every request. License ------- Copyright (c) 2018, Yelp, Inc. All Rights reserved. Apache v2 0.15.0 (2018-10-22) ------------------- - Added support for V2 JSON encoding. - Fixed TransportHandler bug that was affecting also V1 JSON. 0.14.1 (2018-10-09) ------------------- - Fixed memory leak introduced in 0.13.0. 0.14.0 (2018-10-01) ------------------- - Support JSON encoding for V1 spans. - Allow overriding the span_name after creation. 0.13.0 (2018-06-25) ------------------- - Removed deprecated `zipkin_logger.debug()` interface. - `py_zipkin.stack` was renamed as `py_zipkin.storage`. If you were importing this module, you'll need to update your code. 0.12.0 (2018-05-29) ------------------- - Support max payload size for transport handlers. - Transport handlers should now be implemented as classes extending py_zipkin.transport.BaseTransportHandler. 0.11.2 (2018-05-23) ------------------- - Don't overwrite passed in annotations 0.11.1 (2018-05-23) ------------------- - Add binary annotations to the span even if the request is not being sampled. This fixes binary annotations for firehose spans. 0.11.0 (2018-02-08) ------------------- - Add support for "firehose mode", which logs 100% of the spans regardless of sample rate. 0.10.1 (2018-02-05) ------------------- - context_stack will now default to `ThreadLocalStack()` if passed as `None` 0.10.0 (2018-02-05) ------------------- - Add support for using explicit in-process context storage instead of using thread_local. This allows you to use py_zipkin in cooperative multitasking environments e.g. asyncio - `py_zipkin.thread_local` is now deprecated. Instead use `py_zipkin.stack.ThreadLocalStack()` - TraceId and SpanId generation performance improvements. - 128-bit TraceIds now start with an epoch timestamp to support easy interop with AWS X-Ray 0.9.0 (2017-07-31) ------------------ - Add batch span sending. Note that spans are now sent in lists. 0.8.3 (2017-07-10) ------------------ - Be defensive about having logging handlers configured to avoid throwing NullHandler attribute errors 0.8.2 (2017-06-30) ------------------ - Don't log ss and sr annotations when in a client span context - Add error binary annotation if an exception occurs 0.8.1 (2017-06-16) ------------------ - Fixed server send timing to more accurately reflect when server send actually occurs. - Replaced logging_start annotation with logging_end 0.8.0 (2017-06-01) ------------------ - Added 128-bit trace id support - Added ability to explicitly specify host for a span - Added exception handling if host can't be determined automatically - SERVER_ADDR ('sa') binary annotations can be added to spans - py36 support 0.7.1 (2017-05-01) ------------------ - Fixed a bug where `update_binary_annotations` would fail for a child span in a trace that is not being sampled 0.7.0 (2017-03-06) ------------------ - Simplify `update_binary_annotations` for both root and non-root spans 0.6.0 (2017-02-03) ------------------ - Added support for forcing `zipkin_span` to report timestamp/duration. Changes API of `zipkin_span`, but defaults back to existing behavior. 0.5.0 (2017-02-01) ------------------ - Properly set timestamp/duration on server and local spans - Updated thrift spec to include these new fields - The `zipkin_span` entrypoint should be backwards compatible 0.4.4 (2016-11-29) ------------------ - Add optional annotation for when Zipkin logging starts 0.4.3 (2016-11-04) ------------------ - Fix bug in zipkin_span decorator 0.4.2 (2016-11-01) ------------------ - Be defensive about transport_handler when logging spans. 0.4.1 (2016-10-24) ------------------ - Add ability to override span_id when creating new ZipkinAttrs. 0.4.0 (2016-10-20) ------------------ - Added `start` and `stop` functions as friendlier versions of the __enter__ and __exit__ functions. 0.3.1 (2016-09-30) ------------------ - Adds new param to thrift.create_endpoint allowing creation of thrift Endpoint objects on a proxy machine representing another host. 0.2.1 (2016-09-30) ------------------ - Officially "release" v0.2.0. Accidentally pushed a v0.2.0 without the proper version bump, so v0.2.1 is the new real version. Please use this instead of v0.2.0. 0.2.0 (2016-09-30) ------------------ - Fix problem where if zipkin_attrs and sample_rate were passed, but zipkin_attrs.is_sampled=True, new zipkin_attrs were being generated. 0.1.2 (2016-09-29) ------------------ - Fix sampling algorithm that always sampled for rates > 50% 0.1.1 (2016-07-05) ------------------ - First py_zipkin version with context manager/decorator functionality. Platform: UNKNOWN Classifier: Development Status :: 3 - Alpha Classifier: Intended Audience :: Developers Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Provides: py_zipkin Description-Content-Type: text/markdown py_zipkin-0.15.0/py_zipkin/0000755000174700017470000000000013363457445015105 5ustar debiandebianpy_zipkin-0.15.0/py_zipkin/encoding/0000755000174700017470000000000013363457445016673 5ustar debiandebianpy_zipkin-0.15.0/py_zipkin/encoding/_types.py0000644000174700017470000000050113363457355020544 0ustar debiandebian# -*- coding: utf-8 -*- from enum import Enum class Encoding(Enum): """Supported output encodings.""" V1_THRIFT = 'V1_THRIFT' V1_JSON = 'V1_JSON' V2_JSON = 'V2_JSON' V2_PROTOBUF = 'V2_PROTOBUF' class Kind(Enum): """Type of Span.""" CLIENT = 'CLIENT' SERVER = 'SERVER' LOCAL = None py_zipkin-0.15.0/py_zipkin/encoding/_helpers.py0000644000174700017470000001537413363457355021060 0ustar debiandebian# -*- coding: utf-8 -*- import socket from collections import namedtuple from collections import OrderedDict from py_zipkin.encoding._types import Kind from py_zipkin.exception import ZipkinError Endpoint = namedtuple( 'Endpoint', ['service_name', 'ipv4', 'ipv6', 'port'], ) _V1Span = namedtuple( 'V1Span', ['trace_id', 'name', 'parent_id', 'id', 'timestamp', 'duration', 'endpoint', 'annotations', 'binary_annotations', 'sa_endpoint'], ) _V2Span = namedtuple( 'V2Span', ['trace_id', 'name', 'parent_id', 'id', 'kind', 'timestamp', 'duration', 'debug', 'shared', 'local_endpoint', 'remote_endpoint', 'annotations', 'tags'], ) _DROP_ANNOTATIONS_BY_KIND = { Kind.CLIENT: {'ss', 'sr'}, Kind.SERVER: {'cs', 'cr'}, } class SpanBuilder(object): """Internal Span representation. It can generate both v1 and v2 spans. It doesn't exactly map to either V1 or V2, since an intermediate format makes it easier to convert to either format. """ def __init__( self, trace_id, name, parent_id, span_id, timestamp, duration, annotations, tags, kind, local_endpoint=None, service_name=None, sa_endpoint=None, report_timestamp=True, ): """Creates a new SpanBuilder. :param trace_id: Trace id. :type trace_id: str :param name: Name of the span. :type name: str :param parent_id: Parent span id. :type parent_id: str :param span_id: Span id. :type span_id: str :param timestamp: start timestamp in seconds. :type timestamp: float :param duration: span duration in seconds. :type duration: float :param annotations: Optional dict of str -> timestamp annotations. :type annotations: dict :param tags: Optional dict of str -> str span tags. :type tags: dict :param kind: Span type (client, server, local, etc...) :type kind: Kind :param local_endpoint: The host that recorded this span. :type local_endpoint: Endpoint :param service_name: The name of the called service :type service_name: str :param sa_endpoint: Remote server in client spans. :type sa_endpoint: Endpoint :param report_timestamp: Whether the span should report timestamp and duration. :type report_timestamp: bool """ self.trace_id = trace_id self.name = name self.parent_id = parent_id self.span_id = span_id self.kind = kind self.timestamp = timestamp self.duration = duration self.annotations = annotations self.tags = tags self.local_endpoint = local_endpoint self.service_name = service_name self.sa_endpoint = sa_endpoint self.report_timestamp = report_timestamp if not isinstance(kind, Kind): raise ZipkinError( 'Invalid kind value {}. Must be of type Kind.'.format(kind)) def build_v1_span(self): """Builds and returns a V1 Span. :return: newly generated _V1Span :rtype: _V1Span """ # We are simulating a full two-part span locally, so set cs=sr and ss=cr full_annotations = OrderedDict([ ('cs', self.timestamp), ('sr', self.timestamp), ('ss', self.timestamp + self.duration), ('cr', self.timestamp + self.duration), ]) if self.kind != Kind.LOCAL: # If kind is not LOCAL, then we only want client or # server side annotations. for ann in _DROP_ANNOTATIONS_BY_KIND[self.kind]: del full_annotations[ann] # Add user-defined annotations. We write them in full_annotations # instead of the opposite so that user annotations will override # any automatically generated annotation. full_annotations.update(self.annotations) return _V1Span( trace_id=self.trace_id, name=self.name, parent_id=self.parent_id, id=self.span_id, timestamp=self.timestamp if self.report_timestamp else None, duration=self.duration if self.report_timestamp else None, endpoint=self.local_endpoint, annotations=full_annotations, binary_annotations=self.tags, sa_endpoint=self.sa_endpoint, ) def build_v2_span(self): """Builds and returns a V2 Span. :return: newly generated _V2Span :rtype: _V2Span """ remote_endpoint = None if self.sa_endpoint: remote_endpoint = self.sa_endpoint return _V2Span( trace_id=self.trace_id, name=self.name, parent_id=self.parent_id, id=self.span_id, kind=self.kind, timestamp=self.timestamp, duration=self.duration, debug=False, shared=self.report_timestamp is False, local_endpoint=self.local_endpoint, remote_endpoint=remote_endpoint, annotations=self.annotations, tags=self.tags, ) def create_endpoint(port=0, service_name='unknown', host=None): """Creates a new Endpoint object. :param port: TCP/UDP port. Defaults to 0. :type port: int :param service_name: service name as a str. Defaults to 'unknown'. :type service_name: str :param host: ipv4 or ipv6 address of the host. Defaults to the current host ip. :type host: str :returns: zipkin Endpoint object """ if host is None: try: host = socket.gethostbyname(socket.gethostname()) except socket.gaierror: host = '127.0.0.1' ipv4 = None ipv6 = None # Check ipv4 or ipv6. try: socket.inet_pton(socket.AF_INET, host) ipv4 = host except socket.error: # If it's not an ipv4 address, maybe it's ipv6. try: socket.inet_pton(socket.AF_INET6, host) ipv6 = host except socket.error: # If it's neither ipv4 or ipv6, leave both ip addresses unset. pass return Endpoint( ipv4=ipv4, ipv6=ipv6, port=port, service_name=service_name, ) def copy_endpoint_with_new_service_name(endpoint, new_service_name): """Creates a copy of a given endpoint with a new service name. :param endpoint: existing Endpoint object :type endpoint: Endpoint :param new_service_name: new service name :type new_service_name: str :returns: zipkin new Endpoint object """ return Endpoint( service_name=new_service_name, ipv4=endpoint.ipv4, ipv6=endpoint.ipv6, port=endpoint.port, ) py_zipkin-0.15.0/py_zipkin/encoding/__init__.py0000644000174700017470000000000013363457355020772 0ustar debiandebianpy_zipkin-0.15.0/py_zipkin/encoding/_encoders.py0000644000174700017470000002147113363457355021213 0ustar debiandebian# -*- coding: utf-8 -*- import json from py_zipkin import thrift from py_zipkin.encoding._types import Encoding from py_zipkin.exception import ZipkinError def get_encoder(encoding): """Creates encoder object for the given encoding. :param encoding: desired output encoding protocol. :type encoding: Encoding :return: corresponding IEncoder object :rtype: IEncoder """ if encoding == Encoding.V1_THRIFT: return _V1ThriftEncoder() if encoding == Encoding.V1_JSON: return _V1JSONEncoder() if encoding == Encoding.V2_JSON: return _V2JSONEncoder() raise ZipkinError('Unknown encoding: {}'.format(encoding)) class IEncoder(object): """Encoder interface.""" def fits(self, current_count, current_size, max_size, new_span): """Returns whether the new span will fit in the list. :param current_count: number of spans already in the list. :type current_count: int :param current_size: sum of the sizes of all the spans already in the list. :type current_size: int :param max_size: max supported transport payload size. :type max_size: int :param new_span: encoded span object that we want to add the the list. :type new_span: str or bytes :return: True if the new span can be added to the list, False otherwise. :rtype: bool """ raise NotImplementedError() def encode_span(self, span_builder): """Encodes a single span. :param span_builder: span_builder object representing the span. :type span_builder: SpanBuilder :return: encoded span. :rtype: str or bytes """ raise NotImplementedError() def encode_queue(self, queue): """Encodes a list of pre-encoded spans. :param queue: list of encoded spans. :type queue: list :return: encoded list, type depends on the encoding. :rtype: str or bytes """ raise NotImplementedError() class _V1ThriftEncoder(IEncoder): """Thrift encoder for V1 spans.""" def fits(self, current_count, current_size, max_size, new_span): """Checks if the new span fits in the max payload size. Thrift lists have a fixed-size header and no delimiters between elements so it's easy to compute the list size. """ return thrift.LIST_HEADER_SIZE + current_size + len(new_span) <= max_size def encode_span(self, span_builder): """Encodes the current span to thrift.""" span = span_builder.build_v1_span() thrift_endpoint = thrift.create_endpoint( span.endpoint.port, span.endpoint.service_name, span.endpoint.ipv4, span.endpoint.ipv6, ) thrift_annotations = thrift.annotation_list_builder( span.annotations, thrift_endpoint, ) thrift_binary_annotations = thrift.binary_annotation_list_builder( span.binary_annotations, thrift_endpoint, ) # Add sa binary annotation if span.sa_endpoint is not None: thrift_sa_endpoint = thrift.create_endpoint( span.sa_endpoint.port, span.sa_endpoint.service_name, span.sa_endpoint.ipv4, span.sa_endpoint.ipv6, ) thrift_binary_annotations.append(thrift.create_binary_annotation( key=thrift.zipkin_core.SERVER_ADDR, value=thrift.SERVER_ADDR_VAL, annotation_type=thrift.zipkin_core.AnnotationType.BOOL, host=thrift_sa_endpoint, )) thrift_span = thrift.create_span( span.id, span.parent_id, span.trace_id, span.name, thrift_annotations, thrift_binary_annotations, span.timestamp, span.duration, ) encoded_span = thrift.span_to_bytes(thrift_span) return encoded_span def encode_queue(self, queue): """Converts the queue to a thrift list""" return thrift.encode_bytes_list(queue) class _BaseJSONEncoder(IEncoder): """ V1 and V2 JSON encoders need many common helper functions """ def fits(self, current_count, current_size, max_size, new_span): """Checks if the new span fits in the max payload size. Json lists only have a 2 bytes overhead from '[]' plus 1 byte from ',' between elements """ return 2 + current_count + current_size + len(new_span) <= max_size def _create_json_endpoint(self, endpoint, is_v1): """Converts an Endpoint to a JSON endpoint dict. :param endpoint: endpoint object to convert. :type endpoint: Endpoint :param is_v1: whether we're serializing a v1 span. This is needed since in v1 some fields default to an empty string rather than being dropped if they're not set. :type is_v1: bool :return: dict representing a JSON endpoint. :rtype: dict """ json_endpoint = {} if endpoint.service_name: json_endpoint['serviceName'] = endpoint.service_name elif is_v1: # serviceName is mandatory in v1 json_endpoint['serviceName'] = "" if endpoint.port and endpoint.port != 0: json_endpoint['port'] = endpoint.port if endpoint.ipv4 is not None: json_endpoint['ipv4'] = endpoint.ipv4 if endpoint.ipv6 is not None: json_endpoint['ipv6'] = endpoint.ipv6 return json_endpoint def encode_queue(self, queue): """Concatenates the list to a JSON list""" return '[' + ','.join(queue) + ']' class _V1JSONEncoder(_BaseJSONEncoder): """JSON encoder for V1 spans.""" def encode_span(self, span_builder): """Encodes a single span to JSON.""" span = span_builder.build_v1_span() json_span = { 'traceId': span.trace_id, 'name': span.name, 'id': span.id, 'annotations': [], 'binaryAnnotations': [], } if span.parent_id: json_span['parentId'] = span.parent_id if span.timestamp: json_span['timestamp'] = int(span.timestamp * 1000000) if span.duration: json_span['duration'] = int(span.duration * 1000000) v1_endpoint = self._create_json_endpoint(span.endpoint, True) for key, timestamp in span.annotations.items(): json_span['annotations'].append({ 'endpoint': v1_endpoint, 'timestamp': int(timestamp * 1000000), 'value': key, }) for key, value in span.binary_annotations.items(): json_span['binaryAnnotations'].append({ 'key': key, 'value': value, 'endpoint': v1_endpoint, }) # Add sa binary annotations if span.sa_endpoint is not None: json_sa_endpoint = self._create_json_endpoint(span.sa_endpoint, True) json_span['binaryAnnotations'].append({ 'key': 'sa', 'value': '1', 'endpoint': json_sa_endpoint, }) encoded_span = json.dumps(json_span) return encoded_span class _V2JSONEncoder(_BaseJSONEncoder): """JSON encoder for V2 spans.""" def encode_span(self, span_builder): """Encodes a single span to JSON.""" span = span_builder.build_v2_span() json_span = { 'traceId': span.trace_id, 'id': span.id, } if span.name: json_span['name'] = span.name if span.parent_id: json_span['parentId'] = span.parent_id if span.timestamp: json_span['timestamp'] = int(span.timestamp * 1000000) if span.duration: json_span['duration'] = int(span.duration * 1000000) if span.shared is True: json_span['shared'] = True if span.kind and span.kind.value is not None: json_span['kind'] = span.kind.value if span.local_endpoint: json_span['localEndpoint'] = self._create_json_endpoint( span.local_endpoint, False, ) if span.remote_endpoint: json_span['remoteEndpoint'] = self._create_json_endpoint( span.remote_endpoint, False, ) if span.tags and len(span.tags) > 0: json_span['tags'] = span.tags if span.annotations: json_span['annotations'] = [ { 'timestamp': int(timestamp * 1000000), 'value': key, } for key, timestamp in span.annotations.items() ] encoded_span = json.dumps(json_span) return encoded_span py_zipkin-0.15.0/py_zipkin/zipkin.py0000644000174700017470000006512013363457355016767 0ustar debiandebian# -*- coding: utf-8 -*- import functools import random import time import warnings from collections import namedtuple from py_zipkin import Encoding from py_zipkin import Kind from py_zipkin import storage from py_zipkin.encoding._helpers import create_endpoint from py_zipkin.encoding._helpers import SpanBuilder from py_zipkin.exception import ZipkinError from py_zipkin.logging_helper import ZipkinLoggingContext from py_zipkin.storage import ThreadLocalStack from py_zipkin.util import generate_random_128bit_string from py_zipkin.util import generate_random_64bit_string """ Holds the basic attributes needed to log a zipkin trace :param trace_id: Unique trace id :param span_id: Span Id of the current request span :param parent_span_id: Parent span Id of the current request span :param flags: stores flags header. Currently unused :param is_sampled: pre-computed boolean whether the trace should be logged """ ZipkinAttrs = namedtuple( 'ZipkinAttrs', ['trace_id', 'span_id', 'parent_span_id', 'flags', 'is_sampled'], ) ERROR_KEY = 'error' class zipkin_span(object): """Context manager/decorator for all of your zipkin tracing needs. Usage #1: Start a trace with a given sampling rate This begins the zipkin trace and also records the root span. The required params are service_name, transport_handler, and sample_rate. # Start a trace with do_stuff() as the root span def some_batch_job(a, b): with zipkin_span( service_name='my_service', span_name='my_span_name', transport_handler=some_handler, port=22, sample_rate=0.05, ): do_stuff() Usage #2: Trace a service call. The typical use case is instrumenting a framework like Pyramid or Django. Only ss and sr times are recorded for the root span. Required params are service_name, zipkin_attrs, transport_handler, and port. # Used in a pyramid tween def tween(request): zipkin_attrs = some_zipkin_attr_creator(request) with zipkin_span( service_name='my_service,' span_name='my_span_name', zipkin_attrs=zipkin_attrs, transport_handler=some_handler, port=22, ) as zipkin_context: response = handler(request) zipkin_context.update_binary_annotations( some_binary_annotations) return response Usage #3: Log a span within the context of a zipkin trace If you're already in a zipkin trace, you can use this to log a span inside. The only required param is service_name. If you're not in a zipkin trace, this won't do anything. # As a decorator @zipkin_span(service_name='my_service', span_name='my_function') def my_function(): do_stuff() # As a context manager def my_function(): with zipkin_span(service_name='my_service', span_name='do_stuff'): do_stuff() """ def __init__( self, service_name, span_name='span', zipkin_attrs=None, transport_handler=None, max_span_batch_size=None, annotations=None, binary_annotations=None, port=0, sample_rate=None, include=None, add_logging_annotation=False, report_root_timestamp=False, use_128bit_trace_id=False, host=None, context_stack=None, span_storage=None, firehose_handler=None, kind=None, timestamp=None, duration=None, encoding=Encoding.V1_THRIFT, ): """Logs a zipkin span. If this is the root span, then a zipkin trace is started as well. :param service_name: The name of the called service :type service_name: string :param span_name: Optional name of span, defaults to 'span' :type span_name: string :param zipkin_attrs: Optional set of zipkin attributes to be used :type zipkin_attrs: ZipkinAttrs :param transport_handler: Callback function that takes a message parameter and handles logging it :type transport_handler: BaseTransportHandler :param max_span_batch_size: Spans in a trace are sent in batches, max_span_batch_size defines max size of one batch :type max_span_batch_size: int :param annotations: Optional dict of str -> timestamp annotations :type annotations: dict of str -> int :param binary_annotations: Optional dict of str -> str span attrs :type binary_annotations: dict of str -> str :param port: The port number of the service. Defaults to 0. :type port: int :param sample_rate: Rate at which to sample; 0.0 - 100.0. If passed-in zipkin_attrs have is_sampled=False and the sample_rate param is > 0, a new span will be generated at this rate. This means that if you propagate sampling decisions to downstream services, but still have sample_rate > 0 in those services, the actual rate of generated spans for those services will be > sampling_rate. :type sample_rate: float :param include: which annotations to include can be one of {'client', 'server'} corresponding to ('cs', 'cr') and ('ss', 'sr') respectively. DEPRECATED: use kind instead. `include` will be removed in 1.0. :type include: iterable :param add_logging_annotation: Whether to add a 'logging_end' annotation when py_zipkin finishes logging spans :type add_logging_annotation: boolean :param report_root_timestamp: Whether the span should report timestamp and duration. Only applies to "root" spans in this local context, so spans created inside other span contexts will always log timestamp/duration. Note that this is only an override for spans that have zipkin_attrs passed in. Spans that make their own sampling decisions (i.e. are the root spans of entire traces) will always report timestamp/duration. :type report_root_timestamp: boolean :param use_128bit_trace_id: If true, generate 128-bit trace_ids. :type use_128bit_trace_id: boolean :param host: Contains the ipv4 or ipv6 value of the host. The ip value isn't automatically determined in a docker environment. :type host: string :param context_stack: explicit context stack for storing zipkin attributes :type context_stack: object :param span_storage: explicit Span storage for storing zipkin spans before they're emitted. :type span_storage: py_zipkin.storage.SpanStorage :param firehose_handler: [EXPERIMENTAL] Similar to transport_handler, except that it will receive 100% of the spans regardless of trace sampling rate. :type firehose_handler: BaseTransportHandler :param kind: Span type (client, server, local, etc...). :type kind: Kind :param timestamp: Timestamp in seconds, defaults to `time.time()`. Set this if you want to use a custom timestamp. :type timestamp: float :param duration: Duration in seconds, defaults to the time spent in the context. Set this if you want to use a custom duration. :type duration: float :param encoding: Output encoding format, defaults to V1 thrift spans. :type encoding: Encoding """ self.service_name = service_name self.span_name = span_name self.zipkin_attrs = zipkin_attrs self.transport_handler = transport_handler self.max_span_batch_size = max_span_batch_size self.annotations = annotations or {} self.binary_annotations = binary_annotations or {} self.port = port self.sample_rate = sample_rate self.add_logging_annotation = add_logging_annotation self.report_root_timestamp_override = report_root_timestamp self.use_128bit_trace_id = use_128bit_trace_id self.host = host self._context_stack = context_stack or ThreadLocalStack() if span_storage is not None: self._span_storage = span_storage else: self._span_storage = storage.default_span_storage() self.firehose_handler = firehose_handler self.kind = self._generate_kind(kind, include) self.timestamp = timestamp self.duration = duration self.encoding = encoding self._is_local_root_span = False self.logging_context = None self.do_pop_attrs = False # Spans that log a 'cs' timestamp can additionally record a # 'sa' binary annotation that shows where the request is going. self.sa_endpoint = None # It used to be possible to override timestamp and duration by passing # in the cs/cr or sr/ss annotations. We want to keep backward compatibility # for now, so this logic overrides self.timestamp and self.duration in the # same way. # This doesn't fit well with v2 spans since those annotations are gone, so # we also log a deprecation warning. if 'sr' in self.annotations and 'ss' in self.annotations: self.duration = self.annotations['ss'] - self.annotations['sr'] self.timestamp = self.annotations['sr'] warnings.warn( "Manually setting 'sr'/'ss' annotations is deprecated. Please " "use the timestamp and duration parameters.", DeprecationWarning, ) if 'cr' in self.annotations and 'cs' in self.annotations: self.duration = self.annotations['cr'] - self.annotations['cs'] self.timestamp = self.annotations['cs'] warnings.warn( "Manually setting 'cr'/'cs' annotations is deprecated. Please " "use the timestamp and duration parameters.", DeprecationWarning, ) # Root spans have transport_handler and at least one of zipkin_attrs # or sample_rate. if self.zipkin_attrs or self.sample_rate is not None: # transport_handler is mandatory for root spans if self.transport_handler is None: raise ZipkinError( 'Root spans require a transport handler to be given') self._is_local_root_span = True # If firehose_handler than this is a local root span. if self.firehose_handler: self._is_local_root_span = True if self.sample_rate is not None and not (0.0 <= self.sample_rate <= 100.0): raise ZipkinError('Sample rate must be between 0.0 and 100.0') if not isinstance(self._span_storage, storage.SpanStorage): raise ZipkinError('span_storage should be an instance ' 'of py_zipkin.storage.SpanStorage') def __call__(self, f): @functools.wraps(f) def decorated(*args, **kwargs): with zipkin_span( service_name=self.service_name, span_name=self.span_name, zipkin_attrs=self.zipkin_attrs, transport_handler=self.transport_handler, annotations=self.annotations, binary_annotations=self.binary_annotations, port=self.port, sample_rate=self.sample_rate, host=self.host, context_stack=self._context_stack, span_storage=self._span_storage, firehose_handler=self.firehose_handler, kind=self.kind, timestamp=self.timestamp, duration=self.duration, encoding=self.encoding, ): return f(*args, **kwargs) return decorated def __enter__(self): return self.start() def _generate_kind(self, kind, include): # If `kind` is not set, then we generate it from `include`. # This code maintains backward compatibility with old versions of py_zipkin # which used include rather than kind to identify client / server spans. if kind: return kind else: if include: # If `include` contains only one of `client` or `server` # than it's a client or server span respectively. # If neither or both are present, then it's a local span # which is represented by kind = None. warnings.warn( 'The include argument is deprecated. Please use kind.', DeprecationWarning, ) if 'client' in include and 'server' not in include: return Kind.CLIENT elif 'client' not in include and 'server' in include: return Kind.SERVER else: return Kind.LOCAL # If both kind and include are unset, then it's a local span. return Kind.LOCAL def start(self): """Enter the new span context. All annotations logged inside this context will be attributed to this span. All new spans generated inside this context will have this span as their parent. In the unsampled case, this context still generates new span IDs and pushes them onto the threadlocal stack, so downstream services calls made will pass the correct headers. However, the logging handler is never attached in the unsampled case, so the spans are never logged. """ self.do_pop_attrs = False report_root_timestamp = False # This check is technically not necessary since only root spans will have # sample_rate, zipkin_attrs or a transport set. But it helps making the # code clearer by separating the logic for a root span from the one for a # child span. if self._is_local_root_span: # If sample_rate is set, we need to (re)generate a trace context. # If zipkin_attrs (trace context) were passed in as argument there are # 2 possibilities: # is_sampled = False --> we keep the same trace_id but re-roll the dice # for is_sampled. # is_sampled = True --> we don't want to stop sampling halfway through # a sampled trace, so we do nothing. # If no zipkin_attrs were passed in, we generate new ones and start a # new trace. if self.sample_rate is not None: # If this trace is not sampled, we re-roll the dice. if self.zipkin_attrs and not self.zipkin_attrs.is_sampled: # This will be the root span of the trace, so we should # set timestamp and duration. report_root_timestamp = True self.zipkin_attrs = create_attrs_for_span( sample_rate=self.sample_rate, trace_id=self.zipkin_attrs.trace_id, use_128bit_trace_id=self.use_128bit_trace_id, ) # If zipkin_attrs was not passed in, we simply generate new # zipkin_attrs to start a new trace. elif not self.zipkin_attrs: # This will be the root span of the trace, so we should # set timestamp and duration. report_root_timestamp = True self.zipkin_attrs = create_attrs_for_span( sample_rate=self.sample_rate, use_128bit_trace_id=self.use_128bit_trace_id, ) if self.firehose_handler and not self.zipkin_attrs: # If it has gotten here, the only thing that is # causing a trace is the firehose. So we force a trace # with sample rate of 0 report_root_timestamp = True self.zipkin_attrs = create_attrs_for_span( sample_rate=0.0, use_128bit_trace_id=self.use_128bit_trace_id, ) else: # If zipkin_attrs was not passed in, we check if there's already a # trace context in _context_stack. if not self.zipkin_attrs: existing_zipkin_attrs = self._context_stack.get() # If there's an existing context, let's create new zipkin_attrs # with that context as parent. if existing_zipkin_attrs: self.zipkin_attrs = ZipkinAttrs( trace_id=existing_zipkin_attrs.trace_id, span_id=generate_random_64bit_string(), parent_span_id=existing_zipkin_attrs.span_id, flags=existing_zipkin_attrs.flags, is_sampled=existing_zipkin_attrs.is_sampled, ) # If zipkin_attrs are not set up by now, that means this span is not # configured to perform logging itself, and it's not in an existing # Zipkin trace. That means there's nothing else to do and it can exit # early. if not self.zipkin_attrs: return self self._context_stack.push(self.zipkin_attrs) self.do_pop_attrs = True self.start_timestamp = time.time() if self._is_local_root_span: # Don't set up any logging if we're not sampling if not self.zipkin_attrs.is_sampled and not self.firehose_handler: return self endpoint = create_endpoint(self.port, self.service_name, self.host) self.logging_context = ZipkinLoggingContext( self.zipkin_attrs, endpoint, self.span_name, self.transport_handler, report_root_timestamp or self.report_root_timestamp_override, self._span_storage, self.service_name, binary_annotations=self.binary_annotations, add_logging_annotation=self.add_logging_annotation, client_context=self.kind == Kind.CLIENT, max_span_batch_size=self.max_span_batch_size, firehose_handler=self.firehose_handler, encoding=self.encoding, ) self.logging_context.start() self._span_storage.set_transport_configured(configured=True) return self def __exit__(self, _exc_type, _exc_value, _exc_traceback): self.stop(_exc_type, _exc_value, _exc_traceback) def stop(self, _exc_type=None, _exc_value=None, _exc_traceback=None): """Exit the span context. Zipkin attrs are pushed onto the threadlocal stack regardless of sampling, so they always need to be popped off. The actual logging of spans depends on sampling and that the logging was correctly set up. """ if self.do_pop_attrs: self._context_stack.pop() # If no transport is configured, there's no reason to create a new Span. # This also helps avoiding memory leaks since without a transport nothing # would pull spans out of _span_storage. if not self._span_storage.is_transport_configured(): return # Add the error annotation if an exception occurred if any((_exc_type, _exc_value, _exc_traceback)): error_msg = u'{0}: {1}'.format(_exc_type.__name__, _exc_value) self.update_binary_annotations({ ERROR_KEY: error_msg, }) # Logging context is only initialized for "root" spans of the local # process (i.e. this zipkin_span not inside of any other local # zipkin_spans) if self.logging_context: self.logging_context.stop() self.logging_context = None self._span_storage.set_transport_configured(configured=False) return # If we've gotten here, that means that this span is a child span of # this context's root span (i.e. it's a zipkin_span inside another # zipkin_span). end_timestamp = time.time() # If self.duration is set, it means the user wants to override it if self.duration: duration = self.duration else: duration = end_timestamp - self.start_timestamp self._span_storage.append(SpanBuilder( trace_id=self.zipkin_attrs.trace_id, name=self.span_name, parent_id=self.zipkin_attrs.parent_span_id, span_id=self.zipkin_attrs.span_id, timestamp=self.timestamp if self.timestamp else self.start_timestamp, duration=duration, annotations=self.annotations, tags=self.binary_annotations, kind=self.kind, service_name=self.service_name, sa_endpoint=self.sa_endpoint, )) def update_binary_annotations(self, extra_annotations): """Updates the binary annotations for the current span. If this trace is not being sampled then this is a no-op. """ if not self.zipkin_attrs: return if not self.logging_context: # This is not the root span, so binary annotations will be added # to the log handler when this span context exits. self.binary_annotations.update(extra_annotations) else: # Otherwise, we're in the context of the root span, so just update # the binary annotations for the logging context directly. self.logging_context.binary_annotations_dict.update(extra_annotations) def add_sa_binary_annotation( self, port=0, service_name='unknown', host='127.0.0.1', ): """Adds a 'sa' binary annotation to the current span. 'sa' binary annotations are useful for situations where you need to log where a request is going but the destination doesn't support zipkin. Note that the span must have 'cs'/'cr' annotations. :param port: The port number of the destination :type port: int :param service_name: The name of the destination service :type service_name: str :param host: Host address of the destination :type host: str """ if not self.zipkin_attrs: return if self.kind != Kind.CLIENT: # TODO: trying to set a sa binary annotation for a non-client span # should result in a logged error return sa_endpoint = create_endpoint( port=port, service_name=service_name, host=host, ) if not self.logging_context: if self.sa_endpoint is not None: raise ValueError('SA annotation already set.') self.sa_endpoint = sa_endpoint else: if self.logging_context.sa_endpoint is not None: raise ValueError('SA annotation already set.') self.logging_context.sa_endpoint = sa_endpoint def override_span_name(self, name): """Overrides the current span name. This is useful if you don't know the span name yet when you create the zipkin_span object. i.e. pyramid_zipkin doesn't know which route the request matched until the function wrapped by the context manager completes. :param name: New span name :type name: str """ self.span_name = name if self.logging_context: self.logging_context.span_name = name def _validate_args(kwargs): if 'kind' in kwargs: raise ValueError( '"kind" is not valid in this context. ' 'You probably want to use zipkin_span()' ) class zipkin_client_span(zipkin_span): """Logs a client-side zipkin span. Subclass of :class:`zipkin_span` using only annotations relevant to clients """ def __init__(self, *args, **kwargs): """Logs a zipkin span with client annotations. See :class:`zipkin_span` for arguments """ _validate_args(kwargs) kwargs['kind'] = Kind.CLIENT super(zipkin_client_span, self).__init__(*args, **kwargs) class zipkin_server_span(zipkin_span): """Logs a server-side zipkin span. Subclass of :class:`zipkin_span` using only annotations relevant to servers """ def __init__(self, *args, **kwargs): """Logs a zipkin span with server annotations. See :class:`zipkin_span` for arguments """ _validate_args(kwargs) kwargs['kind'] = Kind.SERVER super(zipkin_server_span, self).__init__(*args, **kwargs) def create_attrs_for_span( sample_rate=100.0, trace_id=None, span_id=None, use_128bit_trace_id=False, ): """Creates a set of zipkin attributes for a span. :param sample_rate: Float between 0.0 and 100.0 to determine sampling rate :type sample_rate: float :param trace_id: Optional 16-character hex string representing a trace_id. If this is None, a random trace_id will be generated. :type trace_id: str :param span_id: Optional 16-character hex string representing a span_id. If this is None, a random span_id will be generated. :type span_id: str :param use_128bit_trace_id: If true, generate 128-bit trace_ids :type use_128bit_trace_id: boolean """ # Calculate if this trace is sampled based on the sample rate if trace_id is None: if use_128bit_trace_id: trace_id = generate_random_128bit_string() else: trace_id = generate_random_64bit_string() if span_id is None: span_id = generate_random_64bit_string() if sample_rate == 0.0: is_sampled = False else: is_sampled = (random.random() * 100) < sample_rate return ZipkinAttrs( trace_id=trace_id, span_id=span_id, parent_span_id=None, flags='0', is_sampled=is_sampled, ) def create_http_headers_for_new_span(context_stack=None): """ Generate the headers for a new zipkin span. .. note:: If the method is not called from within a zipkin_trace context, empty dict will be returned back. :returns: dict containing (X-B3-TraceId, X-B3-SpanId, X-B3-ParentSpanId, X-B3-Flags and X-B3-Sampled) keys OR an empty dict. """ if context_stack is None: context_stack = ThreadLocalStack() zipkin_attrs = context_stack.get() if not zipkin_attrs: return {} return { 'X-B3-TraceId': zipkin_attrs.trace_id, 'X-B3-SpanId': generate_random_64bit_string(), 'X-B3-ParentSpanId': zipkin_attrs.span_id, 'X-B3-Flags': '0', 'X-B3-Sampled': '1' if zipkin_attrs.is_sampled else '0', } py_zipkin-0.15.0/py_zipkin/exception.py0000644000174700017470000000016213363457355017454 0ustar debiandebian# -*- coding: utf-8 -*- class ZipkinError(Exception): """Custom error to be raised on Zipkin exceptions.""" py_zipkin-0.15.0/py_zipkin/__init__.py0000644000174700017470000000070113363457355017214 0ustar debiandebian# DeprecationWarnings are silent since Python 2.7. # The `default` filter only prints the first occurrence of matching warnings for # each location where the warning is issued, so that we don't spam our users logs. import warnings warnings.simplefilter('default', DeprecationWarning) # Export useful functions and types from private modules. from py_zipkin.encoding._types import Encoding # noqa from py_zipkin.encoding._types import Kind # noqa py_zipkin-0.15.0/py_zipkin/thread_local.py0000644000174700017470000000427213363457355020105 0ustar debiandebian# -*- coding: utf-8 -*- import threading import warnings _thread_local = threading.local() def get_thread_local_zipkin_attrs(): """A wrapper to return _thread_local.zipkin_attrs Returns a list of ZipkinAttrs objects, used for intra-process context propagation. :returns: list that may contain zipkin attribute tuples :rtype: list """ if not hasattr(_thread_local, 'zipkin_attrs'): _thread_local.zipkin_attrs = [] return _thread_local.zipkin_attrs def get_thread_local_span_storage(): """A wrapper to return _thread_local.span_storage Returns a SpanStorage object used to temporarily store all spans created in the current process. The transport handlers will pull from this storage when they emit the spans. :returns: SpanStore object containing all non-root spans. :rtype: py_zipkin.storage.SpanStore """ if not hasattr(_thread_local, 'span_storage'): from py_zipkin.storage import SpanStorage _thread_local.span_storage = SpanStorage() return _thread_local.span_storage def get_zipkin_attrs(): """Get the topmost level zipkin attributes stored. :returns: tuple containing zipkin attrs :rtype: :class:`zipkin.ZipkinAttrs` """ from py_zipkin.storage import ThreadLocalStack warnings.warn( 'Use py_zipkin.stack.ThreadLocalStack().get', DeprecationWarning, ) return ThreadLocalStack().get() def pop_zipkin_attrs(): """Pop the topmost level zipkin attributes, if present. :returns: tuple containing zipkin attrs :rtype: :class:`zipkin.ZipkinAttrs` """ from py_zipkin.storage import ThreadLocalStack warnings.warn( 'Use py_zipkin.stack.ThreadLocalStack().pop', DeprecationWarning, ) return ThreadLocalStack().pop() def push_zipkin_attrs(zipkin_attr): """Stores the zipkin attributes to thread local. :param zipkin_attr: tuple containing zipkin related attrs :type zipkin_attr: :class:`zipkin.ZipkinAttrs` """ from py_zipkin.storage import ThreadLocalStack warnings.warn( 'Use py_zipkin.stack.ThreadLocalStack().push', DeprecationWarning, ) return ThreadLocalStack().push(zipkin_attr) py_zipkin-0.15.0/py_zipkin/transport.py0000644000174700017470000000262713363457355017522 0ustar debiandebian# -*- coding: utf-8 -*- class BaseTransportHandler(object): def get_max_payload_bytes(self): # pragma: no cover """Returns the maximum payload size for this transport. Most transports have a maximum packet size that can be sent. For example, UDP has a 65507 bytes MTU. py_zipkin automatically batches collected spans for performance reasons. The batch size is going to be the minimum between `get_max_payload_bytes` and `max_span_batch_size` from `zipkin_span`. If you don't want to enforce a max payload size, return None. :returns: max payload size in bytes or None. """ raise NotImplementedError('get_max_payload_bytes is not implemented') def send(self, payload): # pragma: no cover """Sends the encoded payload over the transport. :argument payload: encoded list of spans. """ raise NotImplementedError('send is not implemented') def __call__(self, payload): """Internal wrapper around `send`. Do not override. Mostly used to keep backward compatibility with older transports implemented as functions. However decoupling the function developers override and what's internally called by py_zipkin will allow us to add extra logic here in the future without having the users update their code every time. """ self.send(payload) py_zipkin-0.15.0/py_zipkin/thrift/0000755000174700017470000000000013363457445016405 5ustar debiandebianpy_zipkin-0.15.0/py_zipkin/thrift/__init__.py0000644000174700017470000001456613363457355020532 0ustar debiandebian# -*- coding: utf-8 -*- import os import socket import struct import thriftpy from thriftpy.protocol.binary import TBinaryProtocol from thriftpy.protocol.binary import write_list_begin from thriftpy.thrift import TType from thriftpy.transport import TMemoryBuffer from py_zipkin.util import unsigned_hex_to_signed_int thrift_filepath = os.path.join(os.path.dirname(__file__), 'zipkinCore.thrift') zipkin_core = thriftpy.load(thrift_filepath, module_name="zipkinCore_thrift") SERVER_ADDR_VAL = '\x01' LIST_HEADER_SIZE = 5 # size in bytes of the encoded list header dummy_endpoint = zipkin_core.Endpoint() def create_annotation(timestamp, value, host): """ Create a zipkin annotation object :param timestamp: timestamp of when the annotation occured in microseconds :param value: name of the annotation, such as 'sr' :param host: zipkin endpoint object :returns: zipkin annotation object """ return zipkin_core.Annotation(timestamp=timestamp, value=value, host=host) def create_binary_annotation(key, value, annotation_type, host): """ Create a zipkin binary annotation object :param key: name of the annotation, such as 'http.uri' :param value: value of the annotation, such as a URI :param annotation_type: type of annotation, such as AnnotationType.I32 :param host: zipkin endpoint object :returns: zipkin binary annotation object """ return zipkin_core.BinaryAnnotation( key=key, value=value, annotation_type=annotation_type, host=host, ) def create_endpoint(port=0, service_name='unknown', ipv4=None, ipv6=None): """Create a zipkin Endpoint object. An Endpoint object holds information about the network context of a span. :param port: int value of the port. Defaults to 0 :param service_name: service name as a str. Defaults to 'unknown' :param ipv4: ipv4 host address :param ipv6: ipv6 host address :returns: thrift Endpoint object """ ipv4_int = 0 ipv6_binary = None # Convert ip address to network byte order if ipv4: ipv4_int = struct.unpack('!i', socket.inet_pton(socket.AF_INET, ipv4))[0] if ipv6: ipv6_binary = socket.inet_pton(socket.AF_INET6, ipv6) # Zipkin passes unsigned values in signed types because Thrift has no # unsigned types, so we have to convert the value. port = struct.unpack('h', struct.pack('H', port))[0] return zipkin_core.Endpoint( ipv4=ipv4_int, ipv6=ipv6_binary, port=port, service_name=service_name, ) def copy_endpoint_with_new_service_name(endpoint, service_name): """Copies a copy of a given endpoint with a new service name. This should be very fast, on the order of several microseconds. :param endpoint: existing zipkin_core.Endpoint object :param service_name: str of new service name :returns: zipkin Endpoint object """ return zipkin_core.Endpoint( ipv4=endpoint.ipv4, port=endpoint.port, service_name=service_name, ) def annotation_list_builder(annotations, host): """ Reformat annotations dict to return list of corresponding zipkin_core objects. :param annotations: dict containing key as annotation name, value being timestamp in seconds(float). :type host: :class:`zipkin_core.Endpoint` :returns: a list of annotation zipkin_core objects :rtype: list """ return [ create_annotation(int(timestamp * 1000000), key, host) for key, timestamp in annotations.items() ] def binary_annotation_list_builder(binary_annotations, host): """ Reformat binary annotations dict to return list of zipkin_core objects. The value of the binary annotations MUST be in string format. :param binary_annotations: dict with key, value being the name and value of the binary annotation being logged. :type host: :class:`zipkin_core.Endpoint` :returns: a list of binary annotation zipkin_core objects :rtype: list """ # TODO: Remove the type hard-coding of STRING to take it as a param option. ann_type = zipkin_core.AnnotationType.STRING return [ create_binary_annotation(key, str(value), ann_type, host) for key, value in binary_annotations.items() ] def create_span( span_id, parent_span_id, trace_id, span_name, annotations, binary_annotations, timestamp_s, duration_s, ): """Takes a bunch of span attributes and returns a thriftpy representation of the span. Timestamps passed in are in seconds, they're converted to microseconds before thrift encoding. """ # Check if trace_id is 128-bit. If so, record trace_id_high separately. trace_id_length = len(trace_id) trace_id_high = None if trace_id_length > 16: assert trace_id_length == 32 trace_id, trace_id_high = trace_id[16:], trace_id[:16] if trace_id_high: trace_id_high = unsigned_hex_to_signed_int(trace_id_high) span_dict = { 'trace_id': unsigned_hex_to_signed_int(trace_id), 'name': span_name, 'id': unsigned_hex_to_signed_int(span_id), 'annotations': annotations, 'binary_annotations': binary_annotations, 'timestamp': int(timestamp_s * 1000000) if timestamp_s else None, 'duration': int(duration_s * 1000000) if duration_s else None, 'trace_id_high': trace_id_high, } if parent_span_id: span_dict['parent_id'] = unsigned_hex_to_signed_int(parent_span_id) return zipkin_core.Span(**span_dict) def span_to_bytes(thrift_span): """ Returns a TBinaryProtocol encoded Thrift span. :param thrift_span: thrift object to encode. :returns: thrift object in TBinaryProtocol format bytes. """ transport = TMemoryBuffer() protocol = TBinaryProtocol(transport) thrift_span.write(protocol) return bytes(transport.getvalue()) def encode_bytes_list(binary_thrift_obj_list): # pragma: no cover """ Returns a TBinaryProtocol encoded list of Thrift objects. :param binary_thrift_obj_list: list of TBinaryProtocol objects to encode. :returns: bynary object representing the encoded list. """ transport = TMemoryBuffer() write_list_begin(transport, TType.STRUCT, len(binary_thrift_obj_list)) for thrift_bin in binary_thrift_obj_list: transport.write(thrift_bin) return bytes(transport.getvalue()) py_zipkin-0.15.0/py_zipkin/thrift/zipkinCore.thrift0000644000174700017470000004610213363457355021747 0ustar debiandebian# Copyright 2012 Twitter Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. namespace java com.twitter.zipkin.thriftjava #@namespace scala com.twitter.zipkin.thriftscala namespace rb Zipkin #************** Annotation.value ************** /** * The client sent ("cs") a request to a server. There is only one send per * span. For example, if there's a transport error, each attempt can be logged * as a WIRE_SEND annotation. * * If chunking is involved, each chunk could be logged as a separate * CLIENT_SEND_FRAGMENT in the same span. * * Annotation.host is not the server. It is the host which logged the send * event, almost always the client. When logging CLIENT_SEND, instrumentation * should also log the SERVER_ADDR. */ const string CLIENT_SEND = "cs" /** * The client received ("cr") a response from a server. There is only one * receive per span. For example, if duplicate responses were received, each * can be logged as a WIRE_RECV annotation. * * If chunking is involved, each chunk could be logged as a separate * CLIENT_RECV_FRAGMENT in the same span. * * Annotation.host is not the server. It is the host which logged the receive * event, almost always the client. The actual endpoint of the server is * recorded separately as SERVER_ADDR when CLIENT_SEND is logged. */ const string CLIENT_RECV = "cr" /** * The server sent ("ss") a response to a client. There is only one response * per span. If there's a transport error, each attempt can be logged as a * WIRE_SEND annotation. * * Typically, a trace ends with a server send, so the last timestamp of a trace * is often the timestamp of the root span's server send. * * If chunking is involved, each chunk could be logged as a separate * SERVER_SEND_FRAGMENT in the same span. * * Annotation.host is not the client. It is the host which logged the send * event, almost always the server. The actual endpoint of the client is * recorded separately as CLIENT_ADDR when SERVER_RECV is logged. */ const string SERVER_SEND = "ss" /** * The server received ("sr") a request from a client. There is only one * request per span. For example, if duplicate responses were received, each * can be logged as a WIRE_RECV annotation. * * Typically, a trace starts with a server receive, so the first timestamp of a * trace is often the timestamp of the root span's server receive. * * If chunking is involved, each chunk could be logged as a separate * SERVER_RECV_FRAGMENT in the same span. * * Annotation.host is not the client. It is the host which logged the receive * event, almost always the server. When logging SERVER_RECV, instrumentation * should also log the CLIENT_ADDR. */ const string SERVER_RECV = "sr" /** * Message send ("ms") is a request to send a message to a destination, usually * a broker. This may be the only annotation in a messaging span. If WIRE_SEND * exists in the same span, it follows this moment and clarifies delays sending * the message, such as batching. * * Unlike RPC annotations like CLIENT_SEND, messaging spans never share a span * ID. For example, "ms" should always be the parent of "mr". * * Annotation.host is not the destination, it is the host which logged the send * event: the producer. When annotating MESSAGE_SEND, instrumentation should * also tag the MESSAGE_ADDR. */ const string MESSAGE_SEND = "ms" /** * A consumer received ("mr") a message from a broker. This may be the only * annotation in a messaging span. If WIRE_RECV exists in the same span, it * precedes this moment and clarifies any local queuing delay. * * Unlike RPC annotations like SERVER_RECV, messaging spans never share a span * ID. For example, "mr" should always be a child of "ms" unless it is a root * span. * * Annotation.host is not the broker, it is the host which logged the receive * event: the consumer. When annotating MESSAGE_RECV, instrumentation should * also tag the MESSAGE_ADDR. */ const string MESSAGE_RECV = "mr" /** * Optionally logs an attempt to send a message on the wire. Multiple wire send * events could indicate network retries. A lag between client or server send * and wire send might indicate queuing or processing delay. */ const string WIRE_SEND = "ws" /** * Optionally logs an attempt to receive a message from the wire. Multiple wire * receive events could indicate network retries. A lag between wire receive * and client or server receive might indicate queuing or processing delay. */ const string WIRE_RECV = "wr" /** * Optionally logs progress of a (CLIENT_SEND, WIRE_SEND). For example, this * could be one chunk in a chunked request. */ const string CLIENT_SEND_FRAGMENT = "csf" /** * Optionally logs progress of a (CLIENT_RECV, WIRE_RECV). For example, this * could be one chunk in a chunked response. */ const string CLIENT_RECV_FRAGMENT = "crf" /** * Optionally logs progress of a (SERVER_SEND, WIRE_SEND). For example, this * could be one chunk in a chunked response. */ const string SERVER_SEND_FRAGMENT = "ssf" /** * Optionally logs progress of a (SERVER_RECV, WIRE_RECV). For example, this * could be one chunk in a chunked request. */ const string SERVER_RECV_FRAGMENT = "srf" #***** BinaryAnnotation.key ****** /** * The domain portion of the URL or host header. Ex. "mybucket.s3.amazonaws.com" * * Used to filter by host as opposed to ip address. */ const string HTTP_HOST = "http.host" /** * The HTTP method, or verb, such as "GET" or "POST". * * Used to filter against an http route. */ const string HTTP_METHOD = "http.method" /** * The absolute http path, without any query parameters. Ex. "/objects/abcd-ff" * * Used as a filter or to clarify the request path for a given route. For example, the path for * a route "/objects/:objectId" could be "/objects/abdc-ff". This does not limit cardinality like * HTTP_ROUTE("http.route") can, so is not a good input to a span name. * * The Zipkin query api only supports equals filters. Dropping query parameters makes the number * of distinct URIs less. For example, one can query for the same resource, regardless of signing * parameters encoded in the query line. Dropping query parameters also limits the security impact * of this tag. * * Historical note: This was commonly expressed as "http.uri" in zipkin, even though it was most */ const string HTTP_PATH = "http.path" /** * The route which a request matched or "" (empty string) if routing is supported, but there was no * match. Ex "/users/{userId}" * * Unlike HTTP_PATH("http.path"), this value is fixed cardinality, so is a safe input to a span * name function or a metrics dimension. Different formats are possible. For example, the following * are all valid route templates: "/users" "/users/:userId" "/users/*" * * Route-based span name generation often uses other tags, such as HTTP_METHOD("http.method") and * HTTP_STATUS_CODE("http.status_code"). Route-based names can look like "get /users/{userId}", * "post /users", "get not_found" or "get redirected". */ const string HTTP_ROUTE = "http.route" /** * The entire URL, including the scheme, host and query parameters if available. Ex. * "https://mybucket.s3.amazonaws.com/objects/abcd-ff?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Algorithm=AWS4-HMAC-SHA256..." * * Combined with HTTP_METHOD, you can understand the fully-qualified request line. * * This is optional as it may include private data or be of considerable length. */ const string HTTP_URL = "http.url" /** * The HTTP status code, when not in 2xx range. Ex. "503" * * Used to filter for error status. */ const string HTTP_STATUS_CODE = "http.status_code" /** * The size of the non-empty HTTP request body, in bytes. Ex. "16384" * * Large uploads can exceed limits or contribute directly to latency. */ const string HTTP_REQUEST_SIZE = "http.request.size" /** * The size of the non-empty HTTP response body, in bytes. Ex. "16384" * * Large downloads can exceed limits or contribute directly to latency. */ const string HTTP_RESPONSE_SIZE = "http.response.size" /** * The value of "lc" is the component or namespace of a local span. * * BinaryAnnotation.host adds service context needed to support queries. * * Local Component("lc") supports three key features: flagging, query by * service and filtering Span.name by namespace. * * While structurally the same, local spans are fundamentally different than * RPC spans in how they should be interpreted. For example, zipkin v1 tools * center on RPC latency and service graphs. Root local-spans are neither * indicative of critical path RPC latency, nor have impact on the shape of a * service graph. By flagging with "lc", tools can special-case local spans. * * Zipkin v1 Spans are unqueryable unless they can be indexed by service name. * The only path to a service name is by (Binary)?Annotation.host.serviceName. * By logging "lc", a local span can be queried even if no other annotations * are logged. * * The value of "lc" is the namespace of Span.name. For example, it might be * "finatra2", for a span named "bootstrap". "lc" allows you to resolves * conflicts for the same Span.name, for example "finatra/bootstrap" vs * "finch/bootstrap". Using local component, you'd search for spans named * "bootstrap" where "lc=finch" */ const string LOCAL_COMPONENT = "lc" #***** Annotation.value or BinaryAnnotation.key ****** /** * When an annotation value, this indicates when an error occurred. When a * binary annotation key, the value is a human readable message associated * with an error. * * Due to transient errors, an ERROR annotation should not be interpreted * as a span failure, even the annotation might explain additional latency. * Instrumentation should add the ERROR binary annotation when the operation * failed and couldn't be recovered. * * Here's an example: A span has an ERROR annotation, added when a WIRE_SEND * failed. Another WIRE_SEND succeeded, so there's no ERROR binary annotation * on the span because the overall operation succeeded. * * Note that RPC spans often include both client and server hosts: It is * possible that only one side perceived the error. */ const string ERROR = "error" #***** BinaryAnnotation.key where value = [1] and annotation_type = BOOL ****** /** * Indicates a client address ("ca") in a span. Most likely, there's only one. * Multiple addresses are possible when a client changes its ip or port within * a span. */ const string CLIENT_ADDR = "ca" /** * Indicates a server address ("sa") in a span. Most likely, there's only one. * Multiple addresses are possible when a client is redirected, or fails to a * different server ip or port. */ const string SERVER_ADDR = "sa" /** * Indicates the remote address of a messaging span, usually the broker. */ const string MESSAGE_ADDR = "ma" /** * Indicates the network context of a service recording an annotation with two * exceptions. * * When a BinaryAnnotation, and key is CLIENT_ADDR or SERVER_ADDR, * the endpoint indicates the source or destination of an RPC. This exception * allows zipkin to display network context of uninstrumented services, or * clients such as web browsers. */ struct Endpoint { /** * IPv4 host address packed into 4 bytes. * * Ex for the ip 1.2.3.4, it would be (1 << 24) | (2 << 16) | (3 << 8) | 4 */ 1: i32 ipv4 /** * IPv4 port or 0, if unknown. * * Note: this is to be treated as an unsigned integer, so watch for negatives. */ 2: i16 port /** * Classifier of a source or destination in lowercase, such as "zipkin-web". * * This is the primary parameter for trace lookup, so should be intuitive as * possible, for example, matching names in service discovery. * * Conventionally, when the service name isn't known, service_name = "unknown". * However, it is also permissible to set service_name = "" (empty string). * The difference in the latter usage is that the span will not be queryable * by service name unless more information is added to the span with non-empty * service name, e.g. an additional annotation from the server. * * Particularly clients may not have a reliable service name at ingest. One * approach is to set service_name to "" at ingest, and later assign a * better label based on binary annotations, such as user agent. */ 3: string service_name /** * IPv6 host address packed into 16 bytes. Ex Inet6Address.getBytes() */ 4: optional binary ipv6 } /** * Associates an event that explains latency with a timestamp. * * Unlike log statements, annotations are often codes: for example "sr". */ struct Annotation { /** * Microseconds from epoch. * * This value should use the most precise value possible. For example, * gettimeofday or multiplying currentTimeMillis by 1000. */ 1: i64 timestamp /** * Usually a short tag indicating an event, like "sr" or "finagle.retry". */ 2: string value /** * The host that recorded the value, primarily for query by service name. */ 3: optional Endpoint host // don't reuse 4: optional i32 OBSOLETE_duration // how long did the operation take? microseconds } /** * A subset of thrift base types, except BYTES. */ enum AnnotationType { /** * Set to 0x01 when key is CLIENT_ADDR or SERVER_ADDR */ BOOL, /** * No encoding, or type is unknown. */ BYTES, I16, I32, I64, DOUBLE, /** * the only type zipkin v1 supports search against. */ STRING } /** * Binary annotations are tags applied to a Span to give it context. For * example, a binary annotation of HTTP_PATH ("http.path") could the path * to a resource in a RPC call. * * Binary annotations of type STRING are always queryable, though more a * historical implementation detail than a structural concern. * * Binary annotations can repeat, and vary on the host. Similar to Annotation, * the host indicates who logged the event. This allows you to tell the * difference between the client and server side of the same key. For example, * the key "http.path" might be different on the client and server side due to * rewriting, like "/api/v1/myresource" vs "/myresource. Via the host field, * you can see the different points of view, which often help in debugging. */ struct BinaryAnnotation { /** * Name used to lookup spans, such as "http.path" or "finagle.version". */ 1: string key, /** * Serialized thrift bytes, in TBinaryProtocol format. * * For legacy reasons, byte order is big-endian. See THRIFT-3217. */ 2: binary value, /** * The thrift type of value, most often STRING. * * annotation_type shouldn't vary for the same key. */ 3: AnnotationType annotation_type, /** * The host that recorded value, allowing query by service name or address. * * There are two exceptions: when key is "ca" or "sa", this is the source or * destination of an RPC. This exception allows zipkin to display network * context of uninstrumented services, such as browsers or databases. */ 4: optional Endpoint host } /** * A trace is a series of spans (often RPC calls) which form a latency tree. * * Spans are usually created by instrumentation in RPC clients or servers, but * can also represent in-process activity. Annotations in spans are similar to * log statements, and are sometimes created directly by application developers * to indicate events of interest, such as a cache miss. * * The root span is where parent_id = Nil; it usually has the longest duration * in the trace. * * Span identifiers are packed into i64s, but should be treated opaquely. * String encoding is fixed-width lower-hex, to avoid signed interpretation. */ struct Span { /** * Unique 8-byte identifier for a trace, set on all spans within it. */ 1: i64 trace_id /** * Span name in lowercase, rpc method for example. Conventionally, when the * span name isn't known, name = "unknown". */ 3: string name, /** * Unique 8-byte identifier of this span within a trace. A span is uniquely * identified in storage by (trace_id, id). */ 4: i64 id, /** * The parent's Span.id; absent if this the root span in a trace. */ 5: optional i64 parent_id, /** * Associates events that explain latency with a timestamp. Unlike log * statements, annotations are often codes: for example SERVER_RECV("sr"). * Annotations are sorted ascending by timestamp. */ 6: list annotations, /** * Tags a span with context, usually to support query or aggregation. For * example, a binary annotation key could be "http.path". */ 8: list binary_annotations /** * True is a request to store this span even if it overrides sampling policy. */ 9: optional bool debug = 0 /** * Epoch microseconds of the start of this span, absent if this an incomplete * span. * * This value should be set directly by instrumentation, using the most * precise value possible. For example, gettimeofday or syncing nanoTime * against a tick of currentTimeMillis. * * For compatibility with instrumentation that precede this field, collectors * or span stores can derive this via Annotation.timestamp. * For example, SERVER_RECV.timestamp or CLIENT_SEND.timestamp. * * Timestamp is nullable for input only. Spans without a timestamp cannot be * presented in a timeline: Span stores should not output spans missing a * timestamp. * * There are two known edge-cases where this could be absent: both cases * exist when a collector receives a span in parts and a binary annotation * precedes a timestamp. This is possible when.. * - The span is in-flight (ex not yet received a timestamp) * - The span's start event was lost */ 10: optional i64 timestamp, /** * Measurement in microseconds of the critical path, if known. Durations of * less than one microsecond must be rounded up to 1 microsecond. * * This value should be set directly, as opposed to implicitly via annotation * timestamps. Doing so encourages precision decoupled from problems of * clocks, such as skew or NTP updates causing time to move backwards. * * For compatibility with instrumentation that precede this field, collectors * or span stores can derive this by subtracting Annotation.timestamp. * For example, SERVER_SEND.timestamp - SERVER_RECV.timestamp. * * If this field is persisted as unset, zipkin will continue to work, except * duration query support will be implementation-specific. Similarly, setting * this field non-atomically is implementation-specific. * * This field is i64 vs i32 to support spans longer than 35 minutes. */ 11: optional i64 duration /** * Optional unique 8-byte additional identifier for a trace. If non zero, this * means the trace uses 128 bit traceIds instead of 64 bit. */ 12: optional i64 trace_id_high } py_zipkin-0.15.0/py_zipkin/logging_helper.py0000644000174700017470000001521013363457355020443 0ustar debiandebian# -*- coding: utf-8 -*- import os import time from py_zipkin import Kind from py_zipkin.encoding._helpers import SpanBuilder from py_zipkin.encoding._helpers import copy_endpoint_with_new_service_name from py_zipkin.encoding._encoders import get_encoder from py_zipkin.exception import ZipkinError from py_zipkin.transport import BaseTransportHandler LOGGING_END_KEY = 'py_zipkin.logging_end' class ZipkinLoggingContext(object): """A logging context specific to a Zipkin trace. If the trace is sampled, the logging context sends serialized Zipkin spans to a transport_handler. The logging context sends root "server" or "client" span, as well as all local child spans collected within this context. This class should only be used by the main `zipkin_span` entrypoint. """ def __init__( self, zipkin_attrs, endpoint, span_name, transport_handler, report_root_timestamp, span_storage, service_name, binary_annotations=None, add_logging_annotation=False, client_context=False, max_span_batch_size=None, firehose_handler=None, encoding=None, ): self.zipkin_attrs = zipkin_attrs self.endpoint = endpoint self.span_name = span_name self.transport_handler = transport_handler self.response_status_code = 0 self.span_storage = span_storage self.service_name = service_name self.report_root_timestamp = report_root_timestamp self.binary_annotations_dict = binary_annotations or {} self.add_logging_annotation = add_logging_annotation self.client_context = client_context self.max_span_batch_size = max_span_batch_size self.firehose_handler = firehose_handler self.sa_endpoint = None self.encoder = get_encoder(encoding) def start(self): """Actions to be taken before request is handled.""" # Record the start timestamp. self.start_timestamp = time.time() return self def stop(self): """Actions to be taken post request handling. """ self.emit_spans() def emit_spans(self): """Main function to log all the annotations stored during the entire request. This is done if the request is sampled and the response was a success. It also logs the service (`ss` and `sr`) or the client ('cs' and 'cr') annotations. """ # FIXME: Should have a single aggregate handler if self.firehose_handler: # FIXME: We need to allow different batching settings per handler self._emit_spans_with_span_sender( ZipkinBatchSender(self.firehose_handler, self.max_span_batch_size, self.encoder) ) if not self.zipkin_attrs.is_sampled: self.span_storage.clear() return span_sender = ZipkinBatchSender(self.transport_handler, self.max_span_batch_size, self.encoder) self._emit_spans_with_span_sender(span_sender) self.span_storage.clear() def _emit_spans_with_span_sender(self, span_sender): with span_sender: end_timestamp = time.time() # Collect, annotate, and log client spans from the logging handler for span in self.span_storage: span.local_endpoint = copy_endpoint_with_new_service_name( self.endpoint, span.service_name, ) span_sender.add_span(span) annotations = {} if self.add_logging_annotation: annotations[LOGGING_END_KEY] = time.time() span_sender.add_span(SpanBuilder( trace_id=self.zipkin_attrs.trace_id, name=self.span_name, parent_id=self.zipkin_attrs.parent_span_id, span_id=self.zipkin_attrs.span_id, timestamp=self.start_timestamp, duration=end_timestamp - self.start_timestamp, annotations=annotations, tags=self.binary_annotations_dict, kind=Kind.CLIENT if self.client_context else Kind.SERVER, local_endpoint=self.endpoint, service_name=self.service_name, sa_endpoint=self.sa_endpoint, report_timestamp=self.report_root_timestamp, )) class ZipkinBatchSender(object): MAX_PORTION_SIZE = 100 def __init__(self, transport_handler, max_portion_size, encoder): self.transport_handler = transport_handler self.max_portion_size = max_portion_size or self.MAX_PORTION_SIZE self.encoder = encoder if isinstance(self.transport_handler, BaseTransportHandler): self.max_payload_bytes = self.transport_handler.get_max_payload_bytes() else: self.max_payload_bytes = None def __enter__(self): self._reset_queue() return self def __exit__(self, _exc_type, _exc_value, _exc_traceback): if any((_exc_type, _exc_value, _exc_traceback)): filename = os.path.split(_exc_traceback.tb_frame.f_code.co_filename)[1] error = '({0}:{1}) {2}: {3}'.format( filename, _exc_traceback.tb_lineno, _exc_type.__name__, _exc_value, ) raise ZipkinError(error) else: self.flush() def _reset_queue(self): self.queue = [] self.current_size = 0 def add_span(self, internal_span): encoded_span = self.encoder.encode_span(internal_span) # If we've already reached the max batch size or the new span doesn't # fit in max_payload_bytes, send what we've collected until now and # start a new batch. is_over_size_limit = ( self.max_payload_bytes is not None and not self.encoder.fits( current_count=len(self.queue), current_size=self.current_size, max_size=self.max_payload_bytes, new_span=encoded_span, ) ) is_over_portion_limit = len(self.queue) >= self.max_portion_size if is_over_size_limit or is_over_portion_limit: self.flush() self.queue.append(encoded_span) self.current_size += len(encoded_span) def flush(self): if self.transport_handler and len(self.queue) > 0: message = self.encoder.encode_queue(self.queue) self.transport_handler(message) self._reset_queue() py_zipkin-0.15.0/py_zipkin/storage.py0000644000174700017470000000362513363457355017131 0ustar debiandebianfrom collections import deque from py_zipkin import thread_local class Stack(object): """ Stack is a simple stack class. It offers the operations push, pop and get. The latter two return None if the stack is empty. """ def __init__(self, storage): self._storage = storage def push(self, item): self._storage.append(item) def pop(self): if self._storage: return self._storage.pop() def get(self): if self._storage: return self._storage[-1] class ThreadLocalStack(Stack): """ ThreadLocalStack is variant of Stack that uses a thread local storage. The thread local storage is accessed lazily in every method call, so the thread that calls the method matters, not the thread that instantiated the class. Every instance shares the same thread local data. """ def __init__(self): pass @property def _storage(self): return thread_local.get_thread_local_zipkin_attrs() class SpanStorage(deque): def __init__(self): super(SpanStorage, self).__init__() self._is_transport_configured = False def is_transport_configured(self): """Helper function to check whether a transport is configured. We need to propagate this info to the child zipkin_spans since if no transport is set-up they should not generate any Span to avoid memory leaks. :returns: whether transport is configured or not :rtype: bool """ return self._is_transport_configured def set_transport_configured(self, configured): """Set whether the transport is configured or not. :param configured: whether transport is configured or not :type configured: bool """ self._is_transport_configured = configured def default_span_storage(): return thread_local.get_thread_local_span_storage() py_zipkin-0.15.0/py_zipkin/util.py0000644000174700017470000000354513363457355016443 0ustar debiandebian# -*- coding: utf-8 -*- import random import struct import time def generate_random_64bit_string(): """Returns a 64 bit UTF-8 encoded string. In the interests of simplicity, this is always cast to a `str` instead of (in py2 land) a unicode string. Certain clients (I'm looking at you, Twisted) don't enjoy unicode headers. :returns: random 16-character string """ return '{:016x}'.format(random.getrandbits(64)) def generate_random_128bit_string(): """Returns a 128 bit UTF-8 encoded string. Follows the same conventions as generate_random_64bit_string(). The upper 32 bits are the current time in epoch seconds, and the lower 96 bits are random. This allows for AWS X-Ray `interop `_ :returns: 32-character hex string """ t = int(time.time()) lower_96 = random.getrandbits(96) return '{:032x}'.format((t << 96) | lower_96) def unsigned_hex_to_signed_int(hex_string): """Converts a 64-bit hex string to a signed int value. This is due to the fact that Apache Thrift only has signed values. Examples: '17133d482ba4f605' => 1662740067609015813 'b6dbb1c2b362bf51' => -5270423489115668655 :param hex_string: the string representation of a zipkin ID :returns: signed int representation """ return struct.unpack('q', struct.pack('Q', int(hex_string, 16)))[0] def signed_int_to_unsigned_hex(signed_int): """Converts a signed int value to a 64-bit hex string. Examples: 1662740067609015813 => '17133d482ba4f605' -5270423489115668655 => 'b6dbb1c2b362bf51' :param signed_int: an int to convert :returns: unsigned hex string """ hex_string = hex(struct.unpack('Q', struct.pack('q', signed_int))[0])[2:] if hex_string.endswith('L'): return hex_string[:-1] return hex_string py_zipkin-0.15.0/MANIFEST.in0000644000174700017470000000005013363457355014622 0ustar debiandebianinclude README.md include CHANGELOG.rst py_zipkin-0.15.0/CHANGELOG.rst0000644000174700017470000001072113363457355015113 0ustar debiandebian0.15.0 (2018-10-22) ------------------- - Added support for V2 JSON encoding. - Fixed TransportHandler bug that was affecting also V1 JSON. 0.14.1 (2018-10-09) ------------------- - Fixed memory leak introduced in 0.13.0. 0.14.0 (2018-10-01) ------------------- - Support JSON encoding for V1 spans. - Allow overriding the span_name after creation. 0.13.0 (2018-06-25) ------------------- - Removed deprecated `zipkin_logger.debug()` interface. - `py_zipkin.stack` was renamed as `py_zipkin.storage`. If you were importing this module, you'll need to update your code. 0.12.0 (2018-05-29) ------------------- - Support max payload size for transport handlers. - Transport handlers should now be implemented as classes extending py_zipkin.transport.BaseTransportHandler. 0.11.2 (2018-05-23) ------------------- - Don't overwrite passed in annotations 0.11.1 (2018-05-23) ------------------- - Add binary annotations to the span even if the request is not being sampled. This fixes binary annotations for firehose spans. 0.11.0 (2018-02-08) ------------------- - Add support for "firehose mode", which logs 100% of the spans regardless of sample rate. 0.10.1 (2018-02-05) ------------------- - context_stack will now default to `ThreadLocalStack()` if passed as `None` 0.10.0 (2018-02-05) ------------------- - Add support for using explicit in-process context storage instead of using thread_local. This allows you to use py_zipkin in cooperative multitasking environments e.g. asyncio - `py_zipkin.thread_local` is now deprecated. Instead use `py_zipkin.stack.ThreadLocalStack()` - TraceId and SpanId generation performance improvements. - 128-bit TraceIds now start with an epoch timestamp to support easy interop with AWS X-Ray 0.9.0 (2017-07-31) ------------------ - Add batch span sending. Note that spans are now sent in lists. 0.8.3 (2017-07-10) ------------------ - Be defensive about having logging handlers configured to avoid throwing NullHandler attribute errors 0.8.2 (2017-06-30) ------------------ - Don't log ss and sr annotations when in a client span context - Add error binary annotation if an exception occurs 0.8.1 (2017-06-16) ------------------ - Fixed server send timing to more accurately reflect when server send actually occurs. - Replaced logging_start annotation with logging_end 0.8.0 (2017-06-01) ------------------ - Added 128-bit trace id support - Added ability to explicitly specify host for a span - Added exception handling if host can't be determined automatically - SERVER_ADDR ('sa') binary annotations can be added to spans - py36 support 0.7.1 (2017-05-01) ------------------ - Fixed a bug where `update_binary_annotations` would fail for a child span in a trace that is not being sampled 0.7.0 (2017-03-06) ------------------ - Simplify `update_binary_annotations` for both root and non-root spans 0.6.0 (2017-02-03) ------------------ - Added support for forcing `zipkin_span` to report timestamp/duration. Changes API of `zipkin_span`, but defaults back to existing behavior. 0.5.0 (2017-02-01) ------------------ - Properly set timestamp/duration on server and local spans - Updated thrift spec to include these new fields - The `zipkin_span` entrypoint should be backwards compatible 0.4.4 (2016-11-29) ------------------ - Add optional annotation for when Zipkin logging starts 0.4.3 (2016-11-04) ------------------ - Fix bug in zipkin_span decorator 0.4.2 (2016-11-01) ------------------ - Be defensive about transport_handler when logging spans. 0.4.1 (2016-10-24) ------------------ - Add ability to override span_id when creating new ZipkinAttrs. 0.4.0 (2016-10-20) ------------------ - Added `start` and `stop` functions as friendlier versions of the __enter__ and __exit__ functions. 0.3.1 (2016-09-30) ------------------ - Adds new param to thrift.create_endpoint allowing creation of thrift Endpoint objects on a proxy machine representing another host. 0.2.1 (2016-09-30) ------------------ - Officially "release" v0.2.0. Accidentally pushed a v0.2.0 without the proper version bump, so v0.2.1 is the new real version. Please use this instead of v0.2.0. 0.2.0 (2016-09-30) ------------------ - Fix problem where if zipkin_attrs and sample_rate were passed, but zipkin_attrs.is_sampled=True, new zipkin_attrs were being generated. 0.1.2 (2016-09-29) ------------------ - Fix sampling algorithm that always sampled for rates > 50% 0.1.1 (2016-07-05) ------------------ - First py_zipkin version with context manager/decorator functionality. py_zipkin-0.15.0/py_zipkin.egg-info/0000755000174700017470000000000013363457445016577 5ustar debiandebianpy_zipkin-0.15.0/py_zipkin.egg-info/SOURCES.txt0000644000174700017470000000111213363457445020456 0ustar debiandebianCHANGELOG.rst MANIFEST.in README.md setup.cfg setup.py py_zipkin/__init__.py py_zipkin/exception.py py_zipkin/logging_helper.py py_zipkin/storage.py py_zipkin/thread_local.py py_zipkin/transport.py py_zipkin/util.py py_zipkin/zipkin.py py_zipkin.egg-info/PKG-INFO py_zipkin.egg-info/SOURCES.txt py_zipkin.egg-info/dependency_links.txt py_zipkin.egg-info/requires.txt py_zipkin.egg-info/top_level.txt py_zipkin/encoding/__init__.py py_zipkin/encoding/_encoders.py py_zipkin/encoding/_helpers.py py_zipkin/encoding/_types.py py_zipkin/thrift/__init__.py py_zipkin/thrift/zipkinCore.thriftpy_zipkin-0.15.0/py_zipkin.egg-info/PKG-INFO0000644000174700017470000004040413363457445017676 0ustar debiandebianMetadata-Version: 2.1 Name: py-zipkin Version: 0.15.0 Summary: Library for using Zipkin in Python. Home-page: https://github.com/Yelp/py_zipkin Author: Yelp, Inc. Author-email: opensource+py-zipkin@yelp.com License: Copyright Yelp 2018 Description: [![Build Status](https://travis-ci.org/Yelp/py_zipkin.svg?branch=master)](https://travis-ci.org/Yelp/py_zipkin) [![Coverage Status](https://img.shields.io/coveralls/Yelp/py_zipkin.svg)](https://coveralls.io/r/Yelp/py_zipkin) [![PyPi version](https://img.shields.io/pypi/v/py_zipkin.svg)](https://pypi.python.org/pypi/py_zipkin/) [![Supported Python versions](https://img.shields.io/pypi/pyversions/py_zipkin.svg)](https://pypi.python.org/pypi/py_zipkin/) py_zipkin --------- py_zipkin provides a context manager/decorator along with some utilities to facilitate the usage of Zipkin in Python applications. Install ------- ``` pip install py_zipkin ``` Usage ----- py_zipkin requires a `transport_handler` object that handles logging zipkin messages to a central logging service such as kafka or scribe. `py_zipkin.zipkin.zipkin_span` is the main tool for starting zipkin traces or logging spans inside an ongoing trace. zipkin_span can be used as a context manager or a decorator. #### Usage #1: Start a trace with a given sampling rate ```python from py_zipkin.zipkin import zipkin_span def some_function(a, b): with zipkin_span( service_name='my_service', span_name='my_span_name', transport_handler=some_handler, port=42, sample_rate=0.05, # Value between 0.0 and 100.0 ): do_stuff(a, b) ``` #### Usage #2: Trace a service call The difference between this and Usage #1 is that the zipkin_attrs are calculated separately and passed in, thus negating the need of the sample_rate param. ```python # Define a pyramid tween def tween(request): zipkin_attrs = some_zipkin_attr_creator(request) with zipkin_span( service_name='my_service', span_name='my_span_name', zipkin_attrs=zipkin_attrs, transport_handler=some_handler, port=22, ) as zipkin_context: response = handler(request) zipkin_context.update_binary_annotations( some_binary_annotations) return response ``` #### Usage #3: Log a span inside an ongoing trace This can be also be used inside itself to produce continuously nested spans. ```python @zipkin_span(service_name='my_service', span_name='some_function') def some_function(a, b): return do_stuff(a, b) ``` #### Other utilities `zipkin_span.update_binary_annotations()` can be used inside a zipkin trace to add to the existing set of binary annotations. ```python def some_function(a, b): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, ) as zipkin_context: result = do_stuff(a, b) zipkin_context.update_binary_annotations({'result': result}) ``` `zipkin_span.add_sa_binary_annotation()` can be used to add a binary annotation to the current span with the key 'sa'. This function allows the user to specify the destination address of the service being called (useful if the destination doesn't support zipkin). See http://zipkin.io/pages/data_model.html for more information on the 'sa' binary annotation. > NOTE: the V2 span format only support 1 "sa" endpoint (represented by remoteEndpoint) > so `add_sa_binary_annotation` now raises `ValueError` if you try to set multiple "sa" > annotations for the same span. ```python def some_function(): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, ) as zipkin_context: make_call_to_non_instrumented_service() zipkin_context.add_sa_binary_annotation( port=123, service_name='non_instrumented_service', host='12.34.56.78', ) ``` `create_http_headers_for_new_span()` creates a set of HTTP headers that can be forwarded in a request to another service. ```python headers = {} headers.update(create_http_headers_for_new_span()) http_client.get( path='some_url', headers=headers, ) ``` Transport --------- py_zipkin (for the moment) thrift-encodes spans. The actual transport layer is pluggable, though. The recommended way to implement a new transport handler is to subclass `py_zipkin.transport.BaseTransportHandler` and implement the `send` and `get_max_payload_bytes` methods. `send` receives an already encoded thrift list as argument. `get_max_payload_bytes` should return the maximum payload size supported by your transport, or `None` if you can send arbitrarily big messages. The simplest way to get spans to the collector is via HTTP POST. Here's an example of a simple HTTP transport using the `requests` library. This assumes your Zipkin collector is running at localhost:9411. > NOTE: older versions of py_zipkin suggested implementing the transport handler > as a function with a single argument. That's still supported and should work > with the current py_zipkin version, but it's deprecated. ```python import requests from py_zipkin.transport import BaseTransportHandler class HttpTransport(BaseTransportHandler): def get_max_payload_bytes(self): return None def send(self, encoded_span): # The collector expects a thrift-encoded list of spans. requests.post( 'http://localhost:9411/api/v1/spans', data=encoded_span, headers={'Content-Type': 'application/x-thrift'}, ) ``` If you have the ability to send spans over Kafka (more like what you might do in production), you'd do something like the following, using the [kafka-python](https://pypi.python.org/pypi/kafka-python) package: ```python from kafka import SimpleProducer, KafkaClient from py_zipkin.transport import BaseTransportHandler class KafkaTransport(BaseTransportHandler): def get_max_payload_bytes(self): # By default Kafka rejects messages bigger than 1000012 bytes. return 1000012 def send(self, message): kafka_client = KafkaClient('{}:{}'.format('localhost', 9092)) producer = SimpleProducer(kafka_client) producer.send_messages('kafka_topic_name', message) ``` Using in multithreading evironments ----------------------------------- If you want to use py_zipkin in a cooperative multithreading environment, e.g. asyncio, you need to explicitly pass an instance of `py_zipkin.storage.Stack` as parameter `context_stack` for `zipkin_span` and `create_http_headers_for_new_span`. By default, py_zipkin uses a thread local storage for the attributes, which is defined in `py_zipkin.storage.ThreadLocalStack`. Additionally, you'll also need to explicitly pass an instance of `py_zipkin.storage.SpanStorage` as parameter `span_storage` to `zipkin_span`. ```python from py_zipkin.zipkin import zipkin_span from py_zipkin.storage import Stack from py_zipkin.storage import SpanStorage def my_function(): context_stack = Stack() span_storage = SpanStorage() await my_function(context_stack, span_storage) async def my_function(context_stack, span_storage): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, context_stack=context_stack, span_storage=span_storage, ): result = do_stuff(a, b) ``` Firehose mode [EXPERIMENTAL] ---------------------------- "Firehose mode" records 100% of the spans, regardless of sampling rate. This is useful if you want to treat these spans differently, e.g. send them to a different backend that has limited retention. It works in tandem with normal operation, however there may be additional overhead. In order to use this, you add a `firehose_handler` just like you add a `transport_handler`. This feature should be considered experimental and may be removed at any time without warning. If you do use this, be sure to send asynchronously to avoid excess overhead for every request. License ------- Copyright (c) 2018, Yelp, Inc. All Rights reserved. Apache v2 0.15.0 (2018-10-22) ------------------- - Added support for V2 JSON encoding. - Fixed TransportHandler bug that was affecting also V1 JSON. 0.14.1 (2018-10-09) ------------------- - Fixed memory leak introduced in 0.13.0. 0.14.0 (2018-10-01) ------------------- - Support JSON encoding for V1 spans. - Allow overriding the span_name after creation. 0.13.0 (2018-06-25) ------------------- - Removed deprecated `zipkin_logger.debug()` interface. - `py_zipkin.stack` was renamed as `py_zipkin.storage`. If you were importing this module, you'll need to update your code. 0.12.0 (2018-05-29) ------------------- - Support max payload size for transport handlers. - Transport handlers should now be implemented as classes extending py_zipkin.transport.BaseTransportHandler. 0.11.2 (2018-05-23) ------------------- - Don't overwrite passed in annotations 0.11.1 (2018-05-23) ------------------- - Add binary annotations to the span even if the request is not being sampled. This fixes binary annotations for firehose spans. 0.11.0 (2018-02-08) ------------------- - Add support for "firehose mode", which logs 100% of the spans regardless of sample rate. 0.10.1 (2018-02-05) ------------------- - context_stack will now default to `ThreadLocalStack()` if passed as `None` 0.10.0 (2018-02-05) ------------------- - Add support for using explicit in-process context storage instead of using thread_local. This allows you to use py_zipkin in cooperative multitasking environments e.g. asyncio - `py_zipkin.thread_local` is now deprecated. Instead use `py_zipkin.stack.ThreadLocalStack()` - TraceId and SpanId generation performance improvements. - 128-bit TraceIds now start with an epoch timestamp to support easy interop with AWS X-Ray 0.9.0 (2017-07-31) ------------------ - Add batch span sending. Note that spans are now sent in lists. 0.8.3 (2017-07-10) ------------------ - Be defensive about having logging handlers configured to avoid throwing NullHandler attribute errors 0.8.2 (2017-06-30) ------------------ - Don't log ss and sr annotations when in a client span context - Add error binary annotation if an exception occurs 0.8.1 (2017-06-16) ------------------ - Fixed server send timing to more accurately reflect when server send actually occurs. - Replaced logging_start annotation with logging_end 0.8.0 (2017-06-01) ------------------ - Added 128-bit trace id support - Added ability to explicitly specify host for a span - Added exception handling if host can't be determined automatically - SERVER_ADDR ('sa') binary annotations can be added to spans - py36 support 0.7.1 (2017-05-01) ------------------ - Fixed a bug where `update_binary_annotations` would fail for a child span in a trace that is not being sampled 0.7.0 (2017-03-06) ------------------ - Simplify `update_binary_annotations` for both root and non-root spans 0.6.0 (2017-02-03) ------------------ - Added support for forcing `zipkin_span` to report timestamp/duration. Changes API of `zipkin_span`, but defaults back to existing behavior. 0.5.0 (2017-02-01) ------------------ - Properly set timestamp/duration on server and local spans - Updated thrift spec to include these new fields - The `zipkin_span` entrypoint should be backwards compatible 0.4.4 (2016-11-29) ------------------ - Add optional annotation for when Zipkin logging starts 0.4.3 (2016-11-04) ------------------ - Fix bug in zipkin_span decorator 0.4.2 (2016-11-01) ------------------ - Be defensive about transport_handler when logging spans. 0.4.1 (2016-10-24) ------------------ - Add ability to override span_id when creating new ZipkinAttrs. 0.4.0 (2016-10-20) ------------------ - Added `start` and `stop` functions as friendlier versions of the __enter__ and __exit__ functions. 0.3.1 (2016-09-30) ------------------ - Adds new param to thrift.create_endpoint allowing creation of thrift Endpoint objects on a proxy machine representing another host. 0.2.1 (2016-09-30) ------------------ - Officially "release" v0.2.0. Accidentally pushed a v0.2.0 without the proper version bump, so v0.2.1 is the new real version. Please use this instead of v0.2.0. 0.2.0 (2016-09-30) ------------------ - Fix problem where if zipkin_attrs and sample_rate were passed, but zipkin_attrs.is_sampled=True, new zipkin_attrs were being generated. 0.1.2 (2016-09-29) ------------------ - Fix sampling algorithm that always sampled for rates > 50% 0.1.1 (2016-07-05) ------------------ - First py_zipkin version with context manager/decorator functionality. Platform: UNKNOWN Classifier: Development Status :: 3 - Alpha Classifier: Intended Audience :: Developers Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Provides: py_zipkin Description-Content-Type: text/markdown py_zipkin-0.15.0/py_zipkin.egg-info/top_level.txt0000644000174700017470000000001213363457445021322 0ustar debiandebianpy_zipkin py_zipkin-0.15.0/py_zipkin.egg-info/dependency_links.txt0000644000174700017470000000000113363457445022645 0ustar debiandebian py_zipkin-0.15.0/py_zipkin.egg-info/requires.txt0000644000174700017470000000005613363457445021200 0ustar debiandebiansix thriftpy [:python_version=="2.7"] enum34 py_zipkin-0.15.0/README.md0000644000174700017470000001760413363457355014360 0ustar debiandebian[![Build Status](https://travis-ci.org/Yelp/py_zipkin.svg?branch=master)](https://travis-ci.org/Yelp/py_zipkin) [![Coverage Status](https://img.shields.io/coveralls/Yelp/py_zipkin.svg)](https://coveralls.io/r/Yelp/py_zipkin) [![PyPi version](https://img.shields.io/pypi/v/py_zipkin.svg)](https://pypi.python.org/pypi/py_zipkin/) [![Supported Python versions](https://img.shields.io/pypi/pyversions/py_zipkin.svg)](https://pypi.python.org/pypi/py_zipkin/) py_zipkin --------- py_zipkin provides a context manager/decorator along with some utilities to facilitate the usage of Zipkin in Python applications. Install ------- ``` pip install py_zipkin ``` Usage ----- py_zipkin requires a `transport_handler` object that handles logging zipkin messages to a central logging service such as kafka or scribe. `py_zipkin.zipkin.zipkin_span` is the main tool for starting zipkin traces or logging spans inside an ongoing trace. zipkin_span can be used as a context manager or a decorator. #### Usage #1: Start a trace with a given sampling rate ```python from py_zipkin.zipkin import zipkin_span def some_function(a, b): with zipkin_span( service_name='my_service', span_name='my_span_name', transport_handler=some_handler, port=42, sample_rate=0.05, # Value between 0.0 and 100.0 ): do_stuff(a, b) ``` #### Usage #2: Trace a service call The difference between this and Usage #1 is that the zipkin_attrs are calculated separately and passed in, thus negating the need of the sample_rate param. ```python # Define a pyramid tween def tween(request): zipkin_attrs = some_zipkin_attr_creator(request) with zipkin_span( service_name='my_service', span_name='my_span_name', zipkin_attrs=zipkin_attrs, transport_handler=some_handler, port=22, ) as zipkin_context: response = handler(request) zipkin_context.update_binary_annotations( some_binary_annotations) return response ``` #### Usage #3: Log a span inside an ongoing trace This can be also be used inside itself to produce continuously nested spans. ```python @zipkin_span(service_name='my_service', span_name='some_function') def some_function(a, b): return do_stuff(a, b) ``` #### Other utilities `zipkin_span.update_binary_annotations()` can be used inside a zipkin trace to add to the existing set of binary annotations. ```python def some_function(a, b): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, ) as zipkin_context: result = do_stuff(a, b) zipkin_context.update_binary_annotations({'result': result}) ``` `zipkin_span.add_sa_binary_annotation()` can be used to add a binary annotation to the current span with the key 'sa'. This function allows the user to specify the destination address of the service being called (useful if the destination doesn't support zipkin). See http://zipkin.io/pages/data_model.html for more information on the 'sa' binary annotation. > NOTE: the V2 span format only support 1 "sa" endpoint (represented by remoteEndpoint) > so `add_sa_binary_annotation` now raises `ValueError` if you try to set multiple "sa" > annotations for the same span. ```python def some_function(): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, ) as zipkin_context: make_call_to_non_instrumented_service() zipkin_context.add_sa_binary_annotation( port=123, service_name='non_instrumented_service', host='12.34.56.78', ) ``` `create_http_headers_for_new_span()` creates a set of HTTP headers that can be forwarded in a request to another service. ```python headers = {} headers.update(create_http_headers_for_new_span()) http_client.get( path='some_url', headers=headers, ) ``` Transport --------- py_zipkin (for the moment) thrift-encodes spans. The actual transport layer is pluggable, though. The recommended way to implement a new transport handler is to subclass `py_zipkin.transport.BaseTransportHandler` and implement the `send` and `get_max_payload_bytes` methods. `send` receives an already encoded thrift list as argument. `get_max_payload_bytes` should return the maximum payload size supported by your transport, or `None` if you can send arbitrarily big messages. The simplest way to get spans to the collector is via HTTP POST. Here's an example of a simple HTTP transport using the `requests` library. This assumes your Zipkin collector is running at localhost:9411. > NOTE: older versions of py_zipkin suggested implementing the transport handler > as a function with a single argument. That's still supported and should work > with the current py_zipkin version, but it's deprecated. ```python import requests from py_zipkin.transport import BaseTransportHandler class HttpTransport(BaseTransportHandler): def get_max_payload_bytes(self): return None def send(self, encoded_span): # The collector expects a thrift-encoded list of spans. requests.post( 'http://localhost:9411/api/v1/spans', data=encoded_span, headers={'Content-Type': 'application/x-thrift'}, ) ``` If you have the ability to send spans over Kafka (more like what you might do in production), you'd do something like the following, using the [kafka-python](https://pypi.python.org/pypi/kafka-python) package: ```python from kafka import SimpleProducer, KafkaClient from py_zipkin.transport import BaseTransportHandler class KafkaTransport(BaseTransportHandler): def get_max_payload_bytes(self): # By default Kafka rejects messages bigger than 1000012 bytes. return 1000012 def send(self, message): kafka_client = KafkaClient('{}:{}'.format('localhost', 9092)) producer = SimpleProducer(kafka_client) producer.send_messages('kafka_topic_name', message) ``` Using in multithreading evironments ----------------------------------- If you want to use py_zipkin in a cooperative multithreading environment, e.g. asyncio, you need to explicitly pass an instance of `py_zipkin.storage.Stack` as parameter `context_stack` for `zipkin_span` and `create_http_headers_for_new_span`. By default, py_zipkin uses a thread local storage for the attributes, which is defined in `py_zipkin.storage.ThreadLocalStack`. Additionally, you'll also need to explicitly pass an instance of `py_zipkin.storage.SpanStorage` as parameter `span_storage` to `zipkin_span`. ```python from py_zipkin.zipkin import zipkin_span from py_zipkin.storage import Stack from py_zipkin.storage import SpanStorage def my_function(): context_stack = Stack() span_storage = SpanStorage() await my_function(context_stack, span_storage) async def my_function(context_stack, span_storage): with zipkin_span( service_name='my_service', span_name='some_function', transport_handler=some_handler, port=42, sample_rate=0.05, context_stack=context_stack, span_storage=span_storage, ): result = do_stuff(a, b) ``` Firehose mode [EXPERIMENTAL] ---------------------------- "Firehose mode" records 100% of the spans, regardless of sampling rate. This is useful if you want to treat these spans differently, e.g. send them to a different backend that has limited retention. It works in tandem with normal operation, however there may be additional overhead. In order to use this, you add a `firehose_handler` just like you add a `transport_handler`. This feature should be considered experimental and may be removed at any time without warning. If you do use this, be sure to send asynchronously to avoid excess overhead for every request. License ------- Copyright (c) 2018, Yelp, Inc. All Rights reserved. Apache v2