pax_global_header00006660000000000000000000000064125075764520014527gustar00rootroot0000000000000052 comment=4c8890af86ccd40369c4ccfbb41f2d46f95263d8 hiredis-py-0.2.0/000077500000000000000000000000001250757645200136035ustar00rootroot00000000000000hiredis-py-0.2.0/.gitignore000066400000000000000000000000341250757645200155700ustar00rootroot00000000000000*.pyc /build /dist MANIFEST hiredis-py-0.2.0/.gitmodules000066400000000000000000000001361250757645200157600ustar00rootroot00000000000000[submodule "vendor/hiredis"] path = vendor/hiredis url = git://github.com/redis/hiredis.git hiredis-py-0.2.0/.travis.yml000066400000000000000000000002101250757645200157050ustar00rootroot00000000000000language: python python: - "2.6" - "2.7" - "3.2" - "3.3" - "3.4" install: "python setup.py build" script: "python test.py" hiredis-py-0.2.0/CHANGELOG.md000066400000000000000000000011351250757645200154140ustar00rootroot00000000000000### 0.2.0 (2015-04-03) * Allow usage of setuptools * Upgrade to latest hiredis including basic Windows support * Expose hiredis maxbuf settings in python ### 0.1.6 (2015-01-28) * Updated with hiredis 0.12.1 — now only uses Redis parser, not entire library (#30). ### 0.1.5 * Fix memory leak when many reader instances are created (see #26). ### 0.1.4 * Allow any buffer compatible object as argument to feed (see #22). ### 0.1.3 * Allow `protocolError` and `replyError` to be any type of callable (see #21). ### 0.1.2 * Upgrade hiredis to 0.11.0 to support deeply nested multi bulk replies. hiredis-py-0.2.0/COPYING000066400000000000000000000027061250757645200146430ustar00rootroot00000000000000Copyright (c) 2011, Pieter Noordhuis All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Redis nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. hiredis-py-0.2.0/MANIFEST.in000066400000000000000000000002611250757645200153400ustar00rootroot00000000000000include COPYING include MANIFEST.in include src/*.[ch] include vendor/hiredis/COPYING include vendor/hiredis/*.[ch] exclude vendor/hiredis/example* exclude vendor/hiredis/text* hiredis-py-0.2.0/README.md000066400000000000000000000104461250757645200150670ustar00rootroot00000000000000# hiredis-py [![Build Status](https://travis-ci.org/redis/hiredis-py.svg?branch=v0.1.2)](https://travis-ci.org/redis/hiredis-py) Python extension that wraps protocol parsing code in [hiredis][hiredis]. It primarily speeds up parsing of multi bulk replies. [hiredis]: http://github.com/redis/hiredis ## Install hiredis-py is available on [PyPi](http://pypi.python.org/pypi/hiredis), and can be installed with: ``` easy_install hiredis ``` ### Requirements hiredis-py requires **Python 2.6 or higher**. Make sure Python development headers are available when installing hiredis-py. On Ubuntu/Debian systems, install them with `apt-get install python-dev`. ## Usage The `hiredis` module contains the `Reader` class. This class is responsible for parsing replies from the stream of data that is read from a Redis connection. It does not contain functionality to handle I/O. ### Reply parser The `Reader` class has two methods that are used when parsing replies from a stream of data. `Reader.feed` takes a string argument that is appended to the internal buffer. `Reader.gets` reads this buffer and returns a reply when the buffer contains a full reply. If a single call to `feed` contains multiple replies, `gets` should be called multiple times to extract all replies. Example: ```python >>> reader = hiredis.Reader() >>> reader.feed("$5\r\nhello\r\n") >>> reader.gets() 'hello' ``` When the buffer does not contain a full reply, `gets` returns `False`. This means extra data is needed and `feed` should be called again before calling `gets` again: ```python >>> reader.feed("*2\r\n$5\r\nhello\r\n") >>> reader.gets() False >>> reader.feed("$5\r\nworld\r\n") >>> reader.gets() ['hello', 'world'] ``` #### Unicode `hiredis.Reader` is able to decode bulk data to any encoding Python supports. To do so, specify the encoding you want to use for decoding replies when initializing it: ```python >>> reader = hiredis.Reader(encoding="utf-8") >>> reader.feed("$3\r\n\xe2\x98\x83\r\n") >>> reader.gets() u'☃' ``` When bulk data in a reply could not be properly decoded using the specified encoding, it will be returned as a plain string. When the encoding cannot be found, a `LookupError` will be raised after calling `gets` for the first reply with bulk data (identical to what Python's `unicode` method would do). #### Error handling When a protocol error occurs (because of multiple threads using the same socket, or some other condition that causes a corrupt stream), the error `hiredis.ProtocolError` is raised. Because the buffer is read in a lazy fashion, it will only be raised when `gets` is called and the first reply in the buffer contains an error. There is no way to recover from a faulty protocol state, so when this happens, the I/O code feeding data to `Reader` should probably reconnect. Redis can reply with error replies (`-ERR ...`). For these replies, the custom error class `hiredis.ReplyError` is returned, **but not raised**. When other error types should be used (so existing code doesn't have to change its `except` clauses), `Reader` can be initialized with the `protocolError` and `replyError` keywords. These keywords should contain a *class* that is a subclass of `Exception`. When not provided, `Reader` will use the default error types. ## Benchmarks The repository contains a benchmarking script in the `benchmark` directory, which uses [gevent](http://gevent.org/) to have non-blocking I/O and redis-py to handle connections. These benchmarks are done with a patched version of redis-py that uses hiredis-py when it is available. All benchmarks are done with 10 concurrent connections. * SET key value + GET key * redis-py: 11.76 Kops * redis-py *with* hiredis-py: 13.40 Kops * improvement: **1.1x** List entries in the following tests are 5 bytes. * LRANGE list 0 **9**: * redis-py: 4.78 Kops * redis-py *with* hiredis-py: 12.94 Kops * improvement: **2.7x** * LRANGE list 0 **99**: * redis-py: 0.73 Kops * redis-py *with* hiredis-py: 11.90 Kops * improvement: **16.3x** * LRANGE list 0 **999**: * redis-py: 0.07 Kops * redis-py *with* hiredis-py: 5.83 Kops * improvement: **83.2x** Throughput improvement for simple SET/GET is minimal, but the larger multi bulk replies get, the larger the performance improvement is. ## License This code is released under the BSD license, after the license of hiredis. hiredis-py-0.2.0/appveyor.yml000066400000000000000000000022441250757645200161750ustar00rootroot00000000000000# https://github.com/ogrisel/python-appveyor-demo/blob/master/appveyor.yml environment: global: # SDK v7.0 MSVC Express 2008's SetEnv.cmd script will fail if the # /E:ON and /V:ON options are not enabled in the batch script intepreter # See: http://stackoverflow.com/a/13751649/163740 CMD_IN_ENV: "cmd /E:ON /V:ON /C .\\appveyor\\run_with_env.cmd" matrix: - PYTHON: "C:\\Python27" PYTHON_VERSION: "2.7.8" PYTHON_ARCH: "32" - PYTHON: "C:\\Python27-x64" PYTHON_VERSION: "2.7.8" PYTHON_ARCH: "64" - PYTHON: "C:\\Python33" PYTHON_VERSION: "3.3.5" PYTHON_ARCH: "32" - PYTHON: "C:\\Python33-x64" PYTHON_VERSION: "3.3.5" PYTHON_ARCH: "64" - PYTHON: "C:\\Python34" PYTHON_VERSION: "3.4.1" PYTHON_ARCH: "32" - PYTHON: "C:\\Python34-x64" PYTHON_VERSION: "3.4.1" PYTHON_ARCH: "64" install: - "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%" - "python --version" - "python -c \"import struct; print(struct.calcsize('P') * 8)\"" - "git submodule update --init --recursive" build: off test_script: - "%CMD_IN_ENV% python setup.py build" - "%CMD_IN_ENV% python test.py" hiredis-py-0.2.0/appveyor/000077500000000000000000000000001250757645200154505ustar00rootroot00000000000000hiredis-py-0.2.0/appveyor/run_with_env.cmd000066400000000000000000000034621250757645200206510ustar00rootroot00000000000000:: To build extensions for 64 bit Python 3, we need to configure environment :: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of: :: MS Windows SDK for Windows 7 and .NET Framework 4 (SDK v7.1) :: :: To build extensions for 64 bit Python 2, we need to configure environment :: variables to use the MSVC 2008 C++ compilers from GRMSDKX_EN_DVD.iso of: :: MS Windows SDK for Windows 7 and .NET Framework 3.5 (SDK v7.0) :: :: 32 bit builds do not require specific environment configurations. :: :: Note: this script needs to be run with the /E:ON and /V:ON flags for the :: cmd interpreter, at least for (SDK v7.0) :: :: More details at: :: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows :: http://stackoverflow.com/a/13751649/163740 :: :: Author: Olivier Grisel :: License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/ @ECHO OFF SET COMMAND_TO_RUN=%* SET WIN_SDK_ROOT=C:\Program Files\Microsoft SDKs\Windows SET MAJOR_PYTHON_VERSION="%PYTHON_VERSION:~0,1%" IF %MAJOR_PYTHON_VERSION% == "2" ( SET WINDOWS_SDK_VERSION="v7.0" ) ELSE IF %MAJOR_PYTHON_VERSION% == "3" ( SET WINDOWS_SDK_VERSION="v7.1" ) ELSE ( ECHO Unsupported Python version: "%MAJOR_PYTHON_VERSION%" EXIT 1 ) IF "%PYTHON_ARCH%"=="64" ( ECHO Configuring Windows SDK %WINDOWS_SDK_VERSION% for Python %MAJOR_PYTHON_VERSION% on a 64 bit architecture SET DISTUTILS_USE_SDK=1 SET MSSdk=1 "%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Setup\WindowsSdkVer.exe" -q -version:%WINDOWS_SDK_VERSION% "%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Bin\SetEnv.cmd" /x64 /release ECHO Executing: %COMMAND_TO_RUN% call %COMMAND_TO_RUN% || EXIT 1 ) ELSE ( ECHO Using default MSVC build environment for 32 bit architecture ECHO Executing: %COMMAND_TO_RUN% call %COMMAND_TO_RUN% || EXIT 1 ) hiredis-py-0.2.0/benchmark/000077500000000000000000000000001250757645200155355ustar00rootroot00000000000000hiredis-py-0.2.0/benchmark/benchmark.py000077500000000000000000000023731250757645200200510ustar00rootroot00000000000000#!/usr/bin/env python import sys, optparse, timeit from functools import partial from redis import Redis import gevent from gevent import monkey from gevent.coros import Semaphore monkey.patch_all() parser = optparse.OptionParser() parser.add_option("-n", dest="count", metavar="COUNT", type="int", default=10000) parser.add_option("-c", dest="clients", metavar="CLIENTS", type="int", default=1), (options, args) = parser.parse_args() commands = list() for line in sys.stdin.readlines(): argv = line.strip().split() commands.append((argv[0], argv[1:])) sem = Semaphore(0) count = options.count todo = count def create_client(): global todo redis = Redis(host="localhost", port=6379, db=0) sem.acquire() while todo > 0: todo -= 1 for (cmd, args) in commands: getattr(redis, cmd)(*args) def run(clients): [sem.release() for _ in range(len(clients))] # Time how long it takes for all greenlets to finish join = partial(gevent.joinall, clients) time = timeit.timeit(join, number=1) print "%.2f Kops" % ((len(commands) * count / 1000.0) / time) # Let clients connect, and kickstart benchmark a little later clients = [gevent.spawn(create_client) for _ in range(options.clients)] let = gevent.spawn(run, clients) gevent.joinall([let]) hiredis-py-0.2.0/benchmark/lrange000066400000000000000000000000621250757645200167260ustar00rootroot00000000000000lpush list value ltrim list 0 99 lrange list 0 99 hiredis-py-0.2.0/benchmark/simple000066400000000000000000000000261250757645200167470ustar00rootroot00000000000000set key value get key hiredis-py-0.2.0/hiredis/000077500000000000000000000000001250757645200152325ustar00rootroot00000000000000hiredis-py-0.2.0/hiredis/__init__.py000066400000000000000000000002771250757645200173510ustar00rootroot00000000000000from .hiredis import Reader, HiredisError, ProtocolError, ReplyError from .version import __version__ __all__ = [ "Reader", "HiredisError", "ProtocolError", "ReplyError", "__version__"] hiredis-py-0.2.0/hiredis/version.py000066400000000000000000000000261250757645200172670ustar00rootroot00000000000000__version__ = "0.2.0" hiredis-py-0.2.0/setup.cfg000066400000000000000000000000501250757645200154170ustar00rootroot00000000000000[metadata] description-file = README.md hiredis-py-0.2.0/setup.py000077500000000000000000000057461250757645200153340ustar00rootroot00000000000000#!/usr/bin/env python try: from setuptools import setup, Extension from setuptools.command import install_lib as _install_lib except ImportError: from distutils.core import setup, Extension from distutils.command import install_lib as _install_lib import sys, imp, os, glob def version(): module = imp.load_source("hiredis.version", "hiredis/version.py") return module.__version__ # Patch "install_lib" command to run build_clib before build_ext # to properly work with easy_install. # See: http://bugs.python.org/issue5243 class install_lib(_install_lib.install_lib): def build(self): if not self.skip_build: if self.distribution.has_pure_modules(): self.run_command('build_py') if self.distribution.has_c_libraries(): self.run_command('build_clib') if self.distribution.has_ext_modules(): self.run_command('build_ext') # To link the extension with the C library, distutils passes the "-lLIBRARY" # option to the linker. This makes it go through its library search path. If it # finds a shared object of the specified library in one of the system-wide # library paths, it will dynamically link it. # # We want the linker to statically link the version of hiredis that is included # with hiredis-py. However, the linker may pick up the shared library version # of hiredis, if it is available through one of the system-wide library paths. # To prevent this from happening, we use an obfuscated library name such that # the only version the linker will be able to find is the right version. # # This is a terrible hack, but patching distutils to do the right thing for all # supported Python versions is worse... # # Also see: https://github.com/pietern/hiredis-py/issues/15 lib = ("hiredis_for_hiredis_py", { "sources": ["vendor/hiredis/%s.c" % src for src in ("read", "sds")]}) ext = Extension("hiredis.hiredis", sources=glob.glob("src/*.c"), include_dirs=["vendor"]) setup( name="hiredis", version=version(), description="Python wrapper for hiredis", url="https://github.com/redis/hiredis-py", author="Jan-Erik Rediger, Pieter Noordhuis", author_email="janerik@fnordig.de, pcnoordhuis@gmail.com", keywords=["Redis"], license="BSD", packages=["hiredis"], libraries=[lib], ext_modules=[ext], # Override "install_lib" command cmdclass={ "install_lib": install_lib }, classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: MacOS', 'Operating System :: POSIX', 'Programming Language :: C', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.6', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.2', 'Programming Language :: Python :: 3.3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: Implementation :: CPython', 'Topic :: Software Development', ], ) hiredis-py-0.2.0/src/000077500000000000000000000000001250757645200143725ustar00rootroot00000000000000hiredis-py-0.2.0/src/hiredis.c000066400000000000000000000040761250757645200161740ustar00rootroot00000000000000#include "hiredis.h" #include "reader.h" #if IS_PY3K static int hiredis_ModuleTraverse(PyObject *m, visitproc visit, void *arg) { Py_VISIT(GET_STATE(m)->HiErr_Base); Py_VISIT(GET_STATE(m)->HiErr_ProtocolError); Py_VISIT(GET_STATE(m)->HiErr_ReplyError); return 0; } static int hiredis_ModuleClear(PyObject *m) { Py_CLEAR(GET_STATE(m)->HiErr_Base); Py_CLEAR(GET_STATE(m)->HiErr_ProtocolError); Py_CLEAR(GET_STATE(m)->HiErr_ReplyError); return 0; } static struct PyModuleDef hiredis_ModuleDef = { PyModuleDef_HEAD_INIT, MOD_HIREDIS, NULL, sizeof(struct hiredis_ModuleState), /* m_size */ NULL, /* m_methods */ NULL, /* m_reload */ hiredis_ModuleTraverse, /* m_traverse */ hiredis_ModuleClear, /* m_clear */ NULL /* m_free */ }; #else struct hiredis_ModuleState state; #endif /* Keep pointer around for other classes to access the module state. */ PyObject *mod_hiredis; #if IS_PY3K PyMODINIT_FUNC PyInit_hiredis(void) #else PyMODINIT_FUNC inithiredis(void) #endif { if (PyType_Ready(&hiredis_ReaderType) < 0) { #if IS_PY3K return NULL; #else return; #endif } #if IS_PY3K mod_hiredis = PyModule_Create(&hiredis_ModuleDef); #else mod_hiredis = Py_InitModule(MOD_HIREDIS, NULL); #endif /* Setup custom exceptions */ HIREDIS_STATE->HiErr_Base = PyErr_NewException(MOD_HIREDIS ".HiredisError", PyExc_Exception, NULL); HIREDIS_STATE->HiErr_ProtocolError = PyErr_NewException(MOD_HIREDIS ".ProtocolError", HIREDIS_STATE->HiErr_Base, NULL); HIREDIS_STATE->HiErr_ReplyError = PyErr_NewException(MOD_HIREDIS ".ReplyError", HIREDIS_STATE->HiErr_Base, NULL); PyModule_AddObject(mod_hiredis, "HiredisError", HIREDIS_STATE->HiErr_Base); PyModule_AddObject(mod_hiredis, "ProtocolError", HIREDIS_STATE->HiErr_ProtocolError); PyModule_AddObject(mod_hiredis, "ReplyError", HIREDIS_STATE->HiErr_ReplyError); Py_INCREF(&hiredis_ReaderType); PyModule_AddObject(mod_hiredis, "Reader", (PyObject *)&hiredis_ReaderType); #if IS_PY3K return mod_hiredis; #endif } hiredis-py-0.2.0/src/hiredis.h000066400000000000000000000015371250757645200162000ustar00rootroot00000000000000#ifndef __HIREDIS_PY_H #define __HIREDIS_PY_H #include #include #ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ #define PyMODINIT_FUNC void #endif #if PY_MAJOR_VERSION >= 3 #define IS_PY3K 1 #endif #ifndef MOD_HIREDIS #define MOD_HIREDIS "hiredis" #endif struct hiredis_ModuleState { PyObject *HiErr_Base; PyObject *HiErr_ProtocolError; PyObject *HiErr_ReplyError; }; #if IS_PY3K #define GET_STATE(__s) ((struct hiredis_ModuleState*)PyModule_GetState(__s)) #else extern struct hiredis_ModuleState state; #define GET_STATE(__s) (&state) #endif /* Keep pointer around for other classes to access the module state. */ extern PyObject *mod_hiredis; #define HIREDIS_STATE (GET_STATE(mod_hiredis)) #ifdef IS_PY3K PyMODINIT_FUNC PyInit_hiredis(void); #else PyMODINIT_FUNC inithiredis(void); #endif #endif hiredis-py-0.2.0/src/reader.c000066400000000000000000000252611250757645200160060ustar00rootroot00000000000000#include "reader.h" #include static void Reader_dealloc(hiredis_ReaderObject *self); static int Reader_init(hiredis_ReaderObject *self, PyObject *args, PyObject *kwds); static PyObject *Reader_new(PyTypeObject *type, PyObject *args, PyObject *kwds); static PyObject *Reader_feed(hiredis_ReaderObject *self, PyObject *args); static PyObject *Reader_gets(hiredis_ReaderObject *self); static PyObject *Reader_setmaxbuf(hiredis_ReaderObject *self, PyObject *arg); static PyObject *Reader_getmaxbuf(hiredis_ReaderObject *self); static PyMethodDef hiredis_ReaderMethods[] = { {"feed", (PyCFunction)Reader_feed, METH_VARARGS, NULL }, {"gets", (PyCFunction)Reader_gets, METH_NOARGS, NULL }, {"setmaxbuf", (PyCFunction)Reader_setmaxbuf, METH_O, NULL }, {"getmaxbuf", (PyCFunction)Reader_getmaxbuf, METH_NOARGS, NULL }, { NULL } /* Sentinel */ }; PyTypeObject hiredis_ReaderType = { PyVarObject_HEAD_INIT(NULL, 0) MOD_HIREDIS ".Reader", /*tp_name*/ sizeof(hiredis_ReaderObject), /*tp_basicsize*/ 0, /*tp_itemsize*/ (destructor)Reader_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ 0, /*tp_setattr*/ 0, /*tp_compare*/ 0, /*tp_repr*/ 0, /*tp_as_number*/ 0, /*tp_as_sequence*/ 0, /*tp_as_mapping*/ 0, /*tp_hash */ 0, /*tp_call*/ 0, /*tp_str*/ 0, /*tp_getattro*/ 0, /*tp_setattro*/ 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/ "Hiredis protocol reader", /*tp_doc */ 0, /*tp_traverse */ 0, /*tp_clear */ 0, /*tp_richcompare */ 0, /*tp_weaklistoffset */ 0, /*tp_iter */ 0, /*tp_iternext */ hiredis_ReaderMethods, /*tp_methods */ 0, /*tp_members */ 0, /*tp_getset */ 0, /*tp_base */ 0, /*tp_dict */ 0, /*tp_descr_get */ 0, /*tp_descr_set */ 0, /*tp_dictoffset */ (initproc)Reader_init, /*tp_init */ 0, /*tp_alloc */ Reader_new, /*tp_new */ }; static void *tryParentize(const redisReadTask *task, PyObject *obj) { PyObject *parent; if (task && task->parent) { parent = (PyObject*)task->parent->obj; assert(PyList_CheckExact(parent)); PyList_SET_ITEM(parent, task->idx, obj); } return obj; } static PyObject *createDecodedString(hiredis_ReaderObject *self, const char *str, size_t len) { PyObject *obj; if (self->encoding == NULL) { obj = PyBytes_FromStringAndSize(str, len); } else { obj = PyUnicode_Decode(str, len, self->encoding, NULL); if (obj == NULL) { if (PyErr_ExceptionMatches(PyExc_ValueError)) { /* Ignore encoding and simply return plain string. */ obj = PyBytes_FromStringAndSize(str, len); } else { assert(PyErr_ExceptionMatches(PyExc_LookupError)); /* Store error when this is the first. */ if (self->error.ptype == NULL) PyErr_Fetch(&(self->error.ptype), &(self->error.pvalue), &(self->error.ptraceback)); /* Return Py_None as placeholder to let the error bubble up and * be used when a full reply in Reader#gets(). */ obj = Py_None; Py_INCREF(obj); } PyErr_Clear(); } } assert(obj != NULL); return obj; } static void *createError(PyObject *errorCallable, char *errstr, size_t len) { PyObject *obj; PyObject *args = Py_BuildValue("(s#)", errstr, len); assert(args != NULL); /* TODO: properly handle OOM etc */ obj = PyObject_CallObject(errorCallable, args); assert(obj != NULL); Py_DECREF(args); return obj; } static void *createStringObject(const redisReadTask *task, char *str, size_t len) { hiredis_ReaderObject *self = (hiredis_ReaderObject*)task->privdata; PyObject *obj; if (task->type == REDIS_REPLY_ERROR) { obj = createError(self->replyErrorClass, str, len); } else { obj = createDecodedString(self, str, len); } return tryParentize(task, obj); } static void *createArrayObject(const redisReadTask *task, int elements) { PyObject *obj; obj = PyList_New(elements); return tryParentize(task, obj); } static void *createIntegerObject(const redisReadTask *task, long long value) { PyObject *obj; obj = PyLong_FromLongLong(value); return tryParentize(task, obj); } static void *createNilObject(const redisReadTask *task) { PyObject *obj = Py_None; Py_INCREF(obj); return tryParentize(task, obj); } static void freeObject(void *obj) { Py_XDECREF(obj); } redisReplyObjectFunctions hiredis_ObjectFunctions = { createStringObject, // void *(*createString)(const redisReadTask*, char*, size_t); createArrayObject, // void *(*createArray)(const redisReadTask*, int); createIntegerObject, // void *(*createInteger)(const redisReadTask*, long long); createNilObject, // void *(*createNil)(const redisReadTask*); freeObject // void (*freeObject)(void*); }; static void Reader_dealloc(hiredis_ReaderObject *self) { redisReplyReaderFree(self->reader); if (self->encoding) free(self->encoding); Py_XDECREF(self->protocolErrorClass); Py_XDECREF(self->replyErrorClass); ((PyObject *)self)->ob_type->tp_free((PyObject*)self); } static int _Reader_set_exception(PyObject **target, PyObject *value) { int callable; callable = PyCallable_Check(value); if (callable == 0) { PyErr_SetString(PyExc_TypeError, "Expected a callable"); return 0; } Py_DECREF(*target); *target = value; Py_INCREF(*target); return 1; } static int Reader_init(hiredis_ReaderObject *self, PyObject *args, PyObject *kwds) { static char *kwlist[] = { "protocolError", "replyError", "encoding", NULL }; PyObject *protocolErrorClass = NULL; PyObject *replyErrorClass = NULL; PyObject *encodingObj = NULL; if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OOO", kwlist, &protocolErrorClass, &replyErrorClass, &encodingObj)) return -1; if (protocolErrorClass) if (!_Reader_set_exception(&self->protocolErrorClass, protocolErrorClass)) return -1; if (replyErrorClass) if (!_Reader_set_exception(&self->replyErrorClass, replyErrorClass)) return -1; if (encodingObj) { PyObject *encbytes; char *encstr; int enclen; if (PyUnicode_Check(encodingObj)) encbytes = PyUnicode_AsASCIIString(encodingObj); else encbytes = PyObject_Bytes(encodingObj); if (encbytes == NULL) return -1; enclen = PyBytes_Size(encbytes); encstr = PyBytes_AsString(encbytes); self->encoding = (char*)malloc(enclen+1); memcpy(self->encoding, encstr, enclen); self->encoding[enclen] = '\0'; Py_DECREF(encbytes); } return 0; } static PyObject *Reader_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { hiredis_ReaderObject *self; self = (hiredis_ReaderObject*)type->tp_alloc(type, 0); if (self != NULL) { self->reader = redisReaderCreateWithFunctions(NULL); self->reader->fn = &hiredis_ObjectFunctions; self->reader->privdata = self; self->encoding = NULL; self->protocolErrorClass = HIREDIS_STATE->HiErr_ProtocolError; self->replyErrorClass = HIREDIS_STATE->HiErr_ReplyError; Py_INCREF(self->protocolErrorClass); Py_INCREF(self->replyErrorClass); self->error.ptype = NULL; self->error.pvalue = NULL; self->error.ptraceback = NULL; } return (PyObject*)self; } static PyObject *Reader_feed(hiredis_ReaderObject *self, PyObject *args) { Py_buffer buf; Py_ssize_t off = 0; Py_ssize_t len = -1; if (!PyArg_ParseTuple(args, "s*|nn", &buf, &off, &len)) { return NULL; } if (len == -1) { len = buf.len - off; } if (off < 0 || len < 0) { PyErr_SetString(PyExc_ValueError, "negative input"); goto error; } if ((off + len) > buf.len) { PyErr_SetString(PyExc_ValueError, "input is larger than buffer size"); goto error; } redisReplyReaderFeed(self->reader, (char *)buf.buf + off, len); PyBuffer_Release(&buf); Py_RETURN_NONE; error: PyBuffer_Release(&buf); return NULL; } static PyObject *Reader_gets(hiredis_ReaderObject *self) { PyObject *obj; PyObject *err; char *errstr; if (redisReplyReaderGetReply(self->reader, (void**)&obj) == REDIS_ERR) { errstr = redisReplyReaderGetError(self->reader); /* protocolErrorClass might be a callable. call it, then use it's type */ err = createError(self->protocolErrorClass, errstr, strlen(errstr)); obj = PyObject_Type(err); PyErr_SetString(obj, errstr); Py_DECREF(obj); Py_DECREF(err); return NULL; } if (obj == NULL) { Py_RETURN_FALSE; } else { /* Restore error when there is one. */ if (self->error.ptype != NULL) { Py_DECREF(obj); PyErr_Restore(self->error.ptype, self->error.pvalue, self->error.ptraceback); self->error.ptype = NULL; self->error.pvalue = NULL; self->error.ptraceback = NULL; return NULL; } return obj; } } static PyObject *Reader_setmaxbuf(hiredis_ReaderObject *self, PyObject *arg) { long maxbuf; if (arg == Py_None) maxbuf = REDIS_READER_MAX_BUF; else { maxbuf = PyLong_AsLong(arg); if (maxbuf < 0) { if (!PyErr_Occurred()) PyErr_SetString(PyExc_ValueError, "maxbuf value out of range"); return NULL; } } self->reader->maxbuf = maxbuf; Py_INCREF(Py_None); return Py_None; } static PyObject *Reader_getmaxbuf(hiredis_ReaderObject *self) { return PyLong_FromSize_t(self->reader->maxbuf); } hiredis-py-0.2.0/src/reader.h000066400000000000000000000012111250757645200160000ustar00rootroot00000000000000#ifndef __READER_H #define __READER_H #include "hiredis.h" typedef struct { PyObject_HEAD redisReader *reader; char *encoding; PyObject *protocolErrorClass; PyObject *replyErrorClass; /* Stores error object in between incomplete calls to #gets, in order to * only set the error once a full reply has been read. Otherwise, the * reader could get in an inconsistent state. */ struct { PyObject *ptype; PyObject *pvalue; PyObject *ptraceback; } error; } hiredis_ReaderObject; extern PyTypeObject hiredis_ReaderType; extern redisReplyObjectFunctions hiredis_ObjectFunctions; #endif hiredis-py-0.2.0/test.py000077500000000000000000000002541250757645200151400ustar00rootroot00000000000000#!/usr/bin/env python from unittest import TextTestRunner import test import sys result = TextTestRunner().run(test.tests()) if not result.wasSuccessful(): sys.exit(1) hiredis-py-0.2.0/test/000077500000000000000000000000001250757645200145625ustar00rootroot00000000000000hiredis-py-0.2.0/test/__init__.py000066400000000000000000000005361250757645200166770ustar00rootroot00000000000000import glob, os.path, sys version = sys.version.split(" ")[0] majorminor = version[0:3] # Add path to hiredis.so load path path = glob.glob("build/lib*-%s/hiredis" % majorminor)[0] sys.path.insert(0, path) from unittest import * from . import reader def tests(): suite = TestSuite() suite.addTest(makeSuite(reader.ReaderTest)) return suite hiredis-py-0.2.0/test/reader.py000066400000000000000000000134411250757645200164010ustar00rootroot00000000000000# coding=utf-8 from unittest import * import hiredis import sys class ReaderTest(TestCase): def setUp(self): self.reader = hiredis.Reader() def reply(self): return self.reader.gets() def test_nothing(self): self.assertEquals(False, self.reply()) def test_error_when_feeding_non_string(self): self.assertRaises(TypeError, self.reader.feed, 1) def test_protocol_error(self): self.reader.feed(b"x") self.assertRaises(hiredis.ProtocolError, self.reply) def test_protocol_error_with_custom_class(self): self.reader = hiredis.Reader(protocolError=RuntimeError) self.reader.feed(b"x") self.assertRaises(RuntimeError, self.reply) def test_protocol_error_with_custom_callable(self): class CustomException(Exception): pass self.reader = hiredis.Reader(protocolError=lambda e: CustomException(e)) self.reader.feed(b"x") self.assertRaises(CustomException, self.reply) def test_fail_with_wrong_protocol_error_class(self): self.assertRaises(TypeError, hiredis.Reader, protocolError="wrong") def test_error_string(self): self.reader.feed(b"-error\r\n") error = self.reply() self.assertEquals(hiredis.ReplyError, type(error)) self.assertEquals(("error",), error.args) def test_error_string_with_custom_class(self): self.reader = hiredis.Reader(replyError=RuntimeError) self.reader.feed(b"-error\r\n") error = self.reply() self.assertEquals(RuntimeError, type(error)) self.assertEquals(("error",), error.args) def test_error_string_with_custom_callable(self): class CustomException(Exception): pass self.reader = hiredis.Reader(replyError=lambda e: CustomException(e)) self.reader.feed(b"-error\r\n") error = self.reply() self.assertEquals(CustomException, type(error)) self.assertEquals(("error",), error.args) def test_fail_with_wrong_reply_error_class(self): self.assertRaises(TypeError, hiredis.Reader, replyError="wrong") def test_errors_in_nested_multi_bulk(self): self.reader.feed(b"*2\r\n-err0\r\n-err1\r\n") for r, error in zip(("err0", "err1"), self.reply()): self.assertEquals(hiredis.ReplyError, type(error)) self.assertEquals((r,), error.args) def test_integer(self): value = 2**63-1 # Largest 64-bit signed integer self.reader.feed((":%d\r\n" % value).encode("ascii")) self.assertEquals(value, self.reply()) def test_status_string(self): self.reader.feed(b"+ok\r\n") self.assertEquals(b"ok", self.reply()) def test_empty_bulk_string(self): self.reader.feed(b"$0\r\n\r\n") self.assertEquals(b"", self.reply()) def test_bulk_string(self): self.reader.feed(b"$5\r\nhello\r\n") self.assertEquals(b"hello", self.reply()) def test_bulk_string_without_encoding(self): snowman = b"\xe2\x98\x83" self.reader.feed(b"$3\r\n" + snowman + b"\r\n") self.assertEquals(snowman, self.reply()) def test_bulk_string_with_encoding(self): snowman = b"\xe2\x98\x83" self.reader = hiredis.Reader(encoding="utf-8") self.reader.feed(b"$3\r\n" + snowman + b"\r\n") self.assertEquals(snowman.decode("utf-8"), self.reply()) def test_bulk_string_with_other_encoding(self): snowman = b"\xe2\x98\x83" self.reader = hiredis.Reader(encoding="utf-32") self.reader.feed(b"$3\r\n" + snowman + b"\r\n") self.assertEquals(snowman, self.reply()) def test_bulk_string_with_invalid_encoding(self): self.reader = hiredis.Reader(encoding="unknown") self.reader.feed(b"$5\r\nhello\r\n") self.assertRaises(LookupError, self.reply) def test_null_multi_bulk(self): self.reader.feed(b"*-1\r\n") self.assertEquals(None, self.reply()) def test_empty_multi_bulk(self): self.reader.feed(b"*0\r\n") self.assertEquals([], self.reply()) def test_multi_bulk(self): self.reader.feed(b"*2\r\n$5\r\nhello\r\n$5\r\nworld\r\n") self.assertEquals([b"hello", b"world"], self.reply()) def test_multi_bulk_with_invalid_encoding_and_partial_reply(self): self.reader = hiredis.Reader(encoding="unknown") self.reader.feed(b"*2\r\n$5\r\nhello\r\n") self.assertEquals(False, self.reply()) self.reader.feed(b":1\r\n") self.assertRaises(LookupError, self.reply) def test_nested_multi_bulk(self): self.reader.feed(b"*2\r\n*2\r\n$5\r\nhello\r\n$5\r\nworld\r\n$1\r\n!\r\n") self.assertEquals([[b"hello", b"world"], b"!"], self.reply()) def test_nested_multi_bulk_depth(self): self.reader.feed(b"*1\r\n*1\r\n*1\r\n*1\r\n$1\r\n!\r\n") self.assertEquals([[[[b"!"]]]], self.reply()) def test_subclassable(self): class TestReader(hiredis.Reader): def __init__(self, *args, **kwargs): super(TestReader, self).__init__(*args, **kwargs) reader = TestReader() reader.feed(b"+ok\r\n") self.assertEquals(b"ok", reader.gets()) def test_invalid_offset(self): data = b"+ok\r\n" self.assertRaises(ValueError, self.reader.feed, data, 6) def test_invalid_length(self): data = b"+ok\r\n" self.assertRaises(ValueError, self.reader.feed, data, 0, 6) def test_ok_offset(self): data = b"blah+ok\r\n" self.reader.feed(data, 4) self.assertEquals(b"ok", self.reply()) def test_ok_length(self): data = b"blah+ok\r\n" self.reader.feed(data, 4, len(data)-4) self.assertEquals(b"ok", self.reply()) def test_feed_bytearray(self): if sys.hexversion >= 0x02060000: self.reader.feed(bytearray(b"+ok\r\n")) self.assertEquals(b"ok", self.reply()) def test_maxbuf(self): defaultmaxbuf = self.reader.getmaxbuf() self.reader.setmaxbuf(0) self.assertEquals(0, self.reader.getmaxbuf()) self.reader.setmaxbuf(10000) self.assertEquals(10000, self.reader.getmaxbuf()) self.reader.setmaxbuf(None) self.assertEquals(defaultmaxbuf, self.reader.getmaxbuf()) self.assertRaises(ValueError, self.reader.setmaxbuf, -4) hiredis-py-0.2.0/vendor/000077500000000000000000000000001250757645200151005ustar00rootroot00000000000000hiredis-py-0.2.0/vendor/hiredis/000077500000000000000000000000001250757645200165275ustar00rootroot00000000000000