pax_global_header00006660000000000000000000000064123047172050014513gustar00rootroot0000000000000052 comment=c905774b66120879b7e5cbf11c7518e784d15975 beanstalkc-0.4.0/000077500000000000000000000000001230471720500136235ustar00rootroot00000000000000beanstalkc-0.4.0/.nose.cfg000066400000000000000000000001311230471720500153210ustar00rootroot00000000000000[nosetests] verbosity=3 with-doctest=1 doctest-extension=mkd doctest-fixtures=_fixtures beanstalkc-0.4.0/.travis-requirements.txt000066400000000000000000000000071230471720500204700ustar00rootroot00000000000000pyyaml beanstalkc-0.4.0/.travis.yml000066400000000000000000000010171230471720500157330ustar00rootroot00000000000000language: python python: - "2.6" - "2.7" env: global: - BEANSTALKD=./beanstalkd install: # Install most recent beanstalkd from source - wget https://github.com/downloads/kr/beanstalkd/beanstalkd-1.8.tar.gz - tar xf beanstalkd-1.8.tar.gz - make -C beanstalkd-1.8/ - mv beanstalkd-1.8/beanstalkd . # Install Python dependencies. - pip install -r .travis-requirements.txt --use-mirrors script: nosetests -c .nose.cfg branches: only: - master - wip-travisci beanstalkc-0.4.0/LICENSE000066400000000000000000000261361230471720500146400ustar00rootroot00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. beanstalkc-0.4.0/README.mkd000066400000000000000000000020461230471720500152570ustar00rootroot00000000000000beanstalkc ========== beanstalkc is a simple beanstalkd client library for Python. [beanstalkd][] is a fast, distributed, in-memory workqueue service. beanstalkc depends on [PyYAML][], but there are ways to avoid this dependency. See Appendix A of the tutorial for details. beanstalkc is pure Python, and is compatible with [eventlet][] and [gevent][]. [beanstalkd]: http://kr.github.com/beanstalkd/ [eventlet]: http://eventlet.net/ [gevent]: http://www.gevent.org/ [PyYAML]: http://pyyaml.org/ Usage ----- Here is a short example, to illustrate the flavor of beanstalkc: >>> import beanstalkc >>> beanstalk = beanstalkc.Connection(host='localhost', port=14711) >>> beanstalk.put('hey!') 1 >>> job = beanstalk.reserve() >>> job.body 'hey!' >>> job.delete() For more information, see [the tutorial](TUTORIAL.mkd), which will explain most everything. License ------- Copyright (C) 2008-2014 Andreas Bolka, Licensed under the [Apache License, Version 2.0][license]. [license]: http://www.apache.org/licenses/LICENSE-2.0 beanstalkc-0.4.0/README_fixtures.py000066400000000000000000000000521230471720500170600ustar00rootroot00000000000000from test.fixtures import setup, teardown beanstalkc-0.4.0/TUTORIAL.mkd000066400000000000000000000347501230471720500155740ustar00rootroot00000000000000beanstalkc Tutorial =================== Welcome, dear stranger, to a tour de force through beanstalkd's capabilities. Say hello to your fellow travel companion, the beanstalkc client library for Python. You'll get to know each other fairly well during this trip, so better start off on a friendly note. And now, let's go! Getting Started --------------- You'll need beanstalkd listening at port 14711 to follow along. So simply start it using: `beanstalkd -l 127.0.0.1 -p 14711` Besides having beanstalkc installed, you'll typically also need PyYAML. If you insist, you can also use beanstalkc without PyYAML. For more details see Appendix A of this tutorial. To use beanstalkc we have to import the library and set up a connection to an (already running) beanstalkd server: >>> import beanstalkc >>> beanstalk = beanstalkc.Connection(host='localhost', port=14711) If we leave out the `host` and/or `port` parameters, `'localhost'` and `11300` would be used as defaults, respectively. There is also a `connect_timeout` parameter which determines how long, in seconds, the socket will wait for the server to respond to its initial connection attempt. If it is `None`, then there will be no timeout; it defaults to the result of your system's `socket.getdefaulttimeout()`. Basic Operation --------------- Now that we have a connection set up, we can enqueue jobs: >>> beanstalk.put('hey!') 1 Or we can request jobs: >>> job = beanstalk.reserve() >>> job.body 'hey!' Once we are done with processing a job, we have to mark it as done, otherwise jobs are re-queued by beanstalkd after a "time to run" (120 seconds, per default) is surpassed. A job is marked as done, by calling `delete`: >>> job.delete() `reserve` blocks until a job is ready, possibly forever. If that is not desired, we can invoke `reserve` with a timeout (in seconds) how long we want to wait to receive a job. If such a `reserve` times out, it will return `None`: >>> beanstalk.reserve(timeout=0) is None True If you use a timeout of 0, `reserve` will immediately return either a job or `None`. Note that beanstalkc requires job bodies to be strings, conversion to/from strings is left up to you: >>> beanstalk.put(42) Traceback (most recent call last): ... AssertionError: Job body must be a str instance There is no restriction on what characters you can put in a job body, so they can be used to hold arbitrary binary data. If you want to send images, just `put` the image data as a string. If you want to send Unicode text, just use `unicode.encode` to convert it to a string with some encoding. Tube Management --------------- A single beanstalkd server can provide many different queues, called "tubes" in beanstalkd. To see all available tubes: >>> beanstalk.tubes() ['default'] A beanstalkd client can choose one tube into which its job are put. This is the tube "used" by the client. To see what tube you are currently using: >>> beanstalk.using() 'default' Unless told otherwise, a client uses the 'default' tube. If you want to use a different tube: >>> beanstalk.use('foo') 'foo' >>> beanstalk.using() 'foo' If you decide to use a tube, that does not yet exist, the tube is automatically created by beanstalkd: >>> beanstalk.tubes() ['default', 'foo'] Of course, you can always switch back to the default tube. Tubes that don't have any client using or watching, vanish automatically: >>> beanstalk.use('default') 'default' >>> beanstalk.using() 'default' >>> beanstalk.tubes() ['default'] Further, a beanstalkd client can choose many tubes to reserve jobs from. These tubes are "watched" by the client. To see what tubes you are currently watching: >>> beanstalk.watching() ['default'] To watch an additional tube: >>> beanstalk.watch('bar') 2 >>> beanstalk.watching() ['default', 'bar'] As before, tubes that do not yet exist are created automatically once you start watching them: >>> beanstalk.tubes() ['default', 'bar'] To stop watching a tube: >>> beanstalk.ignore('bar') 1 >>> beanstalk.watching() ['default'] You can't watch zero tubes. So if you try to ignore the last tube you are watching, this is silently ignored: >>> beanstalk.ignore('default') 1 >>> beanstalk.watching() ['default'] To recap: each beanstalkd client manages two separate concerns: which tube newly created jobs are put into, and which tube(s) jobs are reserved from. Accordingly, there are two separate sets of functions for these concerns: - `use` and `using` affect where jobs are `put`; - `watch` and `watching` control where jobs are `reserve`d from. Note that these concerns are fully orthogonal: for example, when you `use` a tube, it is not automatically `watch`ed. Neither does `watch`ing a tube affect the tube you are `using`. Statistics ---------- Beanstalkd accumulates various statistics at the server, tube and job level. Statistical details for a job can only be retrieved during the job's lifecycle. So let's create another job: >>> beanstalk.put('ho?') 2 >>> job = beanstalk.reserve() Now we retrieve job-level statistics: >>> from pprint import pprint >>> pprint(job.stats()) # doctest: +ELLIPSIS {'age': 0, ... 'id': 2, ... 'state': 'reserved', ... 'tube': 'default'} If you try to access job stats after the job was deleted, you'll get a `CommandFailed` exception: >>> job.delete() >>> job.stats() # doctest: +ELLIPSIS Traceback (most recent call last): ... CommandFailed: ('stats-job', 'NOT_FOUND', []) Let's have a look at some numbers for the `'default'` tube: >>> pprint(beanstalk.stats_tube('default')) # doctest: +ELLIPSIS {... 'current-jobs-ready': 0, 'current-jobs-reserved': 0, 'current-jobs-urgent': 0, ... 'name': 'default', ...} Finally, there's an abundant amount of server-level statistics accessible via the `Connection`'s `stats` method. We won't go into details here, but: >>> pprint(beanstalk.stats()) # doctest: +ELLIPSIS {... 'current-connections': 1, 'current-jobs-buried': 0, 'current-jobs-delayed': 0, 'current-jobs-ready': 0, 'current-jobs-reserved': 0, 'current-jobs-urgent': 0, ...} Advanced Operation ------------------ In "Basic Operation" above, we discussed the typical lifecycle of a job: put reserve delete -----> [READY] ---------> [RESERVED] --------> *poof* (This picture was taken from beanstalkd's protocol documentation. It is originally contained in `protocol.txt`, part of the beanstalkd distribution.) #doctest:+SKIP But besides `ready` and `reserved`, a job can also be `delayed` or `buried`. Along with those states come a few transitions, so the full picture looks like the following: put with delay release with delay ----------------> [DELAYED] <------------. | | | (time passes) | | | put v reserve | delete -----------------> [READY] ---------> [RESERVED] --------> *poof* ^ ^ | | | \ release | | | `-------------' | | | | kick | | | | bury | [BURIED] <---------------' | | delete `--------> *poof* (This picture was taken from beanstalkd's protocol documentation. It is originally contained in `protocol.txt`, part of the beanstalkd distribution.) #doctest:+SKIP Now let's have a practical look at those new possibilities. For a start, we can create a job with a delay. Such a job will only be available for reservation once this delay passes: >>> beanstalk.put('yes!', delay=1) 3 >>> beanstalk.reserve(timeout=0) is None True >>> job = beanstalk.reserve(timeout=1) >>> job.body 'yes!' If we are not interested in a job anymore (e.g. after we failed to process it), we can simply release the job back to the tube it came from: >>> job.release() >>> job.stats()['state'] 'ready' Want to get rid of a job? Well, just "bury" it! A buried job is put aside and is therefore not available for reservation anymore: >>> job = beanstalk.reserve() >>> job.bury() >>> job.stats()['state'] 'buried' >>> beanstalk.reserve(timeout=0) is None True Buried jobs are maintained in a special FIFO-queue outside of the normal job processing lifecycle until they are kicked alive again: >>> beanstalk.kick() 1 You can request many jobs to be kicked alive at once, `kick`'s return value will tell you how many jobs were actually kicked alive again: >>> beanstalk.kick(42) 0 If you still have a handle to a job (or know its job ID), you can also kick alive this particular job, overriding the FIFO operation of the burial queue: >>> job = beanstalk.reserve() >>> job.bury() >>> job.stats()['state'] 'buried' >>> job.kick() >>> job.stats()['state'] 'ready' (NOTE: The `kick-job` command was introduced in beanstalkd v1.8.) Inspecting Jobs --------------- Besides reserving jobs, a client can also "peek" at jobs. This allows to inspect jobs without modifying their state. If you know the `id` of a job you're interested, you can directly peek at the job. We still have job #3 hanging around from our previous examples, so: >>> job = beanstalk.peek(3) >>> job.body 'yes!' Note that this peek did *not* reserve the job: >>> job.stats()['state'] 'ready' If you try to peek at a non-existing job, you'll simply see nothing: >>> beanstalk.peek(42) is None True If you are not interested in a particular job, but want to see jobs according to their state, beanstalkd also provides a few commands. To peek at the same job that would be returned by `reserve` -- the next ready job -- use `peek-ready`: >>> job = beanstalk.peek_ready() >>> job.body 'yes!' Note that you can't release, or bury a job that was not reserved by you. Those requests on unreserved jobs are silently ignored: >>> job.release() >>> job.bury() >>> job.stats()['state'] 'ready' You can, though, delete a job that was not reserved by you: >>> job.delete() >>> job.stats() # doctest: +ELLIPSIS Traceback (most recent call last): ... CommandFailed: ('stats-job', 'NOT_FOUND', []) Finally, you can also peek into the special queues for jobs that are delayed: >>> beanstalk.put('o tempores', delay=120) 4 >>> job = beanstalk.peek_delayed() >>> job.stats()['state'] 'delayed' ... or buried: >>> beanstalk.put('o mores!') 5 >>> job = beanstalk.reserve() >>> job.bury() >>> job = beanstalk.peek_buried() >>> job.stats()['state'] 'buried' Job Priorities -------------- Without job priorities, beanstalkd operates as a FIFO queue: >>> _ = beanstalk.put('1') >>> _ = beanstalk.put('2') >>> job = beanstalk.reserve() ; print job.body ; job.delete() 1 >>> job = beanstalk.reserve() ; print job.body ; job.delete() 2 If need arises, you can override this behaviour by giving different jobs different priorities. There are three hard facts to know about job priorities: 1. Jobs with lower priority numbers are reserved before jobs with higher priority numbers. 2. beanstalkd priorities are 32-bit unsigned integers (they range from 0 to 2**32 - 1). 3. beanstalkc uses 2**31 as default job priority (`beanstalkc.DEFAULT_PRIORITY`). To create a job with a custom priority, use the keyword-argument `priority`: >>> _ = beanstalk.put('foo', priority=42) >>> _ = beanstalk.put('bar', priority=21) >>> _ = beanstalk.put('qux', priority=21) >>> job = beanstalk.reserve() ; print job.body ; job.delete() bar >>> job = beanstalk.reserve() ; print job.body ; job.delete() qux >>> job = beanstalk.reserve() ; print job.body ; job.delete() foo Note how `'bar'` and `'qux'` left the queue before `'foo'`, even though they were enqueued well after `'foo'`. Note also that within the same priority jobs are still handled in a FIFO manner. Fin! ---- >>> beanstalk.close() That's it, for now. We've left a few capabilities untouched (touch and time-to-run). But if you've really read through all of the above, send me a message and tell me what you think of it. And then go get yourself a treat. You certainly deserve it. Appendix A: beanstalkc and YAML ------------------------------- As beanstalkd uses YAML for diagnostic information (like the results of `stats()` or `tubes()`), you'll typically need [PyYAML](). Depending on your performance needs, you may want to supplement that with the [libyaml]() C extension. [PyYAML]: http://pyyaml.org/ [libyaml]: http://pyyaml.org/wiki/LibYAML If, for whatever reason, you cannot use PyYAML, you can still use beanstalkc and just leave the YAML responses unparsed. To do that, pass `parse_yaml=False` when creating the `Connection`: >>> beanstalk = beanstalkc.Connection(host='localhost', ... port=14711, ... parse_yaml=False) >>> beanstalk.tubes() '---\n- default\n' >>> beanstalk.stats_tube('default') # doctest: +ELLIPSIS '---\nname: default\ncurrent-jobs-urgent: 0\ncurrent-jobs-ready: 0\n...' >>> beanstalk.close() This possibility is mostly useful if you don't use the introspective capabilities of beanstalkd (`Connection#tubes`, `Connection#watching`, `Connection#stats`, `Connection#stats_tube`, and `Job#stats`). Alternatively, you can also pass a function to be used as YAML parser: >>> beanstalk = beanstalkc.Connection(host='localhost', ... port=14711, ... parse_yaml=lambda x: x.split('\n')) >>> beanstalk.tubes() ['---', '- default', ''] >>> beanstalk.stats_tube('default') # doctest: +ELLIPSIS ['---', 'name: default', 'current-jobs-urgent: 0', ...] >>> beanstalk.close() This should come in handy if PyYAML simply does not fit your needs. beanstalkc-0.4.0/TUTORIAL_fixtures.py000066400000000000000000000000521230471720500173660ustar00rootroot00000000000000from test.fixtures import setup, teardown beanstalkc-0.4.0/beanstalkc.py000077500000000000000000000244751230471720500163230ustar00rootroot00000000000000#!/usr/bin/env python """beanstalkc - A beanstalkd Client Library for Python""" __license__ = ''' Copyright (C) 2008-2014 Andreas Bolka Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ''' __version__ = '0.4.0' import logging import socket DEFAULT_HOST = 'localhost' DEFAULT_PORT = 11300 DEFAULT_PRIORITY = 2 ** 31 DEFAULT_TTR = 120 class BeanstalkcException(Exception): pass class UnexpectedResponse(BeanstalkcException): pass class CommandFailed(BeanstalkcException): pass class DeadlineSoon(BeanstalkcException): pass class SocketError(BeanstalkcException): @staticmethod def wrap(wrapped_function, *args, **kwargs): try: return wrapped_function(*args, **kwargs) except socket.error, err: raise SocketError(err) class Connection(object): def __init__(self, host=DEFAULT_HOST, port=DEFAULT_PORT, parse_yaml=True, connect_timeout=socket.getdefaulttimeout()): if parse_yaml is True: try: parse_yaml = __import__('yaml').load except ImportError: logging.error('Failed to load PyYAML, will not parse YAML') parse_yaml = False self._connect_timeout = connect_timeout self._parse_yaml = parse_yaml or (lambda x: x) self.host = host self.port = port self.connect() def connect(self): """Connect to beanstalkd server.""" self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self._socket.settimeout(self._connect_timeout) SocketError.wrap(self._socket.connect, (self.host, self.port)) self._socket.settimeout(None) self._socket_file = self._socket.makefile('rb') def close(self): """Close connection to server.""" try: self._socket.sendall('quit\r\n') except socket.error: pass try: self._socket.close() except socket.error: pass def reconnect(self): """Re-connect to server.""" self.close() self.connect() def _interact(self, command, expected_ok, expected_err=[]): SocketError.wrap(self._socket.sendall, command) status, results = self._read_response() if status in expected_ok: return results elif status in expected_err: raise CommandFailed(command.split()[0], status, results) else: raise UnexpectedResponse(command.split()[0], status, results) def _read_response(self): line = SocketError.wrap(self._socket_file.readline) if not line: raise SocketError() response = line.split() return response[0], response[1:] def _read_body(self, size): body = SocketError.wrap(self._socket_file.read, size) SocketError.wrap(self._socket_file.read, 2) # trailing crlf if size > 0 and not body: raise SocketError() return body def _interact_value(self, command, expected_ok, expected_err=[]): return self._interact(command, expected_ok, expected_err)[0] def _interact_job(self, command, expected_ok, expected_err, reserved=True): jid, size = self._interact(command, expected_ok, expected_err) body = self._read_body(int(size)) return Job(self, int(jid), body, reserved) def _interact_yaml(self, command, expected_ok, expected_err=[]): size, = self._interact(command, expected_ok, expected_err) body = self._read_body(int(size)) return self._parse_yaml(body) def _interact_peek(self, command): try: return self._interact_job(command, ['FOUND'], ['NOT_FOUND'], False) except CommandFailed, (_, _status, _results): return None # -- public interface -- def put(self, body, priority=DEFAULT_PRIORITY, delay=0, ttr=DEFAULT_TTR): """Put a job into the current tube. Returns job id.""" assert isinstance(body, str), 'Job body must be a str instance' jid = self._interact_value( 'put %d %d %d %d\r\n%s\r\n' % (priority, delay, ttr, len(body), body), ['INSERTED'], ['JOB_TOO_BIG','BURIED','DRAINING']) return int(jid) def reserve(self, timeout=None): """Reserve a job from one of the watched tubes, with optional timeout in seconds. Returns a Job object, or None if the request times out.""" if timeout is not None: command = 'reserve-with-timeout %d\r\n' % timeout else: command = 'reserve\r\n' try: return self._interact_job(command, ['RESERVED'], ['DEADLINE_SOON', 'TIMED_OUT']) except CommandFailed, (_, status, results): if status == 'TIMED_OUT': return None elif status == 'DEADLINE_SOON': raise DeadlineSoon(results) def kick(self, bound=1): """Kick at most bound jobs into the ready queue.""" return int(self._interact_value('kick %d\r\n' % bound, ['KICKED'])) def kick_job(self, jid): """Kick a specific job into the ready queue.""" self._interact('kick-job %d\r\n' % jid, ['KICKED'], ['NOT_FOUND']) def peek(self, jid): """Peek at a job. Returns a Job, or None.""" return self._interact_peek('peek %d\r\n' % jid) def peek_ready(self): """Peek at next ready job. Returns a Job, or None.""" return self._interact_peek('peek-ready\r\n') def peek_delayed(self): """Peek at next delayed job. Returns a Job, or None.""" return self._interact_peek('peek-delayed\r\n') def peek_buried(self): """Peek at next buried job. Returns a Job, or None.""" return self._interact_peek('peek-buried\r\n') def tubes(self): """Return a list of all existing tubes.""" return self._interact_yaml('list-tubes\r\n', ['OK']) def using(self): """Return the tube currently being used.""" return self._interact_value('list-tube-used\r\n', ['USING']) def use(self, name): """Use a given tube.""" return self._interact_value('use %s\r\n' % name, ['USING']) def watching(self): """Return a list of all tubes being watched.""" return self._interact_yaml('list-tubes-watched\r\n', ['OK']) def watch(self, name): """Watch a given tube.""" return int(self._interact_value('watch %s\r\n' % name, ['WATCHING'])) def ignore(self, name): """Stop watching a given tube.""" try: return int(self._interact_value('ignore %s\r\n' % name, ['WATCHING'], ['NOT_IGNORED'])) except CommandFailed: return 1 def stats(self): """Return a dict of beanstalkd statistics.""" return self._interact_yaml('stats\r\n', ['OK']) def stats_tube(self, name): """Return a dict of stats about a given tube.""" return self._interact_yaml('stats-tube %s\r\n' % name, ['OK'], ['NOT_FOUND']) def pause_tube(self, name, delay): """Pause a tube for a given delay time, in seconds.""" self._interact('pause-tube %s %d\r\n' % (name, delay), ['PAUSED'], ['NOT_FOUND']) # -- job interactors -- def delete(self, jid): """Delete a job, by job id.""" self._interact('delete %d\r\n' % jid, ['DELETED'], ['NOT_FOUND']) def release(self, jid, priority=DEFAULT_PRIORITY, delay=0): """Release a reserved job back into the ready queue.""" self._interact('release %d %d %d\r\n' % (jid, priority, delay), ['RELEASED', 'BURIED'], ['NOT_FOUND']) def bury(self, jid, priority=DEFAULT_PRIORITY): """Bury a job, by job id.""" self._interact('bury %d %d\r\n' % (jid, priority), ['BURIED'], ['NOT_FOUND']) def touch(self, jid): """Touch a job, by job id, requesting more time to work on a reserved job before it expires.""" self._interact('touch %d\r\n' % jid, ['TOUCHED'], ['NOT_FOUND']) def stats_job(self, jid): """Return a dict of stats about a job, by job id.""" return self._interact_yaml('stats-job %d\r\n' % jid, ['OK'], ['NOT_FOUND']) class Job(object): def __init__(self, conn, jid, body, reserved=True): self.conn = conn self.jid = jid self.body = body self.reserved = reserved def _priority(self): stats = self.stats() if isinstance(stats, dict): return stats['pri'] return DEFAULT_PRIORITY # -- public interface -- def delete(self): """Delete this job.""" self.conn.delete(self.jid) self.reserved = False def release(self, priority=None, delay=0): """Release this job back into the ready queue.""" if self.reserved: self.conn.release(self.jid, priority or self._priority(), delay) self.reserved = False def bury(self, priority=None): """Bury this job.""" if self.reserved: self.conn.bury(self.jid, priority or self._priority()) self.reserved = False def kick(self): """Kick this job alive.""" self.conn.kick_job(self.jid) def touch(self): """Touch this reserved job, requesting more time to work on it before it expires.""" if self.reserved: self.conn.touch(self.jid) def stats(self): """Return a dict of stats about this job.""" return self.conn.stats_job(self.jid) if __name__ == '__main__': import nose nose.main(argv=['nosetests', '-c', '.nose.cfg']) beanstalkc-0.4.0/setup.py000077500000000000000000000017451230471720500153470ustar00rootroot00000000000000#!/usr/bin/env python import os from setuptools import setup from beanstalkc import __version__ as src_version PKG_VERSION = os.environ.get('BEANSTALKC_PKG_VERSION', src_version) setup( name='beanstalkc', version=PKG_VERSION, py_modules=['beanstalkc'], author='Andreas Bolka', author_email='a@bolka.at', description='A simple beanstalkd client library', long_description=''' beanstalkc is a simple beanstalkd client library for Python. `beanstalkd `_ is a fast, distributed, in-memory workqueue service. ''', url='http://github.com/earl/beanstalkc', license='Apache License, Version 2.0', classifiers=[ 'Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'License :: OSI Approved :: Apache Software License', 'Programming Language :: Python', 'Operating System :: OS Independent', 'Topic :: Software Development :: Libraries :: Python Modules', ], ) beanstalkc-0.4.0/test/000077500000000000000000000000001230471720500146025ustar00rootroot00000000000000beanstalkc-0.4.0/test/__init__.py000066400000000000000000000000001230471720500167010ustar00rootroot00000000000000beanstalkc-0.4.0/test/fixtures.py000066400000000000000000000006241230471720500170270ustar00rootroot00000000000000import os, signal, time _BEANSTALKD_PID = None def setup(module): beanstalkd = os.getenv('BEANSTALKD', 'beanstalkd') module._BEANSTALKD_PID = os.spawnlp( os.P_NOWAIT, beanstalkd, beanstalkd, '-l', '127.0.0.1', '-p', '14711') time.sleep(0.5) # Give beanstalkd some time to ready. def teardown(module): os.kill(module._BEANSTALKD_PID, signal.SIGTERM) beanstalkc-0.4.0/test/no-yaml.mkd000066400000000000000000000025151230471720500166560ustar00rootroot00000000000000Regression tests for YAMLless operation ======================================= Job priorities are not preserved -------------------------------- Setup a connection that won't parse YAML: >>> import beanstalkc >>> beanstalk = beanstalkc.Connection(host='localhost', port=14711, ... parse_yaml=False) Observe that YAML is not parsed: >>> isinstance(beanstalk.stats(), str) True Note that while Job#release and Job#bury will still work, they won't automatically maintain the released/buried Job's priority: >>> jid = beanstalk.put('foo', priority=42) >>> job = beanstalk.reserve() >>> print repr(job.stats()) # doctest: +ELLIPSIS '...pri: 42...' >>> job.release() # Succeeds, but ... >>> job = beanstalk.reserve() # ... may not do what you want. >>> print repr(job.stats()) # doctest: +ELLIPSIS '...pri: 2147483648...' >>> job.delete() Same for Job#bury: >>> jid = beanstalk.put('bar', priority=42) >>> job = beanstalk.reserve() >>> print repr(job.stats()) # doctest: +ELLIPSIS '...pri: 42...' >>> job.bury() >>> print repr(beanstalk.stats_job(jid)) # doctest: +ELLIPSIS '...pri: 2147483648...' And that was that. >>> beanstalk.close() beanstalkc-0.4.0/test/no-yaml_fixtures.py000066400000000000000000000000451230471720500204600ustar00rootroot00000000000000from fixtures import setup, teardown