django-celery-2.5.5/0000755000076500000240000000000011744004562014670 5ustar asksolstaff00000000000000django-celery-2.5.5/AUTHORS0000644000076500000240000000350111741563541015743 0ustar asksolstaff00000000000000========= AUTHORS ========= :order: sorted Aaron Ross Adam Endicott Andrew Frankel Andrew Watts Andrii Kostenko Ask Solem Augusto Becciu Ben Firshman Brad Jasper Brian Rosner Christopher Grebs Darjus Loktevic David Ziegler Donald Stufft Eldon Stegall Felix Berger Glenn Washburn Greg Taylor Grégoire Cachet Hari Ionel Maries Cristian Jannis Leidel Jason Baker Jeff Balogh Jerzy Kozera Jesper Noehr John Andrews John Watson Jonas Haag Jonatan Heyman Josh Drake José Moreira Jude Nagurney Mark Hellewell Mark Lavin Mark Stover Maxim Bodyansky Michael Elsdoerfer Mikhail Korobov Olivier Tabone Patrick Altman Piotr Sikora Reza Lotun Rune Halvorsen Sam Cooke Sean Creeley Serj Zavadsky Timo Sugliani Vincent Driessen Vitaly Babiy Wes Turner Wes Winham WoLpH django-celery-2.5.5/bin/0000755000076500000240000000000011744004562015440 5ustar asksolstaff00000000000000django-celery-2.5.5/bin/djcelerymon0000755000076500000240000000016211656051573017706 0ustar asksolstaff00000000000000#!/usr/bin/env python from djcelery import mon def main(): mon.main() if __name__ == "__main__": main() django-celery-2.5.5/Changelog0000644000076500000240000003031311744004456016504 0ustar asksolstaff00000000000000================ Change history ================ .. contents:: :local: .. _version-2.5.5: 2.5.5 ===== :release-date: 2012-04-19 01:46 P.M BST * Fixed bug where task modules were not imported. .. _version-2.5.4: 2.5.4 ===== :release-date: 2012-04-16 06:31 P.M BST * Compatibility with celery 2.5.3 * Database scheduler now imports ``exchange``, ``routing_key`` and ``queue`` options from ``CELERYBEAT_SCHEDULE``. .. _version-2.5.3: 2.5.3 ===== :release-date: 2012-04-13 06:16 P.M BST :by: Ask Solem * 2.5.2 release broke installation because of an import in the package. Fixed by not having setup.py import the djcelery module anymore, but rather parsing the package file for metadata. .. _version-2.5.2: 2.5.2 ===== :release-date: 2012-04-13 05:00 P.M BST :by: Ask Solem .. _v251-news: * PeriodicTask admin now lists the enabled field in the list view Contributed by Gabe Jackson. .. _v251-fixes: * Fixed a compatibility issue with Django < 1.3 Fix contributed by Roman Barczyski * Admin monitor now properly escapes args and kwargs. Fix contributed by Serj Zavadsky * PeriodicTask admin now gives error if no schedule set (or both set) (Issue #126). * examples/demoproject has been updated to use the Django 1.4 template. * Database connection is no longer closed for eager tasks (Issue #116). Fix contributed by Mark Lavin. * The first-steps document for django-celery has been moved to the main Celery documentation. * djcelerymon command no longer worked properly, this has now been fixed (Issue #123). .. _version-2.5.1: 2.5.1 ===== :release-date: 2012-03-01 01:00 P.M GMT :by: Ask Solem .. _v251-fixes: Fixes ----- * Now depends on Celery 2.5.1 * Fixed problem with recursive imports when USE_I18N was enabled (Issue #109). * The ``CELERY_DB_REUSE_MAX`` setting was not honored. * The djcelerymon command no longer runs with DEBUG. To enable debug you can set the :envvar:`DJCELERYMON_DEBUG` environment variable. * Fixed eventlet/gevent compatability with Django 1.4's new thread sharing detection. * Now depends on django-picklefield 0.2.0 or greater. Previous versions would not work correctly with Django 1.4. .. _version-2.5.0: 2.5.0 ===== :release-date: 2012-02-24 02:00 P.M GMT :by: Ask Solem .. _v250-important: Important Notes --------------- * Now depends on Celery 2.5. * Database schema has been updated. After upgrading you need migrate using South, or migrate manually as described below. These changes means that expiring results will be faster and take less memory than before. In addition a description field to the PeriodicTask model has been added so that the purpose of a periodic task in the database can be documented via the Admin interface. **South Migration** To migrate using South execute the following command:: $ python manage.py migrate djcelery If this is a new project that is also using South then you need to fake the migration: $ python manage.y migrate djcelery --fake **Manual Migration** To manually add the new fields, using PostgreSQL: .. code-block: sql ALTER TABLE celery_taskmeta ADD hidden BOOLEAN NOT NULL DEFAULT FALSE; ALTER TABLE celery_tasksetmeta ADD hidden BOOLEAN NOT NULL DEFAULT FALSE; ALTER TABLE djcelery_periodictask ADD description TEXT NOT NULL DEFAULT "" using MySQL: .. code-block:: sql ALTER TABLE celery_taskmeta ADD hidden TINYINT NOT NULL DEFAULT 0; ALTER TABLE celery_tasksetmeta ADD hidden TINYINT NOT NULL DEFAULT 0; ALTER TABLE djcelery_periodictask ADD description TEXT NOT NULL DEFAULT ""; using SQLite: .. code-block:: sql ALTER TABLE celery_taskmeta ADD hidden BOOL NOT NULL DEFAULT FALSE; ALTER TABLE celery_tasksetmeta ADD hidden BOOL NOT NULL DEFAULT FALSE; ALTER TABLE djcelery_periodictask ADD description VARCHAR(200) NOT NULL DEFAULT ""; .. _v250-news: News ---- * Auto-discovered task modules now works with the new auto-reloader functionality. * The database periodic task scheduler now tried to recover from operational database errors. * The periodic task schedule entry now accepts both int and timedelta (Issue #100). * 'Connection already closed' errors occurring while closing the database connection are now ignored (Issue #93). * The ``djcelerymon`` command used to start a Django admin monitor instance outside of Django projects now starts without a celery config module. * Should now work with Django 1.4's new timezone support. Contributed by Jannis Leidel and Donald Stufft. * South migrations did not work properly. Fix contributed by Christopher Grebs. * celeryd-multi now preserves django-related arguments, like ``--settings`` (Issue #94). * Migrations now work with Django < 1.3 (Issue #92). Fix contributed by Jude Nagurney. * The expiry of the database result backend can now be an int (Issue #84). .. _version-2.4.2: 2.4.2 ===== :release-date: 2011-11-14 12:00 P.M GMT * Fixed syntax error in South migrations code (Issue #88). Fix contributed by Olivier Tabone. .. _version-2.4.1: 2.4.1 ===== :release-date: 2011-11-07 06:00 P.M GMT :by: Ask Solem * Management commands was missing command line arguments because of recent changes to Celery. * Management commands now supports the ``--broker|-b`` option. * South migrations now ignores errors when tables already exist. .. _version-2.4.0: 2.4.0 ===== :release-date: 2011-11-04 04:00 P.M GMT :by: Ask Solem .. _240-important: Important Notes --------------- This release adds `South`_ migrations, which well assist users in automatically updating their database schemas with each django-celery release. .. _`South`: http://pypi.python.org/pypi/South/ .. _240-news: News ---- * Now depends on Celery 2.4.0 or higher. * South migrations have been added. Migration 0001 is a snapshot from the previous stable release (2.3.3). For those who do not use South, no action is required. South users will want to read the :ref:`240-upgrade_south` section below. Contributed by Greg Taylor. * Test runner now compatible with Django 1.4. Test runners are now classes instead of functions, so you have to change the ``TEST_RUNNER`` setting to read:: TEST_RUNNER = "djcelery.contrib.test_runner.CeleryTestSuiteRunner" Contributed by Jonas Haag. .. _240-upgrade_south: Upgrading for south users ------------------------- For those that are already using django-celery 2.3.x, you'll need to fake the newly added migration 0001, since your database already has the current ``djcelery_*`` and ``celery_*`` tables:: $ python manage.py migrate djcelery 0001 --fake If you're upgrading from the 2.2.x series, you'll want to drop/reset your ``celery_*`` and ``djcelery_*`` tables and run the migration:: $ python manage.py migrate djcelery .. _version-2.3.3: 2.3.3 ===== :release-date: 2011-08-22 12:00 AM BST * Precedence issue caused database backend tables to not be created (Issue #62). .. _version-2.3.2: 2.3.2 ===== :release-date: 2011-08-20 12:00 AM BST * Fixes circular import of DatabaseBackend. .. _version-2.3.1: 2.3.1 ===== :release-date: 2011-08-11 12:00 PM BST * Django database result backend tables were not created. If you are having troubles because of this, be sure you do a ``syncdb`` after upgrading, that should resolve the issue. .. _version-2.3.0: 2.3.0 ===== :release-date: 2011-08-05 12:00 PM BST * Now depends on Celery 2.3.0 Please read the Celery 2.3.0 changelog! .. _version-2.2.4: 2.2.4 ===== * celerybeat: DatabaseScheduler would not react to changes when using MySQL and the default transaction isolation level ``REPEATABLE-READ`` (Issue #41). It is still recommended that you use isolation level ``READ-COMMITTED`` (see the Celery FAQ). .. _version-2.2.3: 2.2.3 ===== :release-date: 2011-02-12 16:00 PM CET * celerybeat: DatabaseScheduler did not respect the disabled setting after restart. * celeryevcam: Expiring objects now works on PostgreSQL. * Now requires Celery 2.2.3 .. _version-2.2.2: 2.2.2 ===== :release-date: 2011-02-03 16:00 PM CET * Now requires Celery 2.2.2 * Periodic Task Admin broke if the CELERYBEAT_SCHEDULE setting was not set. * DatabaseScheduler No longer creates duplicate interval models. * The djcelery admin templates were not included in the distribution. .. _version-2.2.1: 2.2.1 ===== :release-date: 2011-02-02 16:00 PM CET * Should now work with Django versions previous to 1.2. .. _version-2.2.0: 2.2.0 ===== :release-date: 2011-02-01 10:00 AM CET * Now depends on Celery v2.2.0 * djceleryadm: Adds task actions Kill and Terminate task * celerycam: Django's queryset.delete() fetches everything in memory THEN deletes, so we need to use raw SQL to expire objects. * djcelerymon: Added Command.stdout + Command.stderr (Issue #23). * Need to close any open database connection after any embedded celerybeat process forks. * Added contrib/requirements/py25.txt * Demoproject now does ``djcelery.setup_loader`` in settings.py. .. _version-2.1.1: 2.1.1 ===== :release-date: 2010-10-14 02:00 PM CEST * Now depends on Celery v2.1.1. * Snapshots: Fixed bug with losing events. * Snapshots: Limited the number of worker timestamp updates to once every second. * Snapshot: Handle transaction manually and commit every 100 task updates. * snapshots: Can now configure when to expire task events. New settings: * ``CELERYCAM_EXPIRE_SUCCESS`` (default 1 day), * ``CELERYCAM_EXPIRE_ERROR`` (default 3 days), and * ``CELERYCAM_EXPIRE_PENDING`` (default 5 days). * Snapshots: ``TaskState.args`` and ``TaskState.kwargs`` are now represented as ``TextField`` instead of ``CharField``. If you need to represent arguments larger than 200 chars you have to migrate the table. * ``transaction.commit_manually`` doesn't accept arguments on older Django version. Should now work with Django versions previous to v1.2. * The tests doesn't need :mod:`unittest2` anymore if running on Python 2.7. .. _version-2.1.0: 2.1.0 ===== :release-date: 2010-10-08 12:00 PM CEST Important Notes --------------- This release depends on Celery version 2.1.0. Be sure to read the Celery changelog before you upgrade: http://ask.github.com/celery/changelog.html#version-2-1-0 News ---- * The periodic task schedule can now be stored in the database and edited via the Django Admin interface. To use the new database schedule you need to start ``celerybeat`` with the following argument:: $ python manage.py celerybeat -S djcelery.schedulers.DatabaseScheduler Note that you need to add your old periodic tasks to the database manually (using the Django admin interface for example). * New Celery monitor for the Django Admin interface. To start monitoring your workers you have to start your workers in event mode:: $ python manage.py celeryd -E (you can do this without restarting the server too:: >>> from celery.task.control import broadcast >>> broadcast("enable_events") You need to do a syncdb to create the new tables: python manage.py syncdb Then you need to start the snapshot camera:: $ python manage.py celerycam -f 2.0 This will take a snapshot of the events every 2 seconds and store it in the database. Fixes ----- * database backend: Now shows warning if polling results with transaction isolation level repeatable-read on MySQL. See http://github.com/ask/django-celery/issues/issue/6 * database backend: get result does no longer store the default result to database. See http://github.com/ask/django-celery/issues/issue/6 2.0.2 ===== Important notes --------------- * Due to some applications loading the Django models lazily, it is recommended that you add the following lines to your ``settings.py``:: import djcelery djcelery.setup_loader() This will ensure the Django celery loader is set even though the model modules haven't been imported yet. News ---- * ``djcelery.views.registered_tasks``: Added a view to list currently known tasks. 2.0.0 ===== :release-date: 2010-07-02 02:30 P.M CEST * Initial release django-celery-2.5.5/contrib/0000755000076500000240000000000011744004562016330 5ustar asksolstaff00000000000000django-celery-2.5.5/contrib/release/0000755000076500000240000000000011744004562017750 5ustar asksolstaff00000000000000django-celery-2.5.5/contrib/release/doc4allmods0000755000076500000240000000252211721673705022112 0ustar asksolstaff00000000000000#!/bin/bash PACKAGE="$1" SKIP_PACKAGES="$PACKAGE tests management urls" SKIP_FILES="djcelery.management.rst djcelery.management.commands.rst djcelery.management.commands.celeryd.rst djcelery.management.commands.celerybeat.rst djcelery.management.commands.celeryev.rst djcelery.mon.rst djcelery.monproj.rst djcelery.monproj.urls.rst djcelery.admin.rst djcelery.admin_utils.rst djcelery.contrib.rst djcelery.backends.rst djcelery.migrations.rst djcelery.migrations.0001_initial.rst djcelery.migrations.0002_v25_changes.rst" modules=$(find "$PACKAGE" -name "*.py") failed=0 for module in $modules; do dotted=$(echo $module | sed 's/\//\./g') name=${dotted%.__init__.py} name=${name%.py} rst=$name.rst skip=0 for skip_package in $SKIP_PACKAGES; do [ $(echo "$name" | cut -d. -f 2) == "$skip_package" ] && skip=1 done for skip_file in $SKIP_FILES; do [ "$skip_file" == "$rst" ] && skip=1 done if [ $skip -eq 0 ]; then if [ ! -f "docs/reference/$rst" ]; then if [ ! -f "docs/internals/reference/$rst" ]; then echo $rst :: FAIL failed=1 fi fi fi done exit $failed django-celery-2.5.5/contrib/release/flakeplus.py0000755000076500000240000000767411656051573022337 0ustar asksolstaff00000000000000#!/usr/bin/env python from __future__ import absolute_import from __future__ import with_statement import os import pprint import re import sys from collections import defaultdict from itertools import starmap from unipath import Path RE_COMMENT = r'^\s*\#' RE_NOQA = r'.+?\#\s+noqa+' RE_MULTILINE_COMMENT_O = r'^\s*(?:\'\'\'|""").+?(?:\'\'\'|""")' RE_MULTILINE_COMMENT_S = r'^\s*(?:\'\'\'|""")' RE_MULTILINE_COMMENT_E = r'(?:^|.+?)(?:\'\'\'|""")' RE_WITH = r'(?:^|\s+)with\s+' RE_WITH_IMPORT = r'''from\s+ __future__\s+ import\s+ with_statement''' RE_PRINT = r'''(?:^|\s+)print\((?:"|')(?:\W+?)?[A-Z0-9:]{2,}''' RE_ABS_IMPORT = r'''from\s+ __future__\s+ import\s+ absolute_import''' acc = defaultdict(lambda: {"abs": False, "print": False}) def compile(regex): return re.compile(regex, re.VERBOSE) class FlakePP(object): re_comment = compile(RE_COMMENT) re_ml_comment_o = compile(RE_MULTILINE_COMMENT_O) re_ml_comment_s = compile(RE_MULTILINE_COMMENT_S) re_ml_comment_e = compile(RE_MULTILINE_COMMENT_E) re_abs_import = compile(RE_ABS_IMPORT) re_print = compile(RE_PRINT) re_with_import = compile(RE_WITH_IMPORT) re_with = compile(RE_WITH) re_noqa = compile(RE_NOQA) map = {"abs": False, "print": False, "with": False, "with-used": False} def __init__(self, verbose=False): self.verbose = verbose self.steps = (("abs", self.re_abs_import), ("with", self.re_with_import), ("with-used", self.re_with), ("print", self.re_print)) def analyze_fh(self, fh): steps = self.steps filename = fh.name acc = dict(self.map) index = 0 errors = [0] def error(fmt, **kwargs): errors[0] += 1 self.announce(fmt, **dict(kwargs, filename=filename)) for index, line in enumerate(self.strip_comments(fh)): for key, pattern in self.steps: if pattern.match(line): acc[key] = True if index: if not acc["abs"]: error("%(filename)s: missing abs import") if acc["with-used"] and not acc["with"]: error("%(filename)s: missing with import") if acc["print"]: error("%(filename)s: left over print statement") return filename, errors[0], acc def analyze_file(self, filename): with open(filename) as fh: return self.analyze_fh(fh) def analyze_tree(self, dir): for dirpath, _, filenames in os.walk(dir): for path in (Path(dirpath, f) for f in filenames): if path.endswith(".py"): yield self.analyze_file(path) def analyze(self, *paths): for path in map(Path, paths): if path.isdir(): for res in self.analyze_tree(path): yield res else: yield self.analyze_file(path) def strip_comments(self, fh): re_comment = self.re_comment re_ml_comment_o = self.re_ml_comment_o re_ml_comment_s = self.re_ml_comment_s re_ml_comment_e = self.re_ml_comment_e re_noqa = self.re_noqa in_ml = False for line in fh.readlines(): if in_ml: if re_ml_comment_e.match(line): in_ml = False else: if re_noqa.match(line) or re_ml_comment_o.match(line): pass elif re_ml_comment_s.match(line): in_ml = True elif re_comment.match(line): pass else: yield line def announce(self, fmt, **kwargs): sys.stderr.write((fmt + "\n") % kwargs) def main(argv=sys.argv, exitcode=0): for _, errors, _ in FlakePP(verbose=True).analyze(*argv[1:]): if errors: exitcode = 1 return exitcode if __name__ == "__main__": sys.exit(main()) django-celery-2.5.5/contrib/release/removepyc.sh0000755000076500000240000000014011656051573022321 0ustar asksolstaff00000000000000#!/bin/bash (cd "${1:-.}"; find . -name "*.pyc" | xargs rm -- 2>/dev/null) || echo "ok" django-celery-2.5.5/contrib/release/sphinx-to-rst.py0000755000076500000240000000347611656051573023104 0ustar asksolstaff00000000000000#!/usr/bin/even/python import os import re import sys dirname = "" RE_CODE_BLOCK = re.compile(r'.. code-block:: (.+?)\s*$') RE_INCLUDE = re.compile(r'.. include:: (.+?)\s*$') RE_REFERENCE = re.compile(r':(.+?):`(.+?)`') def include_file(lines, pos, match): global dirname orig_filename = match.groups()[0] filename = os.path.join(dirname, orig_filename) fh = open(filename) try: old_dirname = dirname dirname = os.path.dirname(orig_filename) try: lines[pos] = sphinx_to_rst(fh) finally: dirname = old_dirname finally: fh.close() def replace_code_block(lines, pos, match): lines[pos] = "" curpos = pos - 1 # Find the first previous line with text to append "::" to it. while True: prev_line = lines[curpos] if not prev_line.isspace(): prev_line_with_text = curpos break curpos -= 1 if lines[prev_line_with_text].endswith(":"): lines[prev_line_with_text] += ":" else: lines[prev_line_with_text] += "::" TO_RST_MAP = {RE_CODE_BLOCK: replace_code_block, RE_REFERENCE: r'``\2``', RE_INCLUDE: include_file} def _process(lines): lines = list(lines) # non-destructive for i, line in enumerate(lines): for regex, alt in TO_RST_MAP.items(): if callable(alt): match = regex.match(line) if match: alt(lines, i, match) line = lines[i] else: lines[i] = regex.sub(alt, line) return lines def sphinx_to_rst(fh): return "".join(_process(fh)) if __name__ == "__main__": dirname = os.path.dirname(sys.argv[1]) fh = open(sys.argv[1]) try: print(sphinx_to_rst(fh)) finally: fh.close() django-celery-2.5.5/contrib/release/verify-reference-index.sh0000755000076500000240000000065511656051573024670 0ustar asksolstaff00000000000000#!/bin/bash verify_index() { modules=$(grep "celery." "$1" | \ perl -ple's/^\s*|\s*$//g;s{\.}{/}g;') retval=0 for module in $modules; do if [ ! -f "$module.py" ]; then if [ ! -f "$module/__init__.py" ]; then echo "Outdated reference: $module" retval=1 fi fi done return $retval } verify_index docs/reference/index.rst django-celery-2.5.5/contrib/supervisord/0000755000076500000240000000000011744004562020715 5ustar asksolstaff00000000000000django-celery-2.5.5/contrib/supervisord/celerybeat.conf0000644000076500000240000000101611656051573023707 0ustar asksolstaff00000000000000; ========================================== ; celerybeat supervisor example for Django ; ========================================== [program:celerybeat] command=/path/to/project/manage.py celerybeat --schedule=/var/lib/celery/celerybeat-schedule --loglevel=INFO directory=/path/to/project user=nobody numprocs=1 stdout_logfile=/var/log/celerybeat.log stderr_logfile=/var/log/celerybeat.log autostart=true autorestart=true startsecs=10 ; if rabbitmq is supervised, set its priority higher ; so it starts first priority=999 django-celery-2.5.5/contrib/supervisord/celeryd.conf0000644000076500000240000000112611656051573023221 0ustar asksolstaff00000000000000; ======================================= ; celeryd supervisor example for Django ; ======================================= [program:celery] command=/path/to/project/manage.py celeryd --loglevel=INFO directory=/path/to/project user=nobody numprocs=1 stdout_logfile=/var/log/celeryd.log stderr_logfile=/var/log/celeryd.log autostart=true autorestart=true startsecs=10 ; Need to wait for currently executing tasks to finish at shutdown. ; Increase this if you have very long running tasks. stopwaitsecs = 600 ; if rabbitmq is supervised, set its priority higher ; so it starts first priority=998 django-celery-2.5.5/django_celery.egg-info/0000755000076500000240000000000011744004562021167 5ustar asksolstaff00000000000000django-celery-2.5.5/django_celery.egg-info/dependency_links.txt0000644000076500000240000000000111744004560025233 0ustar asksolstaff00000000000000 django-celery-2.5.5/django_celery.egg-info/entry_points.txt0000644000076500000240000000006311744004560024462 0ustar asksolstaff00000000000000[console_scripts] djcelerymon = djcelery.mon:main django-celery-2.5.5/django_celery.egg-info/not-zip-safe0000644000076500000240000000000111743056655023426 0ustar asksolstaff00000000000000 django-celery-2.5.5/django_celery.egg-info/PKG-INFO0000644000076500000240000001564211744004560022272 0ustar asksolstaff00000000000000Metadata-Version: 1.0 Name: django-celery Version: 2.5.5 Summary: Django Celery Integration. Home-page: http://celeryproject.org Author: Ask Solem Author-email: ask@celeryproject.org License: BSD Description: =============================================== django-celery - Celery Integration for Django =============================================== .. image:: http://cloud.github.com/downloads/ask/celery/celery_128.png :Version: 2.5.5 :Web: http://celeryproject.org/ :Download: http://pypi.python.org/pypi/django-celery/ :Source: http://github.com/ask/django-celery/ :Keywords: celery, task queue, job queue, asynchronous, rabbitmq, amqp, redis, python, django, webhooks, queue, distributed -- django-celery provides Celery integration for Django; Using the Django ORM and cache backend for storing results, autodiscovery of task modules for applications listed in ``INSTALLED_APPS``, and more. .. contents:: :local: Using django-celery =================== To enable ``django-celery`` for your project you need to add ``djcelery`` to ``INSTALLED_APPS``:: INSTALLED_APPS += ("djcelery", ) then add the following lines to your ``settings.py``:: import djcelery djcelery.setup_loader() Everything works the same as described in the `Celery User Manual`_, except you need to invoke the programs through ``manage.py``: ===================================== ===================================== **Program** **Replace with** ===================================== ===================================== ``celeryd`` ``python manage.py celeryd`` ``celeryctl`` ``python manage.py celeryctl`` ``celerybeat`` ``python manage.py celerybeat`` ``camqadm`` ``python manage.py camqadm`` ``celeryev`` ``python manage.py celeryev`` ``celeryd-multi`` ``python manage.py celeryd_multi`` ===================================== ===================================== The other main difference is that configuration values are stored in your Django projects' ``settings.py`` module rather than in ``celeryconfig.py``. If you're trying celery for the first time you should start by reading `Getting started with django-celery`_ Special note for mod_wsgi users ------------------------------- If you're using ``mod_wsgi`` to deploy your Django application you need to include the following in your ``.wsgi`` module:: import djcelery djcelery.setup_loader() Documentation ============= The `Celery User Manual`_ contains user guides, tutorials and an API reference. Also the `django-celery documentation`_, contains information about the Django integration. .. _`django-celery documentation`: http://django-celery.readthedocs.org/ .. _`Celery User Manual`: http://docs.celeryproject.org/ .. _`Getting started with django-celery`: http://django-celery.readthedocs.org/en/latest/getting-started/first-steps-with-django.html Installation ============= You can install ``django-celery`` either via the Python Package Index (PyPI) or from source. To install using ``pip``,:: $ pip install django-celery To install using ``easy_install``,:: $ easy_install django-celery You will then want to create the necessary tables. If you are using south_ for schema migrations, you'll want to:: $ python manage.py migrate djcelery For those who are not using south, a normal ``syncdb`` will work:: $ python manage.py syncdb .. _south: http://pypi.python.org/pypi/South/ Downloading and installing from source -------------------------------------- Download the latest version of ``django-celery`` from http://pypi.python.org/pypi/django-celery/ You can install it by doing the following,:: $ tar xvfz django-celery-0.0.0.tar.gz $ cd django-celery-0.0.0 # python setup.py install # as root Using the development version ------------------------------ You can clone the git repository by doing the following:: $ git clone git://github.com/ask/django-celery.git Getting Help ============ Mailing list ------------ For discussions about the usage, development, and future of celery, please join the `celery-users`_ mailing list. .. _`celery-users`: http://groups.google.com/group/celery-users/ IRC --- Come chat with us on IRC. The `#celery`_ channel is located at the `Freenode`_ network. .. _`#celery`: irc://irc.freenode.net/celery .. _`Freenode`: http://freenode.net Bug tracker =========== If you have any suggestions, bug reports or annoyances please report them to our issue tracker at http://github.com/ask/django-celery/issues/ Wiki ==== http://wiki.github.com/ask/celery/ Contributing ============ Development of ``django-celery`` happens at Github: http://github.com/ask/django-celery You are highly encouraged to participate in the development. If you don't like Github (for some reason) you're welcome to send regular patches. License ======= This software is licensed under the ``New BSD License``. See the ``LICENSE`` file in the top distribution directory for the full license text. .. # vim: syntax=rst expandtab tabstop=4 shiftwidth=4 shiftround Platform: any Classifier: Development Status :: 5 - Production/Stable Classifier: Framework :: Django Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: POSIX Classifier: Topic :: Communications Classifier: Topic :: System :: Distributed Computing Classifier: Topic :: Software Development :: Libraries :: Python Modules django-celery-2.5.5/django_celery.egg-info/requires.txt0000644000076500000240000000004711744004560023566 0ustar asksolstaff00000000000000django-picklefield>=0.2.0 celery>=2.5.2django-celery-2.5.5/django_celery.egg-info/SOURCES.txt0000644000076500000240000000770511744004560023062 0ustar asksolstaff00000000000000AUTHORS Changelog FAQ INSTALL LICENSE MANIFEST.in README README.rst THANKS TODO setup.cfg setup.py bin/djcelerymon contrib/release/doc4allmods contrib/release/flakeplus.py contrib/release/removepyc.sh contrib/release/sphinx-to-rst.py contrib/release/verify-reference-index.sh contrib/supervisord/celerybeat.conf contrib/supervisord/celeryd.conf django_celery.egg-info/PKG-INFO django_celery.egg-info/SOURCES.txt django_celery.egg-info/dependency_links.txt django_celery.egg-info/entry_points.txt django_celery.egg-info/not-zip-safe django_celery.egg-info/requires.txt django_celery.egg-info/top_level.txt djcelery/__init__.py djcelery/admin.py djcelery/admin_utils.py djcelery/app.py djcelery/humanize.py djcelery/loaders.py djcelery/managers.py djcelery/models.py djcelery/mon.py djcelery/schedulers.py djcelery/snapshot.py djcelery/urls.py djcelery/utils.py djcelery/views.py djcelery/backends/__init__.py djcelery/backends/cache.py djcelery/backends/database.py djcelery/contrib/__init__.py djcelery/contrib/test_runner.py djcelery/management/__init__.py djcelery/management/base.py djcelery/management/commands/__init__.py djcelery/management/commands/camqadm.py djcelery/management/commands/celerybeat.py djcelery/management/commands/celerycam.py djcelery/management/commands/celeryctl.py djcelery/management/commands/celeryd.py djcelery/management/commands/celeryd_detach.py djcelery/management/commands/celeryd_multi.py djcelery/management/commands/celeryev.py djcelery/management/commands/celerymon.py djcelery/management/commands/djcelerymon.py djcelery/migrations/0001_initial.py djcelery/migrations/0002_v25_changes.py djcelery/migrations/__init__.py djcelery/monproj/__init__.py djcelery/monproj/urls.py djcelery/templates/admin/djcelery/change_list.html djcelery/templates/djcelery/confirm_rate_limit.html djcelery/tests/__init__.py djcelery/tests/req.py djcelery/tests/test_conf.py djcelery/tests/test_discovery.py djcelery/tests/test_loaders.py djcelery/tests/test_models.py djcelery/tests/test_schedulers.py djcelery/tests/test_snapshot.py djcelery/tests/test_views.py djcelery/tests/test_worker_job.py djcelery/tests/utils.py djcelery/tests/test_backends/__init__.py djcelery/tests/test_backends/test_cache.py djcelery/tests/test_backends/test_database.py docs/Makefile docs/__init__.py docs/changelog.rst docs/conf.py docs/faq.rst docs/index.rst docs/introduction.rst docs/settings.py docs/.static/.keep docs/.templates/sidebarintro.html docs/.templates/sidebarlogo.html docs/_ext/applyxrefs.py docs/_ext/literals_to_xrefs.py docs/_theme/celery/theme.conf docs/_theme/celery/static/celery.css_t docs/cookbook/index.rst docs/cookbook/unit-testing.rst docs/getting-started/first-steps-with-django.rst docs/getting-started/index.rst docs/reference/djcelery.app.rst docs/reference/djcelery.backends.cache.rst docs/reference/djcelery.backends.database.rst docs/reference/djcelery.contrib.test_runner.rst docs/reference/djcelery.humanize.rst docs/reference/djcelery.loaders.rst docs/reference/djcelery.managers.rst docs/reference/djcelery.models.rst docs/reference/djcelery.schedulers.rst docs/reference/djcelery.snapshot.rst docs/reference/djcelery.urls.rst docs/reference/djcelery.utils.rst docs/reference/djcelery.views.rst docs/reference/index.rst examples/demoproject/manage.py examples/demoproject/demoapp/__init__.py examples/demoproject/demoapp/models.py examples/demoproject/demoapp/tasks.py examples/demoproject/demoapp/tests.py examples/demoproject/demoapp/views.py examples/demoproject/demoproject/__init__.py examples/demoproject/demoproject/settings.py examples/demoproject/demoproject/urls.py examples/demoproject/demoproject/wsgi.py locale/en/LC_MESSAGES/django.po requirements/default.txt requirements/docs.txt requirements/test.txt tests/__init__.py tests/manage.py tests/settings.py tests/urls.py tests/someapp/__init__.py tests/someapp/models.py tests/someapp/tasks.py tests/someapp/tests.py tests/someapp/views.py tests/someappwotask/__init__.py tests/someappwotask/models.py tests/someappwotask/views.pydjango-celery-2.5.5/django_celery.egg-info/top_level.txt0000644000076500000240000000001111744004560023707 0ustar asksolstaff00000000000000djcelery django-celery-2.5.5/djcelery/0000755000076500000240000000000011744004562016471 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/__init__.py0000644000076500000240000000124211744004527020602 0ustar asksolstaff00000000000000"""Django Celery Integration.""" # :copyright: (c) 2009 - 2012 by Ask Solem. # :license: BSD, see LICENSE for more details. from __future__ import absolute_import import os VERSION = (2, 5, 5) __version__ = ".".join(map(str, VERSION[0:3])) + "".join(VERSION[3:]) __author__ = "Ask Solem" __contact__ = "ask@celeryproject.org" __homepage__ = "http://celeryproject.org" __docformat__ = "restructuredtext" __license__ = "BSD (3 clause)" # -eof meta- def setup_loader(): os.environ.setdefault("CELERY_LOADER", "djcelery.loaders.DjangoLoader") # Importing this module enables the Celery Django loader. setup_loader() from celery import current_app as celery # noqa django-celery-2.5.5/djcelery/admin.py0000644000076500000240000002474411741563541020152 0ustar asksolstaff00000000000000from __future__ import absolute_import from __future__ import with_statement from django import forms from django.conf import settings from django.contrib import admin from django.contrib.admin import helpers from django.contrib.admin.views import main as main_views from django.shortcuts import render_to_response from django.template import RequestContext from django.utils.encoding import force_unicode from django.utils.html import escape from django.utils.translation import ugettext_lazy as _ from celery import current_app from celery import states from celery import registry from celery.task.control import broadcast, revoke, rate_limit from celery.utils import abbrtask from . import loaders from .admin_utils import action, display_field, fixedwidth from .models import (TaskState, WorkerState, PeriodicTask, IntervalSchedule, CrontabSchedule) from .humanize import naturaldate TASK_STATE_COLORS = {states.SUCCESS: "green", states.FAILURE: "red", states.REVOKED: "magenta", states.STARTED: "yellow", states.RETRY: "orange", "RECEIVED": "blue"} NODE_STATE_COLORS = {"ONLINE": "green", "OFFLINE": "gray"} class MonitorList(main_views.ChangeList): def __init__(self, *args, **kwargs): super(MonitorList, self).__init__(*args, **kwargs) self.title = self.model_admin.list_page_title @display_field(_("state"), "state") def colored_state(task): state = escape(task.state) color = TASK_STATE_COLORS.get(task.state, "black") return """%s""" % (color, state) @display_field(_("state"), "last_heartbeat") def node_state(node): state = node.is_alive() and "ONLINE" or "OFFLINE" color = NODE_STATE_COLORS[state] return """%s""" % (color, state) @display_field(_("ETA"), "eta") def eta(task): if not task.eta: return """none""" return escape(task.eta) @display_field(_("when"), "tstamp") def tstamp(task): return """
%s
""" % (escape(str(task.tstamp)), escape(naturaldate(task.tstamp))) @display_field(_("name"), "name") def name(task): short_name = abbrtask(task.name, 16) return """
%s
""" % (escape(task.name), escape(short_name)) class ModelMonitor(admin.ModelAdmin): can_add = False can_delete = False def get_changelist(self, request, **kwargs): return MonitorList def change_view(self, request, object_id, extra_context=None): extra_context = extra_context or {} extra_context.setdefault("title", self.detail_title) return super(ModelMonitor, self).change_view(request, object_id, extra_context) def has_delete_permission(self, request, obj=None): if not self.can_delete: return False return super(ModelMonitor, self).has_delete_permission(request, obj) def has_add_permission(self, request): if not self.can_add: return False return super(ModelMonitor, self).has_add_permission(request) class TaskMonitor(ModelMonitor): detail_title = _("Task detail") list_page_title = _("Tasks") rate_limit_confirmation_template = "djcelery/confirm_rate_limit.html" date_hierarchy = "tstamp" fieldsets = ( (None, { "fields": ("state", "task_id", "name", "args", "kwargs", "eta", "runtime", "worker", "tstamp"), "classes": ("extrapretty", ), }), ("Details", { "classes": ("collapse", "extrapretty"), "fields": ("result", "traceback", "expires"), }), ) list_display = (fixedwidth("task_id", name=_("UUID"), pt=8), colored_state, name, fixedwidth("args", pretty=True), fixedwidth("kwargs", pretty=True), eta, tstamp, "worker") readonly_fields = ("state", "task_id", "name", "args", "kwargs", "eta", "runtime", "worker", "result", "traceback", "expires", "tstamp") list_filter = ("state", "name", "tstamp", "eta", "worker") search_fields = ("name", "task_id", "args", "kwargs", "worker__hostname") actions = ["revoke_tasks", "terminate_tasks", "kill_tasks", "rate_limit_tasks"] @action(_("Revoke selected tasks")) def revoke_tasks(self, request, queryset): with current_app.default_connection() as connection: for state in queryset: revoke(state.task_id, connection=connection) @action(_("Terminate selected tasks")) def terminate_tasks(self, request, queryset): with current_app.default_connection() as connection: for state in queryset: revoke(state.task_id, connection=connection, terminate=True) @action(_("Kill selected tasks")) def kill_tasks(self, request, queryset): with current_app.default_connection() as connection: for state in queryset: revoke(state.task_id, connection=connection, terminate=True, signal="KILL") @action(_("Rate limit selected tasks")) def rate_limit_tasks(self, request, queryset): tasks = set([task.name for task in queryset]) opts = self.model._meta app_label = opts.app_label if request.POST.get("post"): rate = request.POST["rate_limit"] with current_app.default_connection() as connection: for task_name in tasks: rate_limit(task_name, rate, connection=connection) return None context = { "title": _("Rate limit selection"), "queryset": queryset, "object_name": force_unicode(opts.verbose_name), "action_checkbox_name": helpers.ACTION_CHECKBOX_NAME, "opts": opts, "app_label": app_label, } return render_to_response(self.rate_limit_confirmation_template, context, context_instance=RequestContext(request)) def get_actions(self, request): actions = super(TaskMonitor, self).get_actions(request) actions.pop("delete_selected", None) return actions def queryset(self, request): qs = super(TaskMonitor, self).queryset(request) return qs.select_related('worker') class WorkerMonitor(ModelMonitor): can_add = True detail_title = _("Node detail") list_page_title = _("Worker Nodes") list_display = ("hostname", node_state) readonly_fields = ("last_heartbeat", ) actions = ["shutdown_nodes", "enable_events", "disable_events"] @action(_("Shutdown selected worker nodes")) def shutdown_nodes(self, request, queryset): broadcast("shutdown", destination=[n.hostname for n in queryset]) @action(_("Enable event mode for selected nodes.")) def enable_events(self, request, queryset): broadcast("enable_events", destination=[n.hostname for n in queryset]) @action(_("Disable event mode for selected nodes.")) def disable_events(self, request, queryset): broadcast("disable_events", destination=[n.hostname for n in queryset]) def get_actions(self, request): actions = super(WorkerMonitor, self).get_actions(request) actions.pop("delete_selected", None) return actions admin.site.register(TaskState, TaskMonitor) admin.site.register(WorkerState, WorkerMonitor) # ### Periodic Tasks class LaxChoiceField(forms.ChoiceField): def valid_value(self, value): return True def periodic_task_form(): loaders.autodiscover() tasks = list(sorted(registry.tasks.regular().keys())) choices = (("", ""), ) + tuple(zip(tasks, tasks)) class PeriodicTaskForm(forms.ModelForm): regtask = LaxChoiceField(label=_(u"Task (registered)"), choices=choices, required=False) task = forms.CharField(label=_("Task (custom)"), required=False, max_length=200) class Meta: model = PeriodicTask def clean(self): data = super(PeriodicTaskForm, self).clean() regtask = data.get("regtask") if regtask: data["task"] = regtask if not data["task"]: exc = forms.ValidationError(_(u"Need name of task")) self._errors["task"] = self.error_class(exc.messages) raise exc return data return PeriodicTaskForm class PeriodicTaskAdmin(admin.ModelAdmin): model = PeriodicTask form = periodic_task_form() list_display = ('__unicode__', 'enabled') fieldsets = ( (None, { "fields": ("name", "regtask", "task", "enabled"), "classes": ("extrapretty", "wide"), }), ("Schedule", { "fields": ("interval", "crontab"), "classes": ("extrapretty", "wide", ), }), ("Arguments", { "fields": ("args", "kwargs"), "classes": ("extrapretty", "wide", "collapse"), }), ("Execution Options", { "fields": ("expires", "queue", "exchange", "routing_key"), "classes": ("extrapretty", "wide", "collapse"), }), ) def __init__(self, *args, **kwargs): super(PeriodicTaskAdmin, self).__init__(*args, **kwargs) self.form = periodic_task_form() def changelist_view(self, request, extra_context=None): extra_context = extra_context or {} scheduler = getattr(settings, "CELERYBEAT_SCHEDULER", None) if scheduler != 'djcelery.schedulers.DatabaseScheduler': extra_context['wrong_scheduler'] = True return super(PeriodicTaskAdmin, self).changelist_view(request, extra_context) def queryset(self, request): qs = super(PeriodicTaskAdmin, self).queryset(request) return qs.select_related('interval', 'crontab') admin.site.register(IntervalSchedule) admin.site.register(CrontabSchedule) admin.site.register(PeriodicTask, PeriodicTaskAdmin) django-celery-2.5.5/djcelery/admin_utils.py0000644000076500000240000000253111741563541021360 0ustar asksolstaff00000000000000from __future__ import absolute_import from pprint import pformat from django.utils.html import escape def attrs(**kwargs): def _inner(fun): for attr_name, attr_value in kwargs.items(): setattr(fun, attr_name, attr_value) return fun return _inner def display_field(short_description, admin_order_field, allow_tags=True, **kwargs): return attrs(short_description=short_description, admin_order_field=admin_order_field, allow_tags=allow_tags, **kwargs) def action(short_description, **kwargs): return attrs(short_description=short_description, **kwargs) def fixedwidth(field, name=None, pt=6, width=16, maxlen=64, pretty=False): @display_field(name or field, field) def f(task): val = getattr(task, field) if pretty: val = pformat(val, width=width) if val.startswith("u'") or val.startswith('u"'): val = val[2:-1] shortval = val.replace(",", ",\n") shortval = shortval.replace("\n", "|br/|") if len(shortval) > maxlen: shortval = shortval[:maxlen] + "..." return """%s""" % ( escape(val[:255]), pt, escape(shortval)).replace("|br/|", "
") return f django-celery-2.5.5/djcelery/app.py0000644000076500000240000000020211656051573017623 0ustar asksolstaff00000000000000from __future__ import absolute_import from celery.app import default_app #: The Django-Celery app instance. app = default_app django-celery-2.5.5/djcelery/backends/0000755000076500000240000000000011744004562020243 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/backends/__init__.py0000644000076500000240000000000011656051573022350 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/backends/cache.py0000644000076500000240000000430011656051573021663 0ustar asksolstaff00000000000000"""celery.backends.cache""" from __future__ import absolute_import from datetime import timedelta import django from django.utils.encoding import smart_str from django.core.cache import cache, get_cache from celery import current_app from celery.utils.timeutils import timedelta_seconds from celery.backends.base import KeyValueStoreBackend # CELERY_CACHE_BACKEND overrides the django-global(tm) backend settings. if current_app.conf.CELERY_CACHE_BACKEND: cache = get_cache(current_app.conf.CELERY_CACHE_BACKEND) # noqa class DjangoMemcacheWrapper(object): """Wrapper class to django's memcache backend class, that overrides the :meth:`get` method in order to remove the forcing of unicode strings since it may cause binary or pickled data to break.""" def __init__(self, cache): self.cache = cache def get(self, key, default=None): val = self.cache._cache.get(smart_str(key)) if val is None: return default else: return val def set(self, key, value, timeout=0): self.cache.set(key, value, timeout) # Check if django is using memcache as the cache backend. If so, wrap the # cache object in a DjangoMemcacheWrapper for Django < 1.2 that fixes a bug # with retrieving pickled data. from django.core.cache.backends.base import InvalidCacheBackendError try: from django.core.cache.backends.memcached import CacheClass except InvalidCacheBackendError: pass else: if django.VERSION[0:2] < (1, 2) and isinstance(cache, CacheClass): cache = DjangoMemcacheWrapper(cache) class CacheBackend(KeyValueStoreBackend): """Backend using the Django cache framework to store task metadata.""" def __init__(self, *args, **kwargs): super(CacheBackend, self).__init__(self, *args, **kwargs) expires = kwargs.get("expires", current_app.conf.CELERY_TASK_RESULT_EXPIRES) if isinstance(expires, timedelta): expires = int(timedelta_seconds(expires)) self.expires = expires def get(self, key): return cache.get(key) def set(self, key, value): cache.set(key, value, self.expires) def delete(self, key): cache.delete(key) django-celery-2.5.5/djcelery/backends/database.py0000644000076500000240000000364011721671621022365 0ustar asksolstaff00000000000000from __future__ import absolute_import from celery import current_app from celery.backends.base import BaseDictBackend from celery.utils.timeutils import maybe_timedelta from ..models import TaskMeta, TaskSetMeta class DatabaseBackend(BaseDictBackend): """The database backend. Using Django models to store task state. """ TaskModel = TaskMeta TaskSetModel = TaskSetMeta expires = current_app.conf.CELERY_TASK_RESULT_EXPIRES create_django_tables = True subpolling_interval = 0.5 def _store_result(self, task_id, result, status, traceback=None): """Store return value and status of an executed task.""" self.TaskModel._default_manager.store_result(task_id, result, status, traceback=traceback) return result def _save_taskset(self, taskset_id, result): """Store the result of an executed taskset.""" self.TaskSetModel._default_manager.store_result(taskset_id, result) return result def _get_task_meta_for(self, task_id): """Get task metadata for a task by id.""" return self.TaskModel._default_manager.get_task(task_id).to_dict() def _restore_taskset(self, taskset_id): """Get taskset metadata for a taskset by id.""" meta = self.TaskSetModel._default_manager.restore_taskset(taskset_id) if meta: return meta.to_dict() def _delete_taskset(self, taskset_id): self.TaskSetModel._default_manager.delete_taskset(taskset_id) def _forget(self, task_id): try: self.TaskModel._default_manager.get(task_id=task_id).delete() except self.TaskModel.DoesNotExist: pass def cleanup(self): """Delete expired metadata.""" expires = maybe_timedelta(self.expires) for model in self.TaskModel, self.TaskSetModel: model._default_manager.delete_expired(expires) django-celery-2.5.5/djcelery/contrib/0000755000076500000240000000000011744004562020131 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/contrib/__init__.py0000644000076500000240000000000011656051573022236 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/contrib/test_runner.py0000644000076500000240000000136411723662707023067 0ustar asksolstaff00000000000000from __future__ import absolute_import from django.conf import settings from django.test.simple import DjangoTestSuiteRunner USAGE = """\ Custom test runner to allow testing of celery delayed tasks. """ class CeleryTestSuiteRunner(DjangoTestSuiteRunner): """Django test runner allowing testing of celery delayed tasks. All tasks are run locally, not in a worker. To use this runner set ``settings.TEST_RUNNER``:: TEST_RUNNER = "djcelery.contrib.test_runner.CeleryTestSuiteRunner" """ def setup_test_environment(self, **kwargs): super(CeleryTestSuiteRunner, self).setup_test_environment(**kwargs) settings.CELERY_ALWAYS_EAGER = True settings.CELERY_EAGER_PROPAGATES_EXCEPTIONS = True # Issue #75 django-celery-2.5.5/djcelery/humanize.py0000644000076500000240000000365711723671451020702 0ustar asksolstaff00000000000000from __future__ import absolute_import from datetime import datetime from django.utils.translation import ungettext, ugettext as _ from .utils import now JUST_NOW = _("just now") SECONDS_AGO = (_("%(seconds)d second ago"), _("%(seconds)d seconds ago")) MINUTES_AGO = (_("%(minutes)d minute ago"), _("%(minutes)d minutes ago")) HOURS_AGO = (_("%(hours)d hour ago"), _("%(hours)d hours ago")) YESTERDAY_AT = _("yesterday at %(time)s") OLDER_YEAR = (_("year"), _("years")) OLDER_MONTH = (_("month"), _("months")) OLDER_WEEK = (_("week"), _("weeks")) OLDER_DAY = (_("day"), _("days")) OLDER_CHUNKS = ( (365.0, OLDER_YEAR), (30.0, OLDER_MONTH), (7.0, OLDER_WEEK), (1.0, OLDER_DAY), ) OLDER_AGO = _("%(number)d %(type)s ago") def _un(singular__plural, n=None): singular, plural = singular__plural return ungettext(singular, plural, n) def naturaldate(date): """Convert datetime into a human natural date string.""" if not date: return '' right_now = now() today = datetime(right_now.year, right_now.month, right_now.day, tzinfo=right_now.tzinfo) delta = right_now - date delta_midnight = today - date days = delta.days hours = round(delta.seconds / 3600, 0) minutes = delta.seconds / 60 if days < 0: return JUST_NOW if days == 0: if hours == 0: if minutes > 0: return _un(MINUTES_AGO, n=minutes) % {"minutes": minutes} else: return JUST_NOW else: return _un(HOURS_AGO, n=hours) % {"hours": hours} if delta_midnight.days == 0: return YESTERDAY_AT % {"time": date.strftime("%H:%M")} count = 0 for chunk, singular_plural in OLDER_CHUNKS: if days >= chunk: count = round((delta_midnight.days + 1) / chunk, 0) type_ = _un(singular_plural, n=count) break return OLDER_AGO % {"number": count, "type": type_} django-celery-2.5.5/djcelery/loaders.py0000644000076500000240000001124711744004503020474 0ustar asksolstaff00000000000000from __future__ import absolute_import import imp import importlib import warnings from celery import signals from celery.loaders.base import BaseLoader from celery.datastructures import DictAttribute from django import db from django.conf import settings from django.core import cache from django.core.mail import mail_admins from .utils import DATABASE_ERRORS, now _RACE_PROTECTION = False class DjangoLoader(BaseLoader): """The Django loader.""" _db_reuse = 0 override_backends = { "database": "djcelery.backends.database.DatabaseBackend", "cache": "djcelery.backends.cache.CacheBackend"} def __init__(self, *args, **kwargs): super(DjangoLoader, self).__init__(*args, **kwargs) self._install_signal_handlers() def _install_signal_handlers(self): # Need to close any open database connection after # any embedded celerybeat process forks. signals.beat_embedded_init.connect(self.close_database) def now(self, **kwargs): return now() def read_configuration(self): """Load configuration from Django settings.""" self.configured = True # Default backend needs to be the database backend for backward # compatibility. backend = getattr(settings, "CELERY_RESULT_BACKEND", None) or \ getattr(settings, "CELERY_BACKEND", None) if not backend: settings.CELERY_RESULT_BACKEND = "database" return DictAttribute(settings) def _close_database(self): try: funs = [conn.close for conn in db.connections] except AttributeError: funs = [db.close_connection] # pre multidb for close in funs: try: close() except DATABASE_ERRORS, exc: if "closed" not in str(exc): raise def close_database(self, **kwargs): db_reuse_max = self.conf.get("CELERY_DB_REUSE_MAX", None) if not db_reuse_max: return self._close_database() if self._db_reuse >= db_reuse_max * 2: self._db_reuse = 0 self._close_database() self._db_reuse += 1 def close_cache(self): try: cache.cache.close() except (TypeError, AttributeError): pass def on_process_cleanup(self): """Does everything necessary for Django to work in a long-living, multiprocessing environment. """ # See http://groups.google.com/group/django-users/ # browse_thread/thread/78200863d0c07c6d/ self.close_database() self.close_cache() def on_task_init(self, task_id, task): """Called before every task.""" try: is_eager = task.request.is_eager except AttributeError: is_eager = False if not is_eager: self.close_database() def on_worker_init(self): """Called when the worker starts. Automatically discovers any ``tasks.py`` files in the applications listed in ``INSTALLED_APPS``. """ if settings.DEBUG: warnings.warn("Using settings.DEBUG leads to a memory leak, never " "use this setting in production environments!") self.import_default_modules() self.close_database() self.close_cache() def import_default_modules(self): super(DjangoLoader, self).import_default_modules() self.autodiscover() def autodiscover(self): self.task_modules.update(mod.__name__ for mod in autodiscover() or ()) def on_worker_process_init(self): # the parent process may have established these, # so need to close them. self.close_database() self.close_cache() def mail_admins(self, subject, body, fail_silently=False, **kwargs): return mail_admins(subject, body, fail_silently=fail_silently) def autodiscover(): """Include tasks for all applications in ``INSTALLED_APPS``.""" global _RACE_PROTECTION if _RACE_PROTECTION: return _RACE_PROTECTION = True try: return filter(None, [find_related_module(app, "tasks") for app in settings.INSTALLED_APPS]) finally: _RACE_PROTECTION = False def find_related_module(app, related_name): """Given an application name and a module name, tries to find that module in the application.""" try: app_path = importlib.import_module(app).__path__ except AttributeError: return try: imp.find_module(related_name, app_path) except ImportError: return return importlib.import_module("%s.%s" % (app, related_name)) django-celery-2.5.5/djcelery/management/0000755000076500000240000000000011744004562020605 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/management/__init__.py0000644000076500000240000000000011656051573022712 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/management/base.py0000644000076500000240000000641311742044217022074 0ustar asksolstaff00000000000000from __future__ import absolute_import import os import sys from django.core.management.base import BaseCommand import celery import djcelery DB_SHARED_THREAD = """\ DatabaseWrapper objects created in a thread can only \ be used in that same thread. The object with alias '%s' \ was created in thread id %s and this is thread id %s.\ """ def patch_thread_ident(): # monkey patch django. # This patch make sure that we use real threads to get the ident which # is going to happen if we are using gevent or eventlet. # -- patch taken from gunicorn if getattr(patch_thread_ident, "called", False): return try: from django.db.backends import BaseDatabaseWrapper, DatabaseError if "validate_thread_sharing" in BaseDatabaseWrapper.__dict__: import thread _get_ident = thread.get_ident __old__init__ = BaseDatabaseWrapper.__init__ def _init(self, *args, **kwargs): __old__init__(self, *args, **kwargs) self._thread_ident = _get_ident() def _validate_thread_sharing(self): if (not self.allow_thread_sharing and self._thread_ident != _get_ident()): raise DatabaseError(DB_SHARED_THREAD % ( self.alias, self._thread_ident, _get_ident())) BaseDatabaseWrapper.__init__ = _init BaseDatabaseWrapper.validate_thread_sharing \ = _validate_thread_sharing patch_thread_ident.called = True except ImportError: pass patch_thread_ident() class CeleryCommand(BaseCommand): options = BaseCommand.option_list skip_opts = ["--app", "--loader", "--config"] keep_base_opts = False def get_version(self): return "celery %s\ndjango-celery %s" % (celery.__version__, djcelery.__version__) def execute(self, *args, **options): broker = options.get("broker") if broker: self.set_broker(broker) super(CeleryCommand, self).execute(*args, **options) def set_broker(self, broker): os.environ["CELERY_BROKER_URL"] = broker def run_from_argv(self, argv): self.handle_default_options(argv[2:]) return super(CeleryCommand, self).run_from_argv(argv) def handle_default_options(self, argv): acc = [] broker = None for i, arg in enumerate(argv): if "--settings=" in arg: _, settings_module = arg.split("=") os.environ["DJANGO_SETTINGS_MODULE"] = settings_module elif "--pythonpath=" in arg: _, pythonpath = arg.split("=") sys.path.insert(0, pythonpath) elif "--broker=" in arg: _, broker = arg.split("=") elif arg == "-b": broker = argv[i + 1] else: acc.append(arg) if broker: self.set_broker(broker) return argv if self.keep_base_opts else acc def die(self, msg): sys.stderr.write(msg) sys.stderr.write("\n") sys.exit() @property def option_list(self): return [x for x in self.options if x._long_opts[0] not in self.skip_opts] django-celery-2.5.5/djcelery/management/commands/0000755000076500000240000000000011744004562022406 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/management/commands/__init__.py0000644000076500000240000000000011656051573024513 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/management/commands/camqadm.py0000644000076500000240000000116511656051573024374 0ustar asksolstaff00000000000000""" Celery AMQP Administration Tool using the AMQP API. """ from __future__ import absolute_import from celery.bin import camqadm from djcelery.app import app from djcelery.management.base import CeleryCommand command = camqadm.AMQPAdminCommand(app=app) class Command(CeleryCommand): """Run the celery daemon.""" options = (CeleryCommand.options + command.get_options() + command.preload_options) help = 'Celery AMQP Administration Tool using the AMQP API.' def handle(self, *args, **options): """Handle the management command.""" command.run(*args, **options) django-celery-2.5.5/djcelery/management/commands/celerybeat.py0000644000076500000240000000111711656051573025105 0ustar asksolstaff00000000000000""" Start the celery clock service from the Django management command. """ from __future__ import absolute_import from celery.bin import celerybeat from djcelery.app import app from djcelery.management.base import CeleryCommand beat = celerybeat.BeatCommand(app=app) class Command(CeleryCommand): """Run the celery periodic task scheduler.""" options = (CeleryCommand.options + beat.get_options() + beat.preload_options) help = 'Runs the Celery periodic task scheduler' def handle(self, *args, **options): beat.run(*args, **options) django-celery-2.5.5/djcelery/management/commands/celerycam.py0000644000076500000240000000122611656051573024733 0ustar asksolstaff00000000000000""" Shortcut to the Django snapshot service. """ from __future__ import absolute_import from celery.bin import celeryev from djcelery.app import app from djcelery.management.base import CeleryCommand ev = celeryev.EvCommand(app=app) class Command(CeleryCommand): """Run the celery curses event viewer.""" options = (CeleryCommand.options + ev.get_options() + ev.preload_options) help = 'Takes snapshots of the clusters state to the database.' def handle(self, *args, **options): """Handle the management command.""" options["camera"] = "djcelery.snapshot.Camera" ev.run(*args, **options) django-celery-2.5.5/djcelery/management/commands/celeryctl.py0000644000076500000240000000150411742044217024745 0ustar asksolstaff00000000000000""" Celery manamagent and monitoring utility. """ from __future__ import absolute_import from celery.bin.celeryctl import celeryctl, Command as _Command from djcelery import __version__ from djcelery.app import app from djcelery.management.base import CeleryCommand # Django hijacks the version output and prints its version before our # version. So display the names of the products so the output is sensible. _Command.version = "celery %s\ndjango-celery %s" % (_Command.version, __version__) class Command(CeleryCommand): """Run the celery control utility.""" help = "celery control utility" keep_base_opts = False def run_from_argv(self, argv): util = celeryctl(app=app) util.execute_from_commandline(self.handle_default_options(argv)[1:]) django-celery-2.5.5/djcelery/management/commands/celeryd.py0000644000076500000240000000112311741615736024414 0ustar asksolstaff00000000000000""" Start the celery daemon from the Django management command. """ from __future__ import absolute_import from celery.bin import celeryd from djcelery.app import app from djcelery.management.base import CeleryCommand worker = celeryd.WorkerCommand(app=app) class Command(CeleryCommand): """Run the celery daemon.""" help = 'Runs a Celery worker node.' requires_model_validation = True options = (CeleryCommand.options + worker.get_options() + worker.preload_options) def handle(self, *args, **options): worker.run(*args, **options) django-celery-2.5.5/djcelery/management/commands/celeryd_detach.py0000644000076500000240000000116411656051573025727 0ustar asksolstaff00000000000000""" Start detached worker node from the Django management utility. """ from __future__ import absolute_import import os import sys from celery.bin import celeryd_detach from djcelery.management.base import CeleryCommand class Command(CeleryCommand): """Run the celery daemon.""" help = 'Runs a detached Celery worker node.' requires_model_validation = True options = celeryd_detach.OPTION_LIST def run_from_argv(self, argv): class detached(celeryd_detach.detached_celeryd): execv_argv = [os.path.abspath(sys.argv[0]), "celeryd"] detached().execute_from_commandline(argv) django-celery-2.5.5/djcelery/management/commands/celeryd_multi.py0000644000076500000240000000130511721671621025621 0ustar asksolstaff00000000000000""" Utility to manage multiple :program:`celeryd` instances. """ from __future__ import absolute_import from celery.bin import celeryd_multi from djcelery.management.base import CeleryCommand class Command(CeleryCommand): """Run the celery daemon.""" args = "[name1, [name2, [...]> [worker options]" help = "Manage multiple Celery worker nodes." requires_model_validation = True options = () keep_base_opts = True def run_from_argv(self, argv): argv = self.handle_default_options(argv) argv.append("--cmd=%s celeryd_detach" % (argv[0], )) celeryd_multi.MultiTool().execute_from_commandline( ["%s %s" % (argv[0], argv[1])] + argv[2:]) django-celery-2.5.5/djcelery/management/commands/celeryev.py0000644000076500000240000000210011656051573024575 0ustar asksolstaff00000000000000""" Curses Celery Event Viewer. """ from __future__ import absolute_import from celery.bin import celeryev from djcelery.app import app from djcelery.management.base import CeleryCommand ev = celeryev.EvCommand(app=app) SS_TRANSPORTS = ["amqplib", "kombu.transport.pyamqplib", "redis", "kombu.transport.pyredis", "pika", "kombu.transport.pypika"] SS_COMPAT = """ ERROR: Snapshots not currently supported by %s transport. Please use one of: %s """ class Command(CeleryCommand): """Run the celery curses event viewer.""" options = (CeleryCommand.options + ev.get_options() + ev.preload_options) help = 'curses celery event viewer' def handle(self, *args, **options): """Handle the management command.""" transport = app.conf.BROKER_TRANSPORT or "amqplib" if options["camera"]: if transport not in SS_TRANSPORTS: self.die(SS_COMPAT % (transport, ", ".join(t for t in SS_TRANSPORTS if "." not in t))) ev.run(*args, **options) django-celery-2.5.5/djcelery/management/commands/celerymon.py0000644000076500000240000000171411656051573024766 0ustar asksolstaff00000000000000""" Start the celery clock service from the Django management command. """ from __future__ import absolute_import import sys from djcelery.app import app from djcelery.management.base import CeleryCommand try: from celerymon.bin.celerymon import MonitorCommand mon = MonitorCommand(app=app) except ImportError: mon = None MISSING = """ You don't have celerymon installed, please install it by running the following command: $ pip install -U celerymon or if you're still using easy_install (shame on you!) $ easy_install -U celerymon """ class Command(CeleryCommand): """Run the celery monitor.""" options = (CeleryCommand.options + (mon and mon.get_options() + mon.preload_options or ())) help = 'Run the celery monitor' def handle(self, *args, **options): """Handle the management command.""" if mon is None: sys.stderr.write(MISSING) else: mon.run(**options) django-celery-2.5.5/djcelery/management/commands/djcelerymon.py0000644000076500000240000000277711722707133025310 0ustar asksolstaff00000000000000from __future__ import absolute_import import sys import threading from celery.bin import celeryev from django.core.management.commands import runserver from djcelery.app import app from djcelery.management.base import CeleryCommand ev = celeryev.EvCommand(app=app) class WebserverThread(threading.Thread): def __init__(self, addrport="", *args, **options): threading.Thread.__init__(self) self.addrport = addrport self.args = args self.options = options def run(self): options = dict(self.options, use_reloader=False) command = runserver.Command() # see http://code.djangoproject.com/changeset/13319 command.stdout, command.stderr = sys.stdout, sys.stderr command.handle(self.addrport, *self.args, **options) class Command(CeleryCommand): """Run the celery curses event viewer.""" args = '[optional port number, or ipaddr:port]' options = (runserver.Command.option_list + ev.get_options() + ev.preload_options) help = 'Starts Django Admin instance and celerycam in the same process.' # see http://code.djangoproject.com/changeset/13319. stdout, stderr = sys.stdout, sys.stderr def handle(self, addrport="", *args, **options): """Handle the management command.""" server = WebserverThread(addrport, *args, **options) server.start() options["camera"] = "djcelery.snapshot.Camera" options["prog_name"] = "djcelerymon" ev.run(*args, **options) django-celery-2.5.5/djcelery/managers.py0000644000076500000240000001643011721671621020645 0ustar asksolstaff00000000000000from __future__ import absolute_import import warnings from functools import wraps from itertools import count from django.db import transaction, connection try: from django.db import connections, router except ImportError: # pre-Django 1.2 connections = router = None # noqa from django.db import models from django.db.models.query import QuerySet from django.conf import settings from celery.utils.timeutils import maybe_timedelta from .utils import now class TxIsolationWarning(UserWarning): pass def transaction_retry(max_retries=1): """Decorator for methods doing database operations. If the database operation fails, it will retry the operation at most ``max_retries`` times. """ def _outer(fun): @wraps(fun) def _inner(*args, **kwargs): _max_retries = kwargs.pop("exception_retry_count", max_retries) for retries in count(0): try: return fun(*args, **kwargs) except Exception: # pragma: no cover # Depending on the database backend used we can experience # various exceptions. E.g. psycopg2 raises an exception # if some operation breaks the transaction, so saving # the task result won't be possible until we rollback # the transaction. if retries >= _max_retries: raise transaction.rollback_unless_managed() return _inner return _outer def update_model_with_dict(obj, fields): [setattr(obj, attr_name, attr_value) for attr_name, attr_value in fields.items()] obj.save() return obj class ExtendedQuerySet(QuerySet): def update_or_create(self, **kwargs): obj, created = self.get_or_create(**kwargs) if not created: fields = dict(kwargs.pop("defaults", {})) fields.update(kwargs) update_model_with_dict(obj, fields) return obj class ExtendedManager(models.Manager): def get_query_set(self): return ExtendedQuerySet(self.model) def update_or_create(self, **kwargs): return self.get_query_set().update_or_create(**kwargs) def connection_for_write(self): if connections: return connections[router.db_for_write(self.model)] return connection def connection_for_read(self): if connections: return connections[self.db] return connection def current_engine(self): try: return settings.DATABASES[self.db]["ENGINE"] except AttributeError: return settings.DATABASE_ENGINE class ResultManager(ExtendedManager): def get_all_expired(self, expires): """Get all expired task results.""" return self.filter(date_done__lt=now() - maybe_timedelta(expires)) def delete_expired(self, expires): """Delete all expired taskset results.""" self.get_all_expired(expires).update(hidden=True) cursor = self.connection_for_write().cursor() cursor.execute("DELETE FROM %s WHERE hidden=%%s" % ( self.model._meta.db_table, ), (True, )) transaction.commit_unless_managed() class PeriodicTaskManager(ExtendedManager): def enabled(self): return self.filter(enabled=True) class TaskManager(ResultManager): """Manager for :class:`celery.models.Task` models.""" _last_id = None def get_task(self, task_id): """Get task meta for task by ``task_id``. :keyword exception_retry_count: How many times to retry by transaction rollback on exception. This could theoretically happen in a race condition if another worker is trying to create the same task. The default is to retry once. """ try: return self.get(task_id=task_id) except self.model.DoesNotExist: if self._last_id == task_id: self.warn_if_repeatable_read() self._last_id = task_id return self.model(task_id=task_id) @transaction_retry(max_retries=2) def store_result(self, task_id, result, status, traceback=None): """Store the result and status of a task. :param task_id: task id :param result: The return value of the task, or an exception instance raised by the task. :param status: Task status. See :meth:`celery.result.AsyncResult.get_status` for a list of possible status values. :keyword traceback: The traceback at the point of exception (if the task failed). :keyword exception_retry_count: How many times to retry by transaction rollback on exception. This could theoretically happen in a race condition if another worker is trying to create the same task. The default is to retry twice. """ return self.update_or_create(task_id=task_id, defaults={"status": status, "result": result, "traceback": traceback}) def warn_if_repeatable_read(self): if "mysql" in self.current_engine().lower(): cursor = self.connection_for_read().cursor() if cursor.execute("SELECT @@tx_isolation"): isolation = cursor.fetchone()[0] if isolation == 'REPEATABLE-READ': warnings.warn(TxIsolationWarning( "Polling results with transaction isolation level " "repeatable-read within the same transaction " "may give outdated results. Be sure to commit the " "transaction for each poll iteration.")) class TaskSetManager(ResultManager): """Manager for :class:`celery.models.TaskSet` models.""" def restore_taskset(self, taskset_id): """Get the async result instance by taskset id.""" try: return self.get(taskset_id=taskset_id) except self.model.DoesNotExist: pass def delete_taskset(self, taskset_id): """Delete a saved taskset result.""" s = self.restore_taskset(taskset_id) if s: s.delete() @transaction_retry(max_retries=2) def store_result(self, taskset_id, result): """Store the async result instance of a taskset. :param taskset_id: task set id :param result: The return value of the taskset """ return self.update_or_create(taskset_id=taskset_id, defaults={"result": result}) class TaskStateManager(ExtendedManager): def active(self): return self.filter(hidden=False) def expired(self, states, expires, nowfun=now): return self.filter(state__in=states, tstamp__lte=nowfun() - maybe_timedelta(expires)) def expire_by_states(self, states, expires): if expires is not None: return self.expired(states, expires).update(hidden=True) def purge(self): cursor = self.connection_for_write().cursor() cursor.execute("DELETE FROM %s WHERE hidden=%%s" % ( self.model._meta.db_table, ), (True, )) transaction.commit_unless_managed() django-celery-2.5.5/djcelery/migrations/0000755000076500000240000000000011744004562020645 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/migrations/0001_initial.py0000644000076500000240000003271211721671621023316 0ustar asksolstaff00000000000000# encoding: utf-8 from __future__ import absolute_import import datetime from south.db import db from south.v2 import SchemaMigration from django.db import models from django.db import DatabaseError class Migration(SchemaMigration): def forwards(self, orm): # Adding model 'TaskMeta' db.create_table('celery_taskmeta', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('task_id', self.gf('django.db.models.fields.CharField')(unique=True, max_length=255)), ('status', self.gf('django.db.models.fields.CharField')(default='PENDING', max_length=50)), ('result', self.gf('picklefield.fields.PickledObjectField')(default=None, null=True)), ('date_done', self.gf('django.db.models.fields.DateTimeField')(auto_now=True, blank=True)), ('traceback', self.gf('django.db.models.fields.TextField')(null=True, blank=True)),)) db.send_create_signal('djcelery', ['TaskMeta']) # Adding model 'TaskSetMeta' db.create_table('celery_tasksetmeta', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('taskset_id', self.gf('django.db.models.fields.CharField')(unique=True, max_length=255)), ('result', self.gf('picklefield.fields.PickledObjectField')()), ('date_done', self.gf('django.db.models.fields.DateTimeField')(auto_now=True, blank=True)),)) db.send_create_signal('djcelery', ['TaskSetMeta']) # Adding model 'IntervalSchedule' db.create_table('djcelery_intervalschedule', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('every', self.gf('django.db.models.fields.IntegerField')()), ('period', self.gf('django.db.models.fields.CharField')(max_length=24)),)) db.send_create_signal('djcelery', ['IntervalSchedule']) # Adding model 'CrontabSchedule' db.create_table('djcelery_crontabschedule', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('minute', self.gf('django.db.models.fields.CharField')(default='*', max_length=64)), ('hour', self.gf('django.db.models.fields.CharField')(default='*', max_length=64)), ('day_of_week', self.gf('django.db.models.fields.CharField')(default='*', max_length=64)),)) db.send_create_signal('djcelery', ['CrontabSchedule']) # Adding model 'PeriodicTasks' db.create_table('djcelery_periodictasks', ( ('ident', self.gf('django.db.models.fields.SmallIntegerField')(default=1, unique=True, primary_key=True)), ('last_update', self.gf('django.db.models.fields.DateTimeField')()),)) db.send_create_signal('djcelery', ['PeriodicTasks']) # Adding model 'PeriodicTask' db.create_table('djcelery_periodictask', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('name', self.gf('django.db.models.fields.CharField')(unique=True, max_length=200)), ('task', self.gf('django.db.models.fields.CharField')(max_length=200)), ('interval', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['djcelery.IntervalSchedule'], null=True, blank=True)), ('crontab', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['djcelery.CrontabSchedule'], null=True, blank=True)), ('args', self.gf('django.db.models.fields.TextField')(default='[]', blank=True)), ('kwargs', self.gf('django.db.models.fields.TextField')(default='{}', blank=True)), ('queue', self.gf('django.db.models.fields.CharField')(default=None, max_length=200, null=True, blank=True)), ('exchange', self.gf('django.db.models.fields.CharField')(default=None, max_length=200, null=True, blank=True)), ('routing_key', self.gf('django.db.models.fields.CharField')(default=None, max_length=200, null=True, blank=True)), ('expires', self.gf('django.db.models.fields.DateTimeField')(null=True, blank=True)), ('enabled', self.gf('django.db.models.fields.BooleanField')(default=True)), ('last_run_at', self.gf('django.db.models.fields.DateTimeField')(null=True, blank=True)), ('total_run_count', self.gf('django.db.models.fields.PositiveIntegerField')(default=0)), ('date_changed', self.gf('django.db.models.fields.DateTimeField')(auto_now=True, blank=True)),)) db.send_create_signal('djcelery', ['PeriodicTask']) # Adding model 'WorkerState' db.create_table('djcelery_workerstate', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('hostname', self.gf('django.db.models.fields.CharField')(unique=True, max_length=255)), ('last_heartbeat', self.gf('django.db.models.fields.DateTimeField')(null=True, db_index=True)),)) db.send_create_signal('djcelery', ['WorkerState']) # Adding model 'TaskState' db.create_table('djcelery_taskstate', ( ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), ('state', self.gf('django.db.models.fields.CharField')(max_length=64, db_index=True)), ('task_id', self.gf('django.db.models.fields.CharField')(unique=True, max_length=36)), ('name', self.gf('django.db.models.fields.CharField')(max_length=200, null=True, db_index=True)), ('tstamp', self.gf('django.db.models.fields.DateTimeField')(db_index=True)), ('args', self.gf('django.db.models.fields.TextField')(null=True)), ('kwargs', self.gf('django.db.models.fields.TextField')(null=True)), ('eta', self.gf('django.db.models.fields.DateTimeField')(null=True)), ('expires', self.gf('django.db.models.fields.DateTimeField')(null=True)), ('result', self.gf('django.db.models.fields.TextField')(null=True)), ('traceback', self.gf('django.db.models.fields.TextField')(null=True)), ('runtime', self.gf('django.db.models.fields.FloatField')(null=True)), ('retries', self.gf('django.db.models.fields.IntegerField')(default=0)), ('worker', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['djcelery.WorkerState'], null=True)), ('hidden', self.gf('django.db.models.fields.BooleanField')(default=False, db_index=True)),)) db.send_create_signal('djcelery', ['TaskState']) def backwards(self, orm): # Deleting model 'TaskMeta' db.delete_table('celery_taskmeta') # Deleting model 'TaskSetMeta' db.delete_table('celery_tasksetmeta') # Deleting model 'IntervalSchedule' db.delete_table('djcelery_intervalschedule') # Deleting model 'CrontabSchedule' db.delete_table('djcelery_crontabschedule') # Deleting model 'PeriodicTasks' db.delete_table('djcelery_periodictasks') # Deleting model 'PeriodicTask' db.delete_table('djcelery_periodictask') # Deleting model 'WorkerState' db.delete_table('djcelery_workerstate') # Deleting model 'TaskState' db.delete_table('djcelery_taskstate') models = { 'djcelery.crontabschedule': { 'Meta': {'object_name': 'CrontabSchedule'}, 'day_of_week': ('django.db.models.fields.CharField', [], {'default': "'*'", 'max_length': '64'}), 'hour': ('django.db.models.fields.CharField', [], {'default': "'*'", 'max_length': '64'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'minute': ('django.db.models.fields.CharField', [], {'default': "'*'", 'max_length': '64'}) }, 'djcelery.intervalschedule': { 'Meta': {'object_name': 'IntervalSchedule'}, 'every': ('django.db.models.fields.IntegerField', [], {}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'period': ('django.db.models.fields.CharField', [], {'max_length': '24'}) }, 'djcelery.periodictask': { 'Meta': {'object_name': 'PeriodicTask'}, 'args': ('django.db.models.fields.TextField', [], {'default': "'[]'", 'blank': 'True'}), 'crontab': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['djcelery.CrontabSchedule']", 'null': 'True', 'blank': 'True'}), 'date_changed': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), 'enabled': ('django.db.models.fields.BooleanField', [], {'default': 'True'}), 'exchange': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '200', 'null': 'True', 'blank': 'True'}), 'expires': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'blank': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'interval': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['djcelery.IntervalSchedule']", 'null': 'True', 'blank': 'True'}), 'kwargs': ('django.db.models.fields.TextField', [], {'default': "'{}'", 'blank': 'True'}), 'last_run_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'blank': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}), 'queue': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '200', 'null': 'True', 'blank': 'True'}), 'routing_key': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '200', 'null': 'True', 'blank': 'True'}), 'task': ('django.db.models.fields.CharField', [], {'max_length': '200'}), 'total_run_count': ('django.db.models.fields.PositiveIntegerField', [], {'default': '0'}) }, 'djcelery.periodictasks': { 'Meta': {'object_name': 'PeriodicTasks'}, 'ident': ('django.db.models.fields.SmallIntegerField', [], {'default': '1', 'unique': 'True', 'primary_key': 'True'}), 'last_update': ('django.db.models.fields.DateTimeField', [], {}) }, 'djcelery.taskmeta': { 'Meta': {'object_name': 'TaskMeta', 'db_table': "'celery_taskmeta'"}, 'date_done': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'result': ('picklefield.fields.PickledObjectField', [], {'default': 'None', 'null': 'True'}), 'status': ('django.db.models.fields.CharField', [], {'default': "'PENDING'", 'max_length': '50'}), 'task_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}), 'traceback': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}) }, 'djcelery.tasksetmeta': { 'Meta': {'object_name': 'TaskSetMeta', 'db_table': "'celery_tasksetmeta'"}, 'date_done': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'result': ('picklefield.fields.PickledObjectField', [], {}), 'taskset_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}) }, 'djcelery.taskstate': { 'Meta': {'ordering': "['-tstamp']", 'object_name': 'TaskState'}, 'args': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'eta': ('django.db.models.fields.DateTimeField', [], {'null': 'True'}), 'expires': ('django.db.models.fields.DateTimeField', [], {'null': 'True'}), 'hidden': ('django.db.models.fields.BooleanField', [], {'default': 'False', 'db_index': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'kwargs': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'max_length': '200', 'null': 'True', 'db_index': 'True'}), 'result': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'retries': ('django.db.models.fields.IntegerField', [], {'default': '0'}), 'runtime': ('django.db.models.fields.FloatField', [], {'null': 'True'}), 'state': ('django.db.models.fields.CharField', [], {'max_length': '64', 'db_index': 'True'}), 'task_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'}), 'traceback': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'tstamp': ('django.db.models.fields.DateTimeField', [], {'db_index': 'True'}), 'worker': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['djcelery.WorkerState']", 'null': 'True'}) }, 'djcelery.workerstate': { 'Meta': {'ordering': "['-last_heartbeat']", 'object_name': 'WorkerState'}, 'hostname': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'last_heartbeat': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}) } } complete_apps = ['djcelery'] django-celery-2.5.5/djcelery/migrations/0002_v25_changes.py0000644000076500000240000001716311721673634024003 0ustar asksolstaff00000000000000# encoding: utf-8 from __future__ import absolute_import import datetime from south.db import db from south.v2 import SchemaMigration from django.db import models class Migration(SchemaMigration): def forwards(self, orm): # Adding field 'PeriodicTask.description' db.add_column('djcelery_periodictask', 'description', self.gf('django.db.models.fields.TextField')(default='', blank=True), keep_default=False) # Adding field 'TaskMeta.hidden' db.add_column('celery_taskmeta', 'hidden', self.gf('django.db.models.fields.BooleanField')(default=False, db_index=True), keep_default=False) # Adding field 'TaskSetMeta.hidden' db.add_column('celery_tasksetmeta', 'hidden', self.gf('django.db.models.fields.BooleanField')(default=False, db_index=True), keep_default=False) def backwards(self, orm): # Deleting field 'PeriodicTask.description' db.delete_column('djcelery_periodictask', 'description') # Deleting field 'TaskMeta.hidden' db.delete_column('celery_taskmeta', 'hidden') # Deleting field 'TaskSetMeta.hidden' db.delete_column('celery_tasksetmeta', 'hidden') models = { 'djcelery.crontabschedule': { 'Meta': {'object_name': 'CrontabSchedule'}, 'day_of_week': ('django.db.models.fields.CharField', [], {'default': "'*'", 'max_length': '64'}), 'hour': ('django.db.models.fields.CharField', [], {'default': "'*'", 'max_length': '64'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'minute': ('django.db.models.fields.CharField', [], {'default': "'*'", 'max_length': '64'}) }, 'djcelery.intervalschedule': { 'Meta': {'object_name': 'IntervalSchedule'}, 'every': ('django.db.models.fields.IntegerField', [], {}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'period': ('django.db.models.fields.CharField', [], {'max_length': '24'}) }, 'djcelery.periodictask': { 'Meta': {'object_name': 'PeriodicTask'}, 'args': ('django.db.models.fields.TextField', [], {'default': "'[]'", 'blank': 'True'}), 'crontab': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['djcelery.CrontabSchedule']", 'null': 'True', 'blank': 'True'}), 'date_changed': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), 'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}), 'enabled': ('django.db.models.fields.BooleanField', [], {'default': 'True'}), 'exchange': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '200', 'null': 'True', 'blank': 'True'}), 'expires': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'blank': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'interval': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['djcelery.IntervalSchedule']", 'null': 'True', 'blank': 'True'}), 'kwargs': ('django.db.models.fields.TextField', [], {'default': "'{}'", 'blank': 'True'}), 'last_run_at': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'blank': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}), 'queue': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '200', 'null': 'True', 'blank': 'True'}), 'routing_key': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '200', 'null': 'True', 'blank': 'True'}), 'task': ('django.db.models.fields.CharField', [], {'max_length': '200'}), 'total_run_count': ('django.db.models.fields.PositiveIntegerField', [], {'default': '0'}) }, 'djcelery.periodictasks': { 'Meta': {'object_name': 'PeriodicTasks'}, 'ident': ('django.db.models.fields.SmallIntegerField', [], {'default': '1', 'unique': 'True', 'primary_key': 'True'}), 'last_update': ('django.db.models.fields.DateTimeField', [], {}) }, 'djcelery.taskmeta': { 'Meta': {'object_name': 'TaskMeta', 'db_table': "'celery_taskmeta'"}, 'date_done': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), 'hidden': ('django.db.models.fields.BooleanField', [], {'default': 'False', 'db_index': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'result': ('picklefield.fields.PickledObjectField', [], {'default': 'None', 'null': 'True'}), 'status': ('django.db.models.fields.CharField', [], {'default': "'PENDING'", 'max_length': '50'}), 'task_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}), 'traceback': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}) }, 'djcelery.tasksetmeta': { 'Meta': {'object_name': 'TaskSetMeta', 'db_table': "'celery_tasksetmeta'"}, 'date_done': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), 'hidden': ('django.db.models.fields.BooleanField', [], {'default': 'False', 'db_index': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'result': ('picklefield.fields.PickledObjectField', [], {}), 'taskset_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}) }, 'djcelery.taskstate': { 'Meta': {'ordering': "['-tstamp']", 'object_name': 'TaskState'}, 'args': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'eta': ('django.db.models.fields.DateTimeField', [], {'null': 'True'}), 'expires': ('django.db.models.fields.DateTimeField', [], {'null': 'True'}), 'hidden': ('django.db.models.fields.BooleanField', [], {'default': 'False', 'db_index': 'True'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'kwargs': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'name': ('django.db.models.fields.CharField', [], {'max_length': '200', 'null': 'True', 'db_index': 'True'}), 'result': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'retries': ('django.db.models.fields.IntegerField', [], {'default': '0'}), 'runtime': ('django.db.models.fields.FloatField', [], {'null': 'True'}), 'state': ('django.db.models.fields.CharField', [], {'max_length': '64', 'db_index': 'True'}), 'task_id': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '36'}), 'traceback': ('django.db.models.fields.TextField', [], {'null': 'True'}), 'tstamp': ('django.db.models.fields.DateTimeField', [], {'db_index': 'True'}), 'worker': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['djcelery.WorkerState']", 'null': 'True'}) }, 'djcelery.workerstate': { 'Meta': {'ordering': "['-last_heartbeat']", 'object_name': 'WorkerState'}, 'hostname': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '255'}), 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), 'last_heartbeat': ('django.db.models.fields.DateTimeField', [], {'null': 'True', 'db_index': 'True'}) } } complete_apps = ['djcelery'] django-celery-2.5.5/djcelery/migrations/__init__.py0000644000076500000240000000000011656051573022752 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/models.py0000644000076500000240000003024711742044217020333 0ustar asksolstaff00000000000000from __future__ import absolute_import from datetime import timedelta from time import time, mktime import django from django.conf import settings from django.core.exceptions import MultipleObjectsReturned, ValidationError from django.db import models from django.db.models import signals from django.utils.translation import ugettext_lazy as _ from picklefield.fields import PickledObjectField from celery import schedules from celery import states from celery.utils.timeutils import timedelta_seconds from . import managers from .utils import now HEARTBEAT_EXPIRE = 150 # 2 minutes, 30 seconds TASK_STATE_CHOICES = zip(states.ALL_STATES, states.ALL_STATES) class TaskMeta(models.Model): """Task result/status.""" task_id = models.CharField(_(u"task id"), max_length=255, unique=True) status = models.CharField(_(u"task status"), max_length=50, default=states.PENDING, choices=TASK_STATE_CHOICES) result = PickledObjectField(null=True, default=None) date_done = models.DateTimeField(_(u"done at"), auto_now=True) traceback = models.TextField(_(u"traceback"), blank=True, null=True) hidden = models.BooleanField(editable=False, default=False, db_index=True) objects = managers.TaskManager() class Meta: """Model meta-data.""" verbose_name = _(u"task meta") verbose_name_plural = _(u"task meta") db_table = "celery_taskmeta" def to_dict(self): return {"task_id": self.task_id, "status": self.status, "result": self.result, "date_done": self.date_done, "traceback": self.traceback} def __unicode__(self): return u"%s>" % (self.task_id, self.status) class TaskSetMeta(models.Model): """TaskSet result""" taskset_id = models.CharField(_(u"task id"), max_length=255, unique=True) result = PickledObjectField() date_done = models.DateTimeField(_(u"done at"), auto_now=True) hidden = models.BooleanField(editable=False, default=False, db_index=True) objects = managers.TaskSetManager() class Meta: """Model meta-data.""" verbose_name = _(u"taskset meta") verbose_name_plural = _(u"taskset meta") db_table = "celery_tasksetmeta" def to_dict(self): return {"taskset_id": self.taskset_id, "result": self.result, "date_done": self.date_done} def __unicode__(self): return u"" % (self.taskset_id) PERIOD_CHOICES = (("days", _(u"Days")), ("hours", _(u"Hours")), ("minutes", _(u"Minutes")), ("seconds", _(u"Seconds")), ("microseconds", _(u"Microseconds"))) class IntervalSchedule(models.Model): every = models.IntegerField(_(u"every"), null=False) period = models.CharField(_(u"period"), max_length=24, choices=PERIOD_CHOICES) class Meta: verbose_name = _(u"interval") verbose_name_plural = _(u"intervals") @property def schedule(self): return schedules.schedule(timedelta(**{self.period: self.every})) @classmethod def from_schedule(cls, schedule, period="seconds"): every = timedelta_seconds(schedule.run_every) try: return cls.objects.get(every=every, period=period) except cls.DoesNotExist: return cls(every=every, period=period) except MultipleObjectsReturned: cls.objects.filter(every=every, period=period).delete() return cls(every=every, period=period) def __unicode__(self): if self.every == 1: return _(u"every %(period)s") % {"period": self.period[:-1]} return _(u"every %(every)s %(period)s") % {"every": self.every, "period": self.period} class CrontabSchedule(models.Model): minute = models.CharField(_(u"minute"), max_length=64, default="*") hour = models.CharField(_(u"hour"), max_length=64, default="*") day_of_week = models.CharField(_(u"day of week"), max_length=64, default="*") class Meta: verbose_name = _(u"crontab") verbose_name_plural = _(u"crontabs") def __unicode__(self): rfield = lambda f: f and str(f).replace(" ", "") or "*" return u"%s %s %s (m/h/d)" % (rfield(self.minute), rfield(self.hour), rfield(self.day_of_week)) @property def schedule(self): return schedules.crontab(minute=self.minute, hour=self.hour, day_of_week=self.day_of_week, nowfun=now) @classmethod def from_schedule(cls, schedule): return cls(minute=schedule._orig_minute, hour=schedule._orig_hour, day_of_week=schedule._orig_day_of_week) class PeriodicTasks(models.Model): ident = models.SmallIntegerField(default=1, primary_key=True, unique=True) last_update = models.DateTimeField(null=False) objects = managers.ExtendedManager() @classmethod def changed(cls, instance, **kwargs): if not instance.no_changes: cls.objects.update_or_create(ident=1, defaults={"last_update": now()}) @classmethod def last_change(cls): try: return cls.objects.get(ident=1).last_update except cls.DoesNotExist: pass class PeriodicTask(models.Model): name = models.CharField(_(u"name"), max_length=200, unique=True, help_text=_(u"Useful description")) task = models.CharField(_(u"task name"), max_length=200) interval = models.ForeignKey(IntervalSchedule, null=True, blank=True, verbose_name=_(u"interval")) crontab = models.ForeignKey(CrontabSchedule, null=True, blank=True, verbose_name=_(u"crontab"), help_text=_(u"Use one of interval/crontab")) args = models.TextField(_(u"Arguments"), blank=True, default="[]", help_text=_(u"JSON encoded positional arguments")) kwargs = models.TextField(_(u"Keyword arguments"), blank=True, default="{}", help_text=_("JSON encoded keyword arguments")) queue = models.CharField(_("queue"), max_length=200, blank=True, null=True, default=None, help_text=_(u"Queue defined in CELERY_QUEUES")) exchange = models.CharField(_(u"exchange"), max_length=200, blank=True, null=True, default=None) routing_key = models.CharField(_(u"routing key"), max_length=200, blank=True, null=True, default=None) expires = models.DateTimeField(_(u"expires"), blank=True, null=True) enabled = models.BooleanField(_(u"enabled"), default=True) last_run_at = models.DateTimeField( auto_now=False, auto_now_add=False, editable=False, blank=True, null=True) total_run_count = models.PositiveIntegerField(default=0, editable=False) date_changed = models.DateTimeField(auto_now=True) description = models.TextField(_("description"), blank=True) objects = managers.PeriodicTaskManager() no_changes = False class Meta: verbose_name = _(u"periodic task") verbose_name_plural = _(u"periodic tasks") def validate_unique(self, *args, **kwargs): super(PeriodicTask, self).validate_unique(*args, **kwargs) if not self.interval and not self.crontab: raise ValidationError( {"interval": ["One of interval or crontab must be set."]}) if self.interval and self.crontab: raise ValidationError( {"crontab": ["Only one of interval or crontab must be set"]}) def save(self, *args, **kwargs): self.exchange = self.exchange or None self.routing_key = self.routing_key or None self.queue = self.queue or None super(PeriodicTask, self).save(*args, **kwargs) def __unicode__(self): if self.interval: return u"%s: %s" % (self.name, unicode(self.interval)) if self.crontab: return u"%s: %s" % (self.name, unicode(self.crontab)) return u"%s: {no schedule}" % (self.name, ) @property def schedule(self): if self.interval: return self.interval.schedule if self.crontab: return self.crontab.schedule signals.pre_delete.connect(PeriodicTasks.changed, sender=PeriodicTask) signals.pre_save.connect(PeriodicTasks.changed, sender=PeriodicTask) class WorkerState(models.Model): hostname = models.CharField(_(u"hostname"), max_length=255, unique=True) last_heartbeat = models.DateTimeField(_(u"last heartbeat"), null=True, db_index=True) objects = managers.ExtendedManager() class Meta: """Model meta-data.""" verbose_name = _(u"worker") verbose_name_plural = _(u"workers") get_latest_by = "last_heartbeat" ordering = ["-last_heartbeat"] def __unicode__(self): return self.hostname def __repr__(self): return "" % (self.hostname, ) def is_alive(self): if self.last_heartbeat: return time() < self.heartbeat_timestamp + HEARTBEAT_EXPIRE return False @property def heartbeat_timestamp(self): return mktime(self.last_heartbeat.timetuple()) class TaskState(models.Model): state = models.CharField(_(u"state"), max_length=64, choices=TASK_STATE_CHOICES, db_index=True) task_id = models.CharField(_(u"UUID"), max_length=36, unique=True) name = models.CharField(_(u"name"), max_length=200, null=True, db_index=True) tstamp = models.DateTimeField(_(u"event received at"), db_index=True) args = models.TextField(_(u"Arguments"), null=True) kwargs = models.TextField(_(u"Keyword arguments"), null=True) eta = models.DateTimeField(_(u"ETA"), null=True, help_text=u"date to execute") expires = models.DateTimeField(_(u"expires"), null=True) result = models.TextField(_(u"result"), null=True) traceback = models.TextField(_(u"traceback"), null=True) runtime = models.FloatField(_(u"execution time"), null=True, help_text=_(u"in seconds if task successful")) retries = models.IntegerField(_(u"number of retries"), default=0) worker = models.ForeignKey(WorkerState, null=True, verbose_name=_("worker")) hidden = models.BooleanField(editable=False, default=False, db_index=True) objects = managers.TaskStateManager() class Meta: """Model meta-data.""" verbose_name = _(u"task") verbose_name_plural = _(u"tasks") get_latest_by = "tstamp" ordering = ["-tstamp"] def __unicode__(self): name = self.name or "UNKNOWN" s = u"%s %s %s" % (self.state.ljust(10), self.task_id.ljust(36), name) if self.eta: s += u" eta:%s" % (self.eta, ) return s def __repr__(self): return "" % (self.state, self.name or "UNKNOWN", self.task_id, self.tstamp) if (django.VERSION[0], django.VERSION[1]) >= (1, 1): # Keep models away from syncdb/reset if Django database # backend is not being used. if "database" not in (getattr(settings, "CELERY_RESULT_BACKEND", None) or "database"): for result_model in (TaskMeta, TaskSetMeta): result_model._meta.managed = False django-celery-2.5.5/djcelery/mon.py0000644000076500000240000000432211741563541017641 0ustar asksolstaff00000000000000from __future__ import absolute_import import os import sys import types from celery.app.defaults import str_to_bool from celery.utils import import_from_cwd DEFAULT_APPS = ("django.contrib.auth", "django.contrib.contenttypes", "django.contrib.sessions", "django.contrib.admin", "django.contrib.admindocs", "djcelery") DEFAULTS = {"ROOT_URLCONF": "djcelery.monproj.urls", "DATABASE_ENGINE": "sqlite3", "DATABASE_NAME": "djcelerymon.db", "DATABASES": {"default": { "ENGINE": "django.db.backends.sqlite3", "NAME": "djcelerymon.db"}}, "BROKER_URL": "amqp://", "SITE_ID": 1, "INSTALLED_APPS": DEFAULT_APPS, "DEBUG": str_to_bool(os.environ.get("DJCELERYMON_DEBUG", "0"))} def default_settings(name="__default_settings__"): c = type(name, (types.ModuleType, ), DEFAULTS)(name) c.__dict__.update({"__file__": __file__}) sys.modules[name] = c return name def configure(): from celery import current_app from celery.loaders.default import DEFAULT_CONFIG_MODULE from django.conf import settings app = current_app conf = {} if not settings.configured: if "loader" in app.__dict__ and app.loader.configured: conf = current_app.loader.conf else: os.environ.pop("CELERY_LOADER", None) settings_module = os.environ.get("CELERY_CONFIG_MODULE", DEFAULT_CONFIG_MODULE) try: import_from_cwd(settings_module) except ImportError: settings_module = default_settings() settings.configure(SETTINGS_MODULE=settings_module, **dict(DEFAULTS, **conf)) def run_monitor(argv): from .management.commands import djcelerymon djcelerymon.Command().run_from_argv([argv[0], "djcelerymon"] + argv[1:]) def main(argv=sys.argv): from django.core import management os.environ["CELERY_LOADER"] = "default" configure() management.call_command("syncdb") run_monitor(argv) if __name__ == "__main__": main() django-celery-2.5.5/djcelery/monproj/0000755000076500000240000000000011744004562020155 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/monproj/__init__.py0000644000076500000240000000000011656051573022262 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/monproj/urls.py0000644000076500000240000000074611656051573021531 0ustar asksolstaff00000000000000from __future__ import absolute_import from django.conf.urls.defaults import (patterns, include, url, # noqa handler500, handler404) from django.contrib import admin admin.autodiscover() urlpatterns = patterns('', # Uncomment the admin/doc line below and add 'django.contrib.admindocs' # to INSTALLED_APPS to enable admin documentation: (r'^doc/', include('django.contrib.admindocs.urls')), (r'', include(admin.site.urls)), ) django-celery-2.5.5/djcelery/schedulers.py0000644000076500000240000001726411744004127021213 0ustar asksolstaff00000000000000from __future__ import absolute_import import logging from warnings import warn from anyjson import deserialize, serialize from celery import schedules from celery.beat import Scheduler, ScheduleEntry from celery.utils.encoding import safe_str, safe_repr from kombu.utils.finalize import Finalize from django.db import transaction from django.core.exceptions import ObjectDoesNotExist from .models import (PeriodicTask, PeriodicTasks, CrontabSchedule, IntervalSchedule) from .utils import DATABASE_ERRORS, now class ModelEntry(ScheduleEntry): model_schedules = ((schedules.crontab, CrontabSchedule, "crontab"), (schedules.schedule, IntervalSchedule, "interval")) save_fields = ["last_run_at", "total_run_count", "no_changes"] def __init__(self, model): self.name = model.name self.task = model.task self.schedule = model.schedule try: self.args = deserialize(model.args or u"[]") self.kwargs = deserialize(model.kwargs or u"{}") except ValueError: # disable because of error deserializing args/kwargs model.no_changes = True model.enabled = False model.save() raise self.options = {"queue": model.queue, "exchange": model.exchange, "routing_key": model.routing_key, "expires": model.expires} self.total_run_count = model.total_run_count self.model = model if not model.last_run_at: model.last_run_at = self._default_now() self.last_run_at = model.last_run_at def is_due(self): if not self.model.enabled: return False, 5.0 # 5 second delay for re-enable. return self.schedule.is_due(self.last_run_at) def _default_now(self): return now() def next(self): self.model.last_run_at = now() self.model.total_run_count += 1 self.model.no_changes = True return self.__class__(self.model) __next__ = next # for 2to3 def save(self): # Object may not be synchronized, so only # change the fields we care about. obj = self.model._default_manager.get(pk=self.model.pk) for field in self.save_fields: setattr(obj, field, getattr(self.model, field)) obj.save() @classmethod def to_model_schedule(cls, schedule): for schedule_type, model_type, model_field in cls.model_schedules: schedule = schedules.maybe_schedule(schedule) if isinstance(schedule, schedule_type): model_schedule = model_type.from_schedule(schedule) model_schedule.save() return model_schedule, model_field raise ValueError("Can't convert schedule type %r to model" % schedule) @classmethod def from_entry(cls, name, skip_fields=("relative", "options"), **entry): options = entry.get("options") or {} fields = dict(entry) for skip_field in skip_fields: fields.pop(skip_field, None) schedule = fields.pop("schedule") model_schedule, model_field = cls.to_model_schedule(schedule) fields[model_field] = model_schedule fields["args"] = serialize(fields.get("args") or []) fields["kwargs"] = serialize(fields.get("kwargs") or {}) fields["queue"] = options.get("queue") fields["exchange"] = options.get("exchange") fields["routing_key"] = options.get("routing_key") return cls(PeriodicTask._default_manager.update_or_create(name=name, defaults=fields)) def __repr__(self): return "" % (safe_str(self.name), self.task, safe_repr(self.args), safe_repr(self.kwargs), self.schedule) class DatabaseScheduler(Scheduler): Entry = ModelEntry Model = PeriodicTask Changes = PeriodicTasks _schedule = None _last_timestamp = None def __init__(self, *args, **kwargs): self._dirty = set() self._finalize = Finalize(self, self.sync, exitpriority=5) Scheduler.__init__(self, *args, **kwargs) self.max_interval = 5 def setup_schedule(self): self.install_default_entries(self.schedule) self.update_from_dict(self.app.conf.CELERYBEAT_SCHEDULE) def all_as_schedule(self): self.logger.debug("DatabaseScheduler: Fetching database schedule") s = {} for model in self.Model.objects.enabled(): try: s[model.name] = self.Entry(model) except ValueError: pass return s def schedule_changed(self): if self._last_timestamp is not None: try: # If MySQL is running with transaction isolation level # REPEATABLE-READ (default), then we won't see changes done by # other transactions until the current transaction is # committed (Issue #41). try: transaction.commit() except transaction.TransactionManagementError: pass # not in transaction management. ts = self.Changes.last_change() if not ts or ts < self._last_timestamp: return False except DATABASE_ERRORS, exc: warn(RuntimeWarning("Database gave error: %r" % (exc, ))) return False self._last_timestamp = now() return True def reserve(self, entry): new_entry = Scheduler.reserve(self, entry) # Need to store entry by name, because the entry may change # in the mean time. self._dirty.add(new_entry.name) return new_entry @transaction.commit_manually def sync(self): self.logger.debug("Writing dirty entries...") try: while self._dirty: try: name = self._dirty.pop() self.schedule[name].save() except (KeyError, ObjectDoesNotExist): pass except: transaction.rollback() raise else: transaction.commit() def update_from_dict(self, dict_): s = {} for name, entry in dict_.items(): try: s[name] = self.Entry.from_entry(name, **entry) except Exception, exc: self.logger.error( "Couldn't add entry %r to database schedule: %r. " "Contents: %r" % (name, exc, entry)) self.schedule.update(s) def install_default_entries(self, data): entries = {} if self.app.conf.CELERY_TASK_RESULT_EXPIRES: entries.setdefault("celery.backend_cleanup", { "task": "celery.backend_cleanup", "schedule": schedules.crontab("0", "4", "*", nowfun=now), "options": {"expires": 12 * 3600}}) self.update_from_dict(entries) def get_schedule(self): if self.schedule_changed(): self.sync() self.logger.debug("DatabaseScheduler: Schedule changed.") self._schedule = self.all_as_schedule() if self.logger.isEnabledFor(logging.DEBUG): self.logger.debug( "Current schedule:\n" + "\n".join(repr(entry) for entry in self._schedule.values())) return self._schedule django-celery-2.5.5/djcelery/snapshot.py0000644000076500000240000001200111721671621020675 0ustar asksolstaff00000000000000from __future__ import absolute_import from collections import defaultdict from datetime import datetime, timedelta from time import time from django.db import transaction from django.conf import settings from django.core.exceptions import ObjectDoesNotExist from celery import states from celery.events.state import Task from celery.events.snapshot import Polaroid from celery.utils.timeutils import maybe_iso8601 from .models import WorkerState, TaskState from .utils import make_aware WORKER_UPDATE_FREQ = 60 # limit worker timestamp write freq. SUCCESS_STATES = frozenset([states.SUCCESS]) # Expiry can be timedelta or None for never expire. EXPIRE_SUCCESS = getattr(settings, "CELERYCAM_EXPIRE_SUCCESS", timedelta(days=1)) EXPIRE_ERROR = getattr(settings, "CELERYCAM_EXPIRE_ERROR", timedelta(days=3)) EXPIRE_PENDING = getattr(settings, "CELERYCAM_EXPIRE_PENDING", timedelta(days=5)) class Camera(Polaroid): TaskState = TaskState WorkerState = WorkerState clear_after = True worker_update_freq = WORKER_UPDATE_FREQ expire_states = { SUCCESS_STATES: EXPIRE_SUCCESS, states.EXCEPTION_STATES: EXPIRE_ERROR, states.UNREADY_STATES: EXPIRE_PENDING, } def __init__(self, *args, **kwargs): super(Camera, self).__init__(*args, **kwargs) self._last_worker_write = defaultdict(lambda: (None, None)) def get_heartbeat(self, worker): try: heartbeat = worker.heartbeats[-1] except IndexError: return return make_aware(datetime.fromtimestamp(heartbeat)) def handle_worker(self, (hostname, worker)): last_write, obj = self._last_worker_write[hostname] if not last_write or time() - last_write > self.worker_update_freq: obj = self.WorkerState.objects.update_or_create( hostname=hostname, defaults={"last_heartbeat": self.get_heartbeat(worker)}) self._last_worker_write[hostname] = (time(), obj) return obj def handle_task(self, (uuid, task), worker=None): if task.worker and task.worker.hostname: worker = self.handle_worker((task.worker.hostname, task.worker)) return self.update_task(task.state, task_id=uuid, defaults={"name": task.name, "args": task.args, "kwargs": task.kwargs, "eta": maybe_iso8601(task.eta), "expires": maybe_iso8601(task.expires), "state": task.state, "tstamp": make_aware(datetime.fromtimestamp( task.timestamp)), "result": task.result or task.exception, "traceback": task.traceback, "runtime": task.runtime, "worker": worker}) def update_task(self, state, **kwargs): objects = self.TaskState.objects defaults = kwargs.pop("defaults", None) or {} try: obj = objects.get(**kwargs) except ObjectDoesNotExist: if not defaults.get("name"): return obj, created = objects.get_or_create(defaults=defaults, **kwargs) return obj else: if states.state(state) < states.state(obj.state): keep = Task.merge_rules[states.RECEIVED] defaults = dict((k, v) for k, v in defaults.items() if k not in keep) for k, v in defaults.items(): setattr(obj, k, v) obj.save() return obj def _autocommit(self, fun): try: fun() except (KeyboardInterrupt, SystemExit): transaction.commit() raise except Exception: transaction.rollback() raise else: transaction.commit() @transaction.commit_manually def on_shutter(self, state, commit_every=100): if not state.event_count: transaction.commit() return def _handle_tasks(): for i, task in enumerate(state.tasks.items()): self.handle_task(task) if not i % commit_every: transaction.commit() self._autocommit(lambda: map(self.handle_worker, state.workers.items())) self._autocommit(_handle_tasks) def on_cleanup(self): dirty = sum(self.TaskState.objects.expire_by_states(states, expires) for states, expires in self.expire_states.items()) if dirty: self.logger.debug( "Cleanup: Marked %s objects as dirty." % (dirty, )) self.TaskState.objects.purge() self.logger.debug("Cleanup: %s objects purged." % (dirty, )) return dirty return 0 django-celery-2.5.5/djcelery/templates/0000755000076500000240000000000011744004562020467 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/templates/admin/0000755000076500000240000000000011744004562021557 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/templates/admin/djcelery/0000755000076500000240000000000011744004562023360 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/templates/admin/djcelery/change_list.html0000644000076500000240000000124311656051573026534 0ustar asksolstaff00000000000000{% extends "admin/change_list.html" %} {% load i18n %} {% block breadcrumbs %}
{% trans "Home" %} › {{ app_label|capfirst }} › {{ cl.opts.verbose_name_plural|capfirst }}
{% if wrong_scheduler %}
  • Periodic tasks won't be dispatched unless you set the CELERYBEAT_SCHEDULER setting to djcelery.schedulers.DatabaseScheduler, or specify it using the -S option to celerybeat
{% endif %} {% endblock %} django-celery-2.5.5/djcelery/templates/djcelery/0000755000076500000240000000000011744004562022270 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/templates/djcelery/confirm_rate_limit.html0000644000076500000240000000150411656051573027032 0ustar asksolstaff00000000000000{% extends "admin/base_site.html" %} {% load i18n %} {% block breadcrumbs %}
{% trans "Home" %} › {{ app_label|capfirst }} › {{ opts.verbose_name_plural|capfirst }} › {% trans 'Rate limit selected tasks' %}
{% endblock %} {% block content %}
{% csrf_token %}
{% for obj in queryset %} {% endfor %}
{% endblock %} django-celery-2.5.5/djcelery/tests/0000755000076500000240000000000011744004562017633 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/tests/__init__.py0000644000076500000240000000000011703564016021732 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/tests/req.py0000644000076500000240000000451411742037755021010 0ustar asksolstaff00000000000000from __future__ import absolute_import from django.test import Client from django.core.handlers.wsgi import WSGIRequest from django.core.handlers.base import BaseHandler from celery.utils.compat import WhateverIO class RequestFactory(Client): """Class that lets you create mock Request objects for use in testing. Usage: rf = RequestFactory() get_request = rf.get('/hello/') post_request = rf.post('/submit/', {'foo': 'bar'}) This class re-uses the django.test.client.Client interface, docs here: http://www.djangoproject.com/documentation/testing/#the-test-client Once you have a request object you can pass it to any view function, just as if that view had been hooked up using a URLconf. """ def request(self, **request): """Similar to parent class, but returns the request object as soon as it has created it.""" environ = { 'HTTP_COOKIE': self.cookies, 'HTTP_USER_AGENT': 'Django UnitTest Client 1.0', 'REMOTE_ADDR': '127.0.0.1', 'PATH_INFO': '/', 'QUERY_STRING': '', 'REQUEST_METHOD': 'GET', 'SCRIPT_NAME': '', 'SERVER_NAME': 'testserver', 'SERVER_PORT': 80, 'SERVER_PROTOCOL': 'HTTP/1.1', 'wsgi.input': WhateverIO(), } environ.update(self.defaults) environ.update(request) return WSGIRequest(environ) class MockRequest(object): def __init__(self): handler = BaseHandler() handler.load_middleware() self.request_factory = RequestFactory() self.middleware = handler._request_middleware def _make_request(self, request_method, *args, **kwargs): request_method_handler = getattr(self.request_factory, request_method) request = request_method_handler(*args, **kwargs) [middleware_processor(request) for middleware_processor in self.middleware] return request def get(self, *args, **kwargs): return self._make_request("get", *args, **kwargs) def post(self, *args, **kwargs): return self._make_request("post", *args, **kwargs) def put(self, *args, **kwargs): return self._make_request("put", *args, **kwargs) def delete(self, *args, **kwargs): return self._make_request("delete", *args, **kwargs) django-celery-2.5.5/djcelery/tests/test_backends/0000755000076500000240000000000011744004562022444 5ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/tests/test_backends/__init__.py0000644000076500000240000000000011656051573024551 0ustar asksolstaff00000000000000django-celery-2.5.5/djcelery/tests/test_backends/test_cache.py0000644000076500000240000001160011656051573025124 0ustar asksolstaff00000000000000from __future__ import absolute_import import sys from datetime import timedelta from django.core.cache.backends.base import InvalidCacheBackendError from celery import result from celery import states from celery.utils import gen_unique_id from celery.datastructures import ExceptionInfo from djcelery.backends.cache import CacheBackend from djcelery.tests.utils import unittest class SomeClass(object): def __init__(self, data): self.data = data class test_CacheBackend(unittest.TestCase): def test_mark_as_done(self): cb = CacheBackend() tid = gen_unique_id() self.assertEqual(cb.get_status(tid), states.PENDING) self.assertIsNone(cb.get_result(tid)) cb.mark_as_done(tid, 42) self.assertEqual(cb.get_status(tid), states.SUCCESS) self.assertEqual(cb.get_result(tid), 42) self.assertTrue(cb.get_result(tid), 42) def test_forget(self): b = CacheBackend() tid = gen_unique_id() b.mark_as_done(tid, {"foo": "bar"}) self.assertEqual(b.get_result(tid).get("foo"), "bar") b.forget(tid) self.assertNotIn(tid, b._cache) self.assertIsNone(b.get_result(tid)) def test_save_restore_delete_taskset(self): backend = CacheBackend() taskset_id = gen_unique_id() subtask_ids = [gen_unique_id() for i in range(10)] subtasks = map(result.AsyncResult, subtask_ids) res = result.TaskSetResult(taskset_id, subtasks) res.save(backend=backend) saved = result.TaskSetResult.restore(taskset_id, backend=backend) self.assertListEqual(saved.subtasks, subtasks) self.assertEqual(saved.taskset_id, taskset_id) saved.delete(backend=backend) self.assertIsNone(result.TaskSetResult.restore(taskset_id, backend=backend)) def test_is_pickled(self): cb = CacheBackend() tid2 = gen_unique_id() result = {"foo": "baz", "bar": SomeClass(12345)} cb.mark_as_done(tid2, result) # is serialized properly. rindb = cb.get_result(tid2) self.assertEqual(rindb.get("foo"), "baz") self.assertEqual(rindb.get("bar").data, 12345) def test_mark_as_failure(self): cb = CacheBackend() einfo = None tid3 = gen_unique_id() try: raise KeyError("foo") except KeyError, exception: einfo = ExceptionInfo(sys.exc_info()) pass cb.mark_as_failure(tid3, exception, traceback=einfo.traceback) self.assertEqual(cb.get_status(tid3), states.FAILURE) self.assertIsInstance(cb.get_result(tid3), KeyError) self.assertEqual(cb.get_traceback(tid3), einfo.traceback) def test_process_cleanup(self): cb = CacheBackend() cb.process_cleanup() def test_set_expires(self): cb1 = CacheBackend(expires=timedelta(seconds=16)) self.assertEqual(cb1.expires, 16) cb2 = CacheBackend(expires=32) self.assertEqual(cb2.expires, 32) class test_custom_CacheBackend(unittest.TestCase): def test_custom_cache_backend(self): from celery import current_app prev_backend = current_app.conf.CELERY_CACHE_BACKEND prev_module = sys.modules["djcelery.backends.cache"] current_app.conf.CELERY_CACHE_BACKEND = "dummy://" sys.modules.pop("djcelery.backends.cache") try: from djcelery.backends.cache import cache from django.core.cache import cache as django_cache self.assertEqual(cache.__class__.__module__, "django.core.cache.backends.dummy") self.assertIsNot(cache, django_cache) finally: current_app.conf.CELERY_CACHE_BACKEND = prev_backend sys.modules["djcelery.backends.cache"] = prev_module class test_MemcacheWrapper(unittest.TestCase): def test_memcache_wrapper(self): try: from django.core.cache.backends import memcached from django.core.cache.backends import locmem except InvalidCacheBackendError: sys.stderr.write( "\n* Memcache library is not installed. Skipping test.\n") return prev_cache_cls = memcached.CacheClass memcached.CacheClass = locmem.CacheClass prev_backend_module = sys.modules.pop("djcelery.backends.cache") try: from djcelery.backends.cache import cache key = "cu.test_memcache_wrapper" val = "The quick brown fox." default = "The lazy dog." self.assertEqual(cache.get(key, default=default), default) cache.set(key, val) self.assertEqual(cache.get(key, default=default), val) finally: memcached.CacheClass = prev_cache_cls sys.modules["djcelery.backends.cache"] = prev_backend_module django-celery-2.5.5/djcelery/tests/test_backends/test_database.py0000644000076500000240000000573511721671622025635 0ustar asksolstaff00000000000000from __future__ import absolute_import from datetime import timedelta from celery import current_app from celery import states from celery.result import AsyncResult from celery.task import PeriodicTask from celery.utils import gen_unique_id from djcelery.backends.database import DatabaseBackend from djcelery.utils import now from djcelery.tests.utils import unittest class SomeClass(object): def __init__(self, data): self.data = data class MyPeriodicTask(PeriodicTask): name = "c.u.my-periodic-task-244" run_every = timedelta(seconds=1) def run(self, **kwargs): return 42 class TestDatabaseBackend(unittest.TestCase): def test_backend(self): b = DatabaseBackend() tid = gen_unique_id() self.assertEqual(b.get_status(tid), states.PENDING) self.assertIsNone(b.get_result(tid)) b.mark_as_done(tid, 42) self.assertEqual(b.get_status(tid), states.SUCCESS) self.assertEqual(b.get_result(tid), 42) tid2 = gen_unique_id() result = {"foo": "baz", "bar": SomeClass(12345)} b.mark_as_done(tid2, result) # is serialized properly. rindb = b.get_result(tid2) self.assertEqual(rindb.get("foo"), "baz") self.assertEqual(rindb.get("bar").data, 12345) tid3 = gen_unique_id() try: raise KeyError("foo") except KeyError, exception: pass b.mark_as_failure(tid3, exception) self.assertEqual(b.get_status(tid3), states.FAILURE) self.assertIsInstance(b.get_result(tid3), KeyError) def test_forget(self): b = DatabaseBackend() tid = gen_unique_id() b.mark_as_done(tid, {"foo": "bar"}) x = AsyncResult(tid) self.assertEqual(x.result.get("foo"), "bar") x.forget() self.assertIsNone(x.result) def test_taskset_store(self): b = DatabaseBackend() tid = gen_unique_id() self.assertIsNone(b.restore_taskset(tid)) result = {"foo": "baz", "bar": SomeClass(12345)} b.save_taskset(tid, result) rindb = b.restore_taskset(tid) self.assertIsNotNone(rindb) self.assertEqual(rindb.get("foo"), "baz") self.assertEqual(rindb.get("bar").data, 12345) b.delete_taskset(tid) self.assertIsNone(b.restore_taskset(tid)) def test_cleanup(self): b = DatabaseBackend() b.TaskModel._default_manager.all().delete() ids = [gen_unique_id() for _ in xrange(3)] for i, res in enumerate((16, 32, 64)): b.mark_as_done(ids[i], res) self.assertEqual(b.TaskModel._default_manager.count(), 3) then = now() - current_app.conf.CELERY_TASK_RESULT_EXPIRES * 2 # Have to avoid save() because it applies the auto_now=True. b.TaskModel._default_manager.filter(task_id__in=ids[:-1]) \ .update(date_done=then) b.cleanup() self.assertEqual(b.TaskModel._default_manager.count(), 1) django-celery-2.5.5/djcelery/tests/test_conf.py0000644000076500000240000000251111656051573022176 0ustar asksolstaff00000000000000from __future__ import absolute_import from django.conf import settings from celery import conf from djcelery.tests.utils import unittest SETTING_VARS = ( ("CELERY_DEFAULT_QUEUE", "DEFAULT_QUEUE"), ("CELERY_DEFAULT_ROUTING_KEY", "DEFAULT_ROUTING_KEY"), ("CELERY_DEFAULT_EXCHANGE_TYPE", "DEFAULT_EXCHANGE_TYPE"), ("CELERY_DEFAULT_EXCHANGE", "DEFAULT_EXCHANGE"), ("CELERYD_CONCURRENCY", "CELERYD_CONCURRENCY"), ("CELERYD_LOG_FILE", "CELERYD_LOG_FILE"), ("CELERYD_LOG_FORMAT", "CELERYD_LOG_FORMAT"), ) class TestConf(unittest.TestCase): def assertDefaultSetting(self, setting_name, result_var): if hasattr(settings, setting_name): self.assertEqual(getattr(conf, result_var), getattr(settings, setting_name), "Overwritten setting %s is written to %s" % ( setting_name, result_var)) else: self.assertEqual(conf._DEFAULTS.get(setting_name), getattr(conf, result_var), "Default setting %s is written to %s" % ( setting_name, result_var)) def test_configuration_cls(self): for setting_name, result_var in SETTING_VARS: self.assertDefaultSetting(setting_name, result_var) django-celery-2.5.5/djcelery/tests/test_discovery.py0000644000076500000240000000166111656051573023265 0ustar asksolstaff00000000000000from __future__ import absolute_import from django.conf import settings from celery.registry import tasks from djcelery.loaders import autodiscover from djcelery.tests.utils import unittest class TestDiscovery(unittest.TestCase): def assertDiscovery(self): apps = autodiscover() self.assertTrue(apps) self.assertIn("c.unittest.SomeAppTask", tasks) self.assertEqual(tasks["c.unittest.SomeAppTask"].run(), 42) def test_discovery(self): if "someapp" in settings.INSTALLED_APPS: self.assertDiscovery() def test_discovery_with_broken(self): if "someapp" in settings.INSTALLED_APPS: installed_apps = list(settings.INSTALLED_APPS) settings.INSTALLED_APPS = installed_apps + ["xxxnot.aexist"] try: self.assertRaises(ImportError, autodiscover) finally: settings.INSTALLED_APPS = installed_apps django-celery-2.5.5/djcelery/tests/test_loaders.py0000644000076500000240000000263411743760457022715 0ustar asksolstaff00000000000000from __future__ import absolute_import from celery import loaders from djcelery import loaders as djloaders from djcelery.tests.utils import unittest class TestDjangoLoader(unittest.TestCase): def setUp(self): self.loader = djloaders.DjangoLoader() def test_get_loader_cls(self): self.assertEqual(loaders.get_loader_cls("django"), self.loader.__class__) # Execute cached branch. self.assertEqual(loaders.get_loader_cls("django"), self.loader.__class__) def test_on_worker_init(self): from django.conf import settings old_imports = getattr(settings, "CELERY_IMPORTS", None) settings.CELERY_IMPORTS = ("xxx.does.not.exist", ) try: self.assertRaises(ImportError, self.loader.import_default_modules) finally: settings.CELERY_IMPORTS = old_imports def test_race_protection(self): djloaders._RACE_PROTECTION = True try: self.assertFalse(self.loader.on_worker_init()) finally: djloaders._RACE_PROTECTION = False def test_find_related_module_no_path(self): self.assertFalse(djloaders.find_related_module("sys", "tasks")) def test_find_related_module_no_related(self): self.assertFalse(djloaders.find_related_module("someapp", "frobulators")) django-celery-2.5.5/djcelery/tests/test_models.py0000644000076500000240000000660411721671622022537 0ustar asksolstaff00000000000000from __future__ import absolute_import from datetime import datetime, timedelta from celery import current_app from celery import states from celery.utils import gen_unique_id from djcelery.models import TaskMeta, TaskSetMeta from djcelery.tests.utils import unittest from djcelery.utils import now class TestModels(unittest.TestCase): def createTaskMeta(self): id = gen_unique_id() taskmeta, created = TaskMeta.objects.get_or_create(task_id=id) return taskmeta def createTaskSetMeta(self): id = gen_unique_id() tasksetmeta, created = TaskSetMeta.objects.get_or_create(taskset_id=id) return tasksetmeta def test_taskmeta(self): m1 = self.createTaskMeta() m2 = self.createTaskMeta() m3 = self.createTaskMeta() self.assertTrue(unicode(m1).startswith("" % (self.__class__.__name__, self.args) class MyRetryTaskError(Exception): def __repr__(self): return "<%s: %r>" % (self.__class__.__name__, self.args) task_is_successful = partial(reversestar, "celery-is_task_successful") task_status = partial(reversestar, "celery-task_status") task_apply = partial(reverse, "celery-apply") registered_tasks = partial(reverse, "celery-tasks") scratch = {} @task() def mytask(x, y): ret = scratch["result"] = int(x) * int(y) return ret def create_exception(name, base=Exception): return type(name, (base, ), {}) def catch_exception(exception): try: raise exception except exception.__class__, exc: exc = current_app.backend.prepare_exception(exc) return exc, ExceptionInfo(sys.exc_info()).traceback class ViewTestCase(DjangoTestCase): def assertJSONEqual(self, json, py): json = isinstance(json, HttpResponse) and json.content or json try: self.assertEqual(deserialize(json), py) except TypeError, exc: raise TypeError("%s: %s" % (exc, json)) def assertIn(self, expected, source, *args): try: DjangoTestCase.assertIn(self, expected, source, *args) except AttributeError: self.assertTrue(expected in source) def assertDictContainsSubset(self, a, b, *args): try: DjangoTestCase.assertDictContainsSubset(self, a, b, *args) except AttributeError: for key, value in a.items(): self.assertTrue(key in b) self.assertEqual(b[key], value) class test_task_apply(ViewTestCase): def test_apply(self): current_app.conf.CELERY_ALWAYS_EAGER = True try: self.client.get(task_apply(kwargs={"task_name": mytask.name}) + "?x=4&y=4") self.assertEqual(scratch["result"], 16) finally: current_app.conf.CELERY_ALWAYS_EAGER = False def test_apply_raises_404_on_unregistered_task(self): current_app.conf.CELERY_ALWAYS_EAGER = True try: name = "xxx.does.not.exist" action = partial(self.client.get, task_apply(kwargs={ "task_name": name}) + "?x=4&y=4") self.assertRaises(TemplateDoesNotExist, action) finally: current_app.conf.CELERY_ALWAYS_EAGER = False class test_registered_tasks(ViewTestCase): def test_list_registered_tasks(self): json = self.client.get(registered_tasks()) tasks = deserialize(json.content) self.assertIn("celery.backend_cleanup", tasks["regular"]) class test_webhook_task(ViewTestCase): def test_successful_request(self): @task_webhook def add_webhook(request): x = int(request.GET["x"]) y = int(request.GET["y"]) return x + y request = MockRequest().get("/tasks/add", dict(x=10, y=10)) response = add_webhook(request) self.assertDictContainsSubset({"status": "success", "retval": 20}, deserialize(response.content)) def test_failed_request(self): @task_webhook def error_webhook(request): x = int(request.GET["x"]) y = int(request.GET["y"]) raise MyError(x + y) request = MockRequest().get("/tasks/error", dict(x=10, y=10)) response = error_webhook(request) self.assertDictContainsSubset({"status": "failure", "reason": ""}, deserialize(response.content)) class test_task_status(ViewTestCase): def assertStatusForIs(self, status, res, traceback=None): uuid = gen_unique_id() current_app.backend.store_result(uuid, res, status, traceback=traceback) json = self.client.get(task_status(task_id=uuid)) expect = dict(id=uuid, status=status, result=res) if status in current_app.backend.EXCEPTION_STATES: instore = current_app.backend.get_result(uuid) self.assertEqual(str(instore.args[0]), str(res.args[0])) expect["result"] = repr(res) expect["exc"] = get_full_cls_name(res.__class__) expect["traceback"] = traceback self.assertJSONEqual(json, dict(task=expect)) def test_success(self): self.assertStatusForIs(states.SUCCESS, "The quick brown fox") def test_failure(self): exc, tb = catch_exception(MyError("foo")) self.assertStatusForIs(states.FAILURE, exc, tb) def test_retry(self): oexc, _ = catch_exception(MyError("Resource not available")) exc, tb = catch_exception(MyRetryTaskError(str(oexc), oexc)) self.assertStatusForIs(states.RETRY, exc, tb) class test_task_is_successful(ViewTestCase): def assertStatusForIs(self, status, outcome): uuid = gen_unique_id() result = gen_unique_id() current_app.backend.store_result(uuid, result, status) json = self.client.get(task_is_successful(task_id=uuid)) self.assertJSONEqual(json, {"task": {"id": uuid, "executed": outcome}}) def test_success(self): self.assertStatusForIs(states.SUCCESS, True) def test_pending(self): self.assertStatusForIs(states.PENDING, False) def test_failure(self): self.assertStatusForIs(states.FAILURE, False) def test_retry(self): self.assertStatusForIs(states.RETRY, False) django-celery-2.5.5/djcelery/tests/test_worker_job.py0000644000076500000240000000504111656051573023415 0ustar asksolstaff00000000000000# -*- coding: utf-8 -*- from __future__ import absolute_import from django.core import cache from celery.utils import gen_unique_id from celery.task import task as task_dec from celery.tests.test_worker.test_worker_job import jail from djcelery.tests.utils import unittest @task_dec() def mytask(i): return i ** i @task_dec() def get_db_connection(i): from django.db import connection return id(connection) get_db_connection.ignore_result = True class TestJail(unittest.TestCase): def test_django_db_connection_is_closed(self): from django.db import connection connection._was_closed = False old_connection_close = connection.close def monkeypatched_connection_close(*args, **kwargs): connection._was_closed = True return old_connection_close(*args, **kwargs) connection.close = monkeypatched_connection_close try: jail(gen_unique_id(), get_db_connection.name, [2], {}) self.assertTrue(connection._was_closed) finally: connection.close = old_connection_close def test_django_cache_connection_is_closed(self): old_cache_close = getattr(cache.cache, "close", None) cache._was_closed = False old_cache_parse_backend = getattr(cache, "parse_backend_uri", None) if old_cache_parse_backend: # checks to make sure attr exists delattr(cache, 'parse_backend_uri') def monkeypatched_cache_close(*args, **kwargs): cache._was_closed = True cache.cache.close = monkeypatched_cache_close jail(gen_unique_id(), mytask.name, [4], {}) self.assertTrue(cache._was_closed) cache.cache.close = old_cache_close if old_cache_parse_backend: cache.parse_backend_uri = old_cache_parse_backend def test_django_cache_connection_is_closed_django_1_1(self): old_cache_close = getattr(cache.cache, "close", None) cache._was_closed = False old_cache_parse_backend = getattr(cache, "parse_backend_uri", None) cache.parse_backend_uri = lambda uri: ["libmemcached", "1", "2"] def monkeypatched_cache_close(*args, **kwargs): cache._was_closed = True cache.cache.close = monkeypatched_cache_close jail(gen_unique_id(), mytask.name, [4], {}) self.assertTrue(cache._was_closed) cache.cache.close = old_cache_close if old_cache_parse_backend: cache.parse_backend_uri = old_cache_parse_backend else: del(cache.parse_backend_uri) django-celery-2.5.5/djcelery/tests/utils.py0000644000076500000240000000022311656051573021350 0ustar asksolstaff00000000000000from __future__ import absolute_import try: import unittest unittest.skip except AttributeError: import unittest2 as unittest # noqa django-celery-2.5.5/djcelery/urls.py0000644000076500000240000000117711656051573020044 0ustar asksolstaff00000000000000""" URLs defined for celery. * ``/$task_id/done/`` URL to :func:`~celery.views.is_successful`. * ``/$task_id/status/`` URL to :func:`~celery.views.task_status`. """ from __future__ import absolute_import from django.conf.urls.defaults import patterns, url from . import views task_pattern = r'(?P[\w\d\-\.]+)' urlpatterns = patterns("", url(r'^%s/done/?$' % task_pattern, views.is_task_successful, name="celery-is_task_successful"), url(r'^%s/status/?$' % task_pattern, views.task_status, name="celery-task_status"), url(r'^tasks/?$', views.registered_tasks, name='celery-tasks'), ) django-celery-2.5.5/djcelery/utils.py0000644000076500000240000000320411741563541020206 0ustar asksolstaff00000000000000# -- XXX This module must not use translation as that causes # -- a recursive loader import! from __future__ import absolute_import from datetime import datetime from django.conf import settings # Database-related exceptions. from django.db import DatabaseError try: import MySQLdb as mysql _my_database_errors = (mysql.DatabaseError, ) except ImportError: _my_database_errors = () # noqa try: import psycopg2 as pg _pg_database_errors = (pg.DatabaseError, ) except ImportError: _pg_database_errors = () # noqa try: import sqlite3 _lite_database_errors = (sqlite3.DatabaseError, ) except ImportError: _lite_database_errors = () # noqa try: import cx_Oracle as oracle _oracle_database_errors = (oracle.DatabaseError, ) except ImportError: _oracle_database_errors = () # noqa DATABASE_ERRORS = ((DatabaseError, ) + _my_database_errors + _pg_database_errors + _lite_database_errors + _oracle_database_errors) try: from django.utils import timezone def make_aware(value): if getattr(settings, "USE_TZ", False): default_tz = timezone.get_default_timezone() value = timezone.make_aware(value, default_tz) return value def make_naive(value): if getattr(settings, "USE_TZ", False): default_tz = timezone.get_default_timezone() value = timezone.make_naive(value, default_tz) return value def now(): return timezone.localtime(timezone.now()) except ImportError: now = datetime.now make_aware = make_naive = lambda x: x django-celery-2.5.5/djcelery/views.py0000644000076500000240000000700511742044217020201 0ustar asksolstaff00000000000000from __future__ import absolute_import from functools import wraps from django.http import HttpResponse, Http404 from anyjson import serialize from celery import states from celery.registry import tasks from celery.result import AsyncResult from celery.utils import get_full_cls_name, kwdict from celery.utils.encoding import safe_repr # Ensure built-in tasks are loaded for task_list view import celery.task # noqa def JsonResponse(response): return HttpResponse(serialize(response), mimetype="application/json") def task_view(task): """Decorator turning any task into a view that applies the task asynchronously. Keyword arguments (via URLconf, etc.) will supercede GET or POST parameters when there are conflicts. Returns a JSON dictionary containing the keys ``ok``, and ``task_id``. """ def _applier(request, **options): kwargs = kwdict(request.method == "POST" and \ request.POST or request.GET) # no multivalue kwargs = dict(((k, v) for k, v in kwargs.iteritems()), **options) result = task.apply_async(kwargs=kwargs) return JsonResponse({"ok": "true", "task_id": result.task_id}) return _applier def apply(request, task_name): """View applying a task. **Note:** Please use this with caution. Preferably you shouldn't make this publicly accessible without ensuring your code is safe! """ try: task = tasks[task_name] except KeyError: raise Http404("apply: no such task") return task_view(task)(request) def is_task_successful(request, task_id): """Returns task execute status in JSON format.""" response_data = {"task": {"id": task_id, "executed": AsyncResult(task_id).successful()}} return HttpResponse(serialize(response_data), mimetype="application/json") def task_status(request, task_id): """Returns task status and result in JSON format.""" result = AsyncResult(task_id) state, retval = result.state, result.result response_data = dict(id=task_id, status=state, result=retval) if state in states.EXCEPTION_STATES: traceback = result.traceback response_data.update({"result": safe_repr(retval), "exc": get_full_cls_name(retval.__class__), "traceback": traceback}) return JsonResponse({"task": response_data}) def registered_tasks(request): """ A view returning all defined tasks as a JSON object. """ return JsonResponse({"regular": tasks.regular().keys(), "periodic": tasks.periodic().keys()}) def task_webhook(fun): """Decorator turning a function into a task webhook. If an exception is raised within the function, the decorated function catches this and returns an error JSON response, otherwise it returns the result as a JSON response. Example: .. code-block:: python @task_webhook def add(request): x = int(request.GET["x"]) y = int(request.GET["y"]) return x + y >>> response = add(request) >>> response.content '{"status": "success", "retval": 100}' """ @wraps(fun) def _inner(*args, **kwargs): try: retval = fun(*args, **kwargs) except Exception, exc: response = {"status": "failure", "reason": safe_repr(exc)} else: response = {"status": "success", "retval": retval} return JsonResponse(response) return _inner django-celery-2.5.5/docs/0000755000076500000240000000000011744004562015620 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/.static/0000755000076500000240000000000011744004562017165 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/.static/.keep0000644000076500000240000000000011656051573020106 0ustar asksolstaff00000000000000django-celery-2.5.5/docs/.templates/0000755000076500000240000000000011744004562017674 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/.templates/sidebarintro.html0000644000076500000240000000011211545441323023240 0ustar asksolstaff00000000000000

Celery

Celery is a Distributed Task Queue for Python.

django-celery-2.5.5/docs/.templates/sidebarlogo.html0000644000076500000240000000025011545441323023050 0ustar asksolstaff00000000000000 django-celery-2.5.5/docs/__init__.py0000644000076500000240000000000011721735236017723 0ustar asksolstaff00000000000000django-celery-2.5.5/docs/_ext/0000755000076500000240000000000011744004562016557 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/_ext/applyxrefs.py0000644000076500000240000000415111656051573021335 0ustar asksolstaff00000000000000"""Adds xref targets to the top of files.""" import sys import os testing = False DONT_TOUCH = ( './index.txt', ) def target_name(fn): if fn.endswith('.txt'): fn = fn[:-4] return '_' + fn.lstrip('./').replace('/', '-') def process_file(fn, lines): lines.insert(0, '\n') lines.insert(0, '.. %s:\n' % target_name(fn)) try: f = open(fn, 'w') except IOError: print("Can't open %s for writing. Not touching it." % fn) return try: f.writelines(lines) except IOError: print("Can't write to %s. Not touching it." % fn) finally: f.close() def has_target(fn): try: f = open(fn, 'r') except IOError: print("Can't open %s. Not touching it." % fn) return (True, None) readok = True try: lines = f.readlines() except IOError: print("Can't read %s. Not touching it." % fn) readok = False finally: f.close() if not readok: return (True, None) #print fn, len(lines) if len(lines) < 1: print("Not touching empty file %s." % fn) return (True, None) if lines[0].startswith('.. _'): return (True, None) return (False, lines) def main(argv=None): if argv is None: argv = sys.argv if len(argv) == 1: argv.extend('.') files = [] for root in argv[1:]: for (dirpath, dirnames, filenames) in os.walk(root): files.extend([(dirpath, f) for f in filenames]) files.sort() files = [os.path.join(p, fn) for p, fn in files if fn.endswith('.txt')] #print files for fn in files: if fn in DONT_TOUCH: print("Skipping blacklisted file %s." % fn) continue target_found, lines = has_target(fn) if not target_found: if testing: print '%s: %s' % (fn, lines[0]), else: print "Adding xref to %s" % fn process_file(fn, lines) else: print "Skipping %s: already has a xref" % fn if __name__ == '__main__': sys.exit(main()) django-celery-2.5.5/docs/_ext/literals_to_xrefs.py0000644000076500000240000001116511656051573022673 0ustar asksolstaff00000000000000""" Runs through a reST file looking for old-style literals, and helps replace them with new-style references. """ import re import sys import shelve refre = re.compile(r'``([^`\s]+?)``') ROLES = ( 'attr', 'class', "djadmin", 'data', 'exc', 'file', 'func', 'lookup', 'meth', 'mod', "djadminopt", "ref", "setting", "term", "tfilter", "ttag", # special "skip", ) ALWAYS_SKIP = [ "NULL", "True", "False", ] def fixliterals(fname): data = open(fname).read() last = 0 new = [] storage = shelve.open("/tmp/literals_to_xref.shelve") lastvalues = storage.get("lastvalues", {}) for m in refre.finditer(data): new.append(data[last:m.start()]) last = m.end() line_start = data.rfind("\n", 0, m.start()) line_end = data.find("\n", m.end()) prev_start = data.rfind("\n", 0, line_start) next_end = data.find("\n", line_end + 1) # Skip always-skip stuff if m.group(1) in ALWAYS_SKIP: new.append(m.group(0)) continue # skip when the next line is a title next_line = data[m.end():next_end].strip() if next_line[0] in "!-/:-@[-`{-~" and \ all(c == next_line[0] for c in next_line): new.append(m.group(0)) continue sys.stdout.write("\n" + "-" * 80 + "\n") sys.stdout.write(data[prev_start + 1:m.start()]) sys.stdout.write(colorize(m.group(0), fg="red")) sys.stdout.write(data[m.end():next_end]) sys.stdout.write("\n\n") replace_type = None while replace_type is None: replace_type = raw_input( colorize("Replace role: ", fg="yellow")).strip().lower() if replace_type and replace_type not in ROLES: replace_type = None if replace_type == "": new.append(m.group(0)) continue if replace_type == "skip": new.append(m.group(0)) ALWAYS_SKIP.append(m.group(1)) continue default = lastvalues.get(m.group(1), m.group(1)) if default.endswith("()") and \ replace_type in ("class", "func", "meth"): default = default[:-2] replace_value = raw_input( colorize("Text [", fg="yellow") + default + \ colorize("]: ", fg="yellow")).strip() if not replace_value: replace_value = default new.append(":%s:`%s`" % (replace_type, replace_value)) lastvalues[m.group(1)] = replace_value new.append(data[last:]) open(fname, "w").write("".join(new)) storage["lastvalues"] = lastvalues storage.close() def colorize(text='', opts=(), **kwargs): """ Returns your text, enclosed in ANSI graphics codes. Depends on the keyword arguments 'fg' and 'bg', and the contents of the opts tuple/list. Returns the RESET code if no parameters are given. Valid colors: 'black', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'white' Valid options: 'bold' 'underscore' 'blink' 'reverse' 'conceal' 'noreset' - string will not be auto-terminated with the RESET code Examples: colorize('hello', fg='red', bg='blue', opts=('blink',)) colorize() colorize('goodbye', opts=('underscore',)) print colorize('first line', fg='red', opts=('noreset',)) print 'this should be red too' print colorize('and so should this') print 'this should not be red' """ color_names = ('black', 'red', 'green', 'yellow', 'blue', 'magenta', 'cyan', 'white') foreground = dict([(color_names[x], '3%s' % x) for x in range(8)]) background = dict([(color_names[x], '4%s' % x) for x in range(8)]) RESET = '0' opt_dict = {'bold': '1', 'underscore': '4', 'blink': '5', 'reverse': '7', 'conceal': '8'} text = str(text) code_list = [] if text == '' and len(opts) == 1 and opts[0] == 'reset': return '\x1b[%sm' % RESET for k, v in kwargs.iteritems(): if k == 'fg': code_list.append(foreground[v]) elif k == 'bg': code_list.append(background[v]) for o in opts: if o in opt_dict: code_list.append(opt_dict[o]) if 'noreset' not in opts: text = text + '\x1b[%sm' % RESET return ('\x1b[%sm' % ';'.join(code_list)) + text if __name__ == '__main__': try: fixliterals(sys.argv[1]) except (KeyboardInterrupt, SystemExit): print django-celery-2.5.5/docs/_theme/0000755000076500000240000000000011744004562017061 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/_theme/celery/0000755000076500000240000000000011744004562020344 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/_theme/celery/static/0000755000076500000240000000000011744004562021633 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/_theme/celery/static/celery.css_t0000644000076500000240000001474611656051573024175 0ustar asksolstaff00000000000000/* * celery.css_t * ~~~~~~~~~~~~ * * :copyright: Copyright 2010 by Armin Ronacher. * :license: BSD, see LICENSE for details. */ {% set page_width = 940 %} {% set sidebar_width = 220 %} {% set body_font_stack = 'Optima, Segoe, "Segoe UI", Candara, Calibri, Arial, sans-serif' %} {% set headline_font_stack = 'Futura, "Trebuchet MS", Arial, sans-serif' %} {% set code_font_stack = "'Consolas', 'Menlo', 'Deja Vu Sans Mono', 'Bitstream Vera Sans Mono', monospace" %} @import url("basic.css"); /* -- page layout ----------------------------------------------------------- */ body { font-family: {{ body_font_stack }}; font-size: 17px; background-color: white; color: #000; margin: 30px 0 0 0; padding: 0; } div.document { width: {{ page_width }}px; margin: 0 auto; } div.related { width: {{ page_width - 20 }}px; padding: 5px 10px; background: #F2FCEE; margin: 15px auto 15px auto; } div.documentwrapper { float: left; width: 100%; } div.bodywrapper { margin: 0 0 0 {{ sidebar_width }}px; } div.sphinxsidebar { width: {{ sidebar_width }}px; } hr { border: 1px solid #B1B4B6; } div.body { background-color: #ffffff; color: #3E4349; padding: 0 30px 0 30px; } img.celerylogo { padding: 0 0 10px 10px; float: right; } div.footer { width: {{ page_width - 15 }}px; margin: 10px auto 30px auto; padding-right: 15px; font-size: 14px; color: #888; text-align: right; } div.footer a { color: #888; } div.sphinxsidebar a { color: #444; text-decoration: none; border-bottom: 1px dashed #DCF0D5; } div.sphinxsidebar a:hover { border-bottom: 1px solid #999; } div.sphinxsidebar { font-size: 14px; line-height: 1.5; } div.sphinxsidebarwrapper { padding: 7px 10px; } div.sphinxsidebarwrapper p.logo { padding: 0 0 20px 0; margin: 0; } div.sphinxsidebar h3, div.sphinxsidebar h4 { font-family: {{ headline_font_stack }}; color: #444; font-size: 24px; font-weight: normal; margin: 0 0 5px 0; padding: 0; } div.sphinxsidebar h4 { font-size: 20px; } div.sphinxsidebar h3 a { color: #444; } div.sphinxsidebar p.logo a, div.sphinxsidebar h3 a, div.sphinxsidebar p.logo a:hover, div.sphinxsidebar h3 a:hover { border: none; } div.sphinxsidebar p { color: #555; margin: 10px 0; } div.sphinxsidebar ul { margin: 10px 0; padding: 0; color: #000; } div.sphinxsidebar input { border: 1px solid #ccc; font-family: {{ body_font_stack }}; font-size: 1em; } /* -- body styles ----------------------------------------------------------- */ a { color: #348613; text-decoration: underline; } a:hover { color: #59B833; text-decoration: underline; } div.body h1, div.body h2, div.body h3, div.body h4, div.body h5, div.body h6 { font-family: {{ headline_font_stack }}; font-weight: normal; margin: 30px 0px 10px 0px; padding: 0; } div.body h1 { margin-top: 0; padding-top: 0; font-size: 200%; } div.body h2 { font-size: 180%; } div.body h3 { font-size: 150%; } div.body h4 { font-size: 130%; } div.body h5 { font-size: 100%; } div.body h6 { font-size: 100%; } div.body h1 a.toc-backref, div.body h2 a.toc-backref, div.body h3 a.toc-backref, div.body h4 a.toc-backref, div.body h5 a.toc-backref, div.body h6 a.toc-backref { color: inherit!important; text-decoration: none; } a.headerlink { color: #ddd; padding: 0 4px; text-decoration: none; } a.headerlink:hover { color: #444; background: #eaeaea; } div.body p, div.body dd, div.body li { line-height: 1.4em; } div.admonition { background: #fafafa; margin: 20px -30px; padding: 10px 30px; border-top: 1px solid #ccc; border-bottom: 1px solid #ccc; } div.admonition p.admonition-title { font-family: {{ headline_font_stack }}; font-weight: normal; font-size: 24px; margin: 0 0 10px 0; padding: 0; line-height: 1; } div.admonition p.last { margin-bottom: 0; } div.highlight{ background-color: white; } dt:target, .highlight { background: #FAF3E8; } div.note { background-color: #eee; border: 1px solid #ccc; } div.seealso { background-color: #ffc; border: 1px solid #ff6; } div.topic { background-color: #eee; } div.warning { background-color: #ffe4e4; border: 1px solid #f66; } p.admonition-title { display: inline; } p.admonition-title:after { content: ":"; } pre, tt { font-family: {{ code_font_stack }}; font-size: 0.9em; } img.screenshot { } tt.descname, tt.descclassname { font-size: 0.95em; } tt.descname { padding-right: 0.08em; } img.screenshot { -moz-box-shadow: 2px 2px 4px #eee; -webkit-box-shadow: 2px 2px 4px #eee; box-shadow: 2px 2px 4px #eee; } table.docutils { border: 1px solid #888; -moz-box-shadow: 2px 2px 4px #eee; -webkit-box-shadow: 2px 2px 4px #eee; box-shadow: 2px 2px 4px #eee; } table.docutils td, table.docutils th { border: 1px solid #888; padding: 0.25em 0.7em; } table.field-list, table.footnote { border: none; -moz-box-shadow: none; -webkit-box-shadow: none; box-shadow: none; } table.footnote { margin: 15px 0; width: 100%; border: 1px solid #eee; background: #fdfdfd; font-size: 0.9em; } table.footnote + table.footnote { margin-top: -15px; border-top: none; } table.field-list th { padding: 0 0.8em 0 0; } table.field-list td { padding: 0; } table.footnote td.label { width: 0px; padding: 0.3em 0 0.3em 0.5em; } table.footnote td { padding: 0.3em 0.5em; } dl { margin: 0; padding: 0; } dl dd { margin-left: 30px; } blockquote { margin: 0 0 0 30px; padding: 0; } ul { margin: 10px 0 10px 30px; padding: 0; } pre { background: #F0FFEB; padding: 7px 10px; margin: 15px 0; border: 1px solid #C7ECB8; border-radius: 2px; -moz-border-radius: 2px; -webkit-border-radius: 2px; line-height: 1.3em; } tt { background: #F0FFEB; color: #222; /* padding: 1px 2px; */ } tt.xref, a tt { background: #F0FFEB; border-bottom: 1px solid white; } a.reference { text-decoration: none; border-bottom: 1px dashed #DCF0D5; } a.reference:hover { border-bottom: 1px solid #6D4100; } a.footnote-reference { text-decoration: none; font-size: 0.7em; vertical-align: top; border-bottom: 1px dashed #DCF0D5; } a.footnote-reference:hover { border-bottom: 1px solid #6D4100; } a:hover tt { background: #EEE; } django-celery-2.5.5/docs/_theme/celery/theme.conf0000644000076500000240000000007311656051573022323 0ustar asksolstaff00000000000000[theme] inherit = basic stylesheet = celery.css [options] django-celery-2.5.5/docs/changelog.rst0000644000076500000240000000000011744004456025272 1django-celery-2.5.5/Changelogustar asksolstaff00000000000000django-celery-2.5.5/docs/conf.py0000644000076500000240000000462411723437575017140 0ustar asksolstaff00000000000000# -*- coding: utf-8 -*- import sys import os # If your extensions are in another directory, add it here. If the directory # is relative to the documentation root, use os.path.abspath to make it # absolute, like shown here. sys.path.insert(0, os.getcwd()) import django os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") if django.VERSION < (1, 4): from django.core.management import setup_environ setup_environ(__import__(os.environ["DJANGO_SETTINGS_MODULE"])) import djcelery # General configuration # --------------------- extensions = ['sphinx.ext.autodoc', 'sphinx.ext.coverage', 'sphinxcontrib.issuetracker'] # Add any paths that contain templates here, relative to this directory. templates_path = ['.templates'] # The suffix of source filenames. source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = u'django-celery' copyright = u'2009-2011, Ask Solem' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = ".".join(map(str, djcelery.VERSION[0:2])) # The full version, including alpha/beta/rc tags. release = djcelery.__version__ exclude_trees = ['.build'] # If true, '()' will be appended to :func: etc. cross-reference text. add_function_parentheses = True # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'trac' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['.static'] html_use_smartypants = True # If false, no module index is generated. html_use_modindex = True # If false, no index is generated. html_use_index = True latex_documents = [ ('index', 'django-celery.tex', ur'django-celery Documentation', ur'Ask Solem', 'manual'), ] html_theme = "celery" html_theme_path = ["_theme"] html_sidebars = { 'index': ['sidebarintro.html', 'sourcelink.html', 'searchbox.html'], '**': ['sidebarlogo.html', 'localtoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html'], } ### Issuetracker issuetracker = "github" issuetracker_project = "ask/django-celery" issuetracker_issue_pattern = r'[Ii]ssue #(\d+)' django-celery-2.5.5/docs/cookbook/0000755000076500000240000000000011744004562017426 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/cookbook/index.rst0000644000076500000240000000020611656051573021273 0ustar asksolstaff00000000000000=========== Cookbook =========== .. toctree:: :maxdepth: 2 unit-testing This page contains common recipes and techniques. django-celery-2.5.5/docs/cookbook/unit-testing.rst0000644000076500000240000000407111656051574022623 0ustar asksolstaff00000000000000================ Unit Testing ================ Testing with Django ------------------- The first problem you'll run in to when trying to write a test that runs a task is that Django's test runner doesn't use the same database as your celery daemon is using. If you're using the database backend, this means that your tombstones won't show up in your test database and you won't be able to get the return value or check the status of your tasks. There are two ways to get around this. You can either take advantage of ``CELERY_ALWAYS_EAGER = True`` to skip the daemon, or you can avoid testing anything that needs to check the status or result of a task. Using a custom test runner to test with celery ---------------------------------------------- If you're going the ``CELERY_ALWAYS_EAGER`` route, which is probably better than just never testing some parts of your app, a custom Django test runner does the trick. Celery provides a simple test runner, but it's easy enough to roll your own if you have other things that need to be done. http://docs.djangoproject.com/en/dev/topics/testing/#defining-a-test-runner For this example, we'll use the ``djcelery.contrib.test_runner`` to test the ``add`` task from the `User Guide: Tasks`_ examples in the Celery documentation. .. _`User Guide: Tasks`: http://docs.celeryq.org/en/latest/userguide/tasks.html To enable the test runner, set the following settings: .. code-block:: python TEST_RUNNER = 'djcelery.contrib.test_runner.CeleryTestSuiteRunner' Then we can put the tests in a ``tests.py`` somewhere: .. code-block:: python from django.test import TestCase from myapp.tasks import add class AddTestCase(TestCase): def testNoError(self): """Test that the ``add`` task runs with no errors, and returns the correct result.""" result = add.delay(8, 8) self.assertEquals(result.get(), 16) self.assertTrue(result.successful()) This test assumes that you put your example ``add`` task in ``maypp.tasks`` so adjust the import for wherever you put the class. django-celery-2.5.5/docs/faq.rst0000644000076500000240000000472211656051572017133 0ustar asksolstaff00000000000000============================ Frequently Asked Questions ============================ Generating a template in a task doesn't seem to respect my i18n settings? ------------------------------------------------------------------------- **Answer**: To enable the Django translation machinery you need to activate it with a language. **Note**: Be sure to reset to the previous language when done. >>> from django.utils import translation >>> prev_language = translation.get_language() >>> translation.activate(language) >>> try: ... render_template() ... finally: translation.activate(prev_language) The common pattern here would be for the task to take a ``language`` argument: .. code-block:: python from celery.decorators import task from django.utils import translation from django.template.loader import render_to_string @task() def generate_report(template="report.html", language=None): prev_language = translation.get_language() language and translation.activate(language) try: report = render_to_string(template) finally: translation.activate(prev_language) save_report_somewhere(report) The celery test-suite is failing -------------------------------- **Answer**: If you're running tests from your Django project, and the celery test suite is failing in that context, then follow the steps below. If the celery tests are failing in another context, please report an issue to our issue tracker at GitHub: http://github.com/ask/celery/issues/ That Django is running tests for all applications in ``INSTALLED_APPS`` by default is a pet peeve for many. You should use a test runner that either 1) Explicitly lists the apps you want to run tests for, or 2) Make a test runner that skips tests for apps you don't want to run. For example the test runner that celery is using: http://github.com/ask/celery/blob/f90491fe0194aa472b5aecdefe5cc83289e65e69/celery/tests/runners.py To use this test runner, add the following to your ``settings.py``: .. code-block:: python TEST_RUNNER = "djcelery.tests.runners.CeleryTestSuiteRunner", TEST_APPS = ( "app1", "app2", "app3", "app4", ) Or, if you just want to skip the celery tests: .. code-block:: python INSTALLED_APPS = (.....) TEST_RUNNER = "djcelery.tests.runners.CeleryTestSuiteRunner", TEST_APPS = filter(lambda k: k != "celery", INSTALLED_APPS) django-celery-2.5.5/docs/getting-started/0000755000076500000240000000000011744004562020725 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/getting-started/first-steps-with-django.rst0000644000076500000240000000035211741563541026157 0ustar asksolstaff00000000000000========================= First steps with Django ========================= This document has been moved into the main Celery documentation, you can find it at:; http://ask.github.com/celery/django/first-steps-with-django.html django-celery-2.5.5/docs/getting-started/index.rst0000644000076500000240000000016111656051574022573 0ustar asksolstaff00000000000000================= Getting Started ================= .. toctree:: :maxdepth: 2 first-steps-with-django django-celery-2.5.5/docs/index.rst0000644000076500000240000000063011656051574017467 0ustar asksolstaff00000000000000.. image:: http://cloud.github.com/downloads/ask/celery/celery_128.png =============================== Celery Integration for Django =============================== Contents: .. toctree:: :maxdepth: 2 introduction getting-started/index faq cookbook/index reference/index changelog Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` django-celery-2.5.5/docs/introduction.rst0000644000076500000240000001162711744004534021101 0ustar asksolstaff00000000000000=============================================== django-celery - Celery Integration for Django =============================================== .. image:: http://cloud.github.com/downloads/ask/celery/celery_128.png :Version: 2.5.5 :Web: http://celeryproject.org/ :Download: http://pypi.python.org/pypi/django-celery/ :Source: http://github.com/ask/django-celery/ :Keywords: celery, task queue, job queue, asynchronous, rabbitmq, amqp, redis, python, django, webhooks, queue, distributed -- django-celery provides Celery integration for Django; Using the Django ORM and cache backend for storing results, autodiscovery of task modules for applications listed in ``INSTALLED_APPS``, and more. .. contents:: :local: Using django-celery =================== To enable ``django-celery`` for your project you need to add ``djcelery`` to ``INSTALLED_APPS``:: INSTALLED_APPS += ("djcelery", ) then add the following lines to your ``settings.py``:: import djcelery djcelery.setup_loader() Everything works the same as described in the `Celery User Manual`_, except you need to invoke the programs through ``manage.py``: ===================================== ===================================== **Program** **Replace with** ===================================== ===================================== ``celeryd`` ``python manage.py celeryd`` ``celeryctl`` ``python manage.py celeryctl`` ``celerybeat`` ``python manage.py celerybeat`` ``camqadm`` ``python manage.py camqadm`` ``celeryev`` ``python manage.py celeryev`` ``celeryd-multi`` ``python manage.py celeryd_multi`` ===================================== ===================================== The other main difference is that configuration values are stored in your Django projects' ``settings.py`` module rather than in ``celeryconfig.py``. If you're trying celery for the first time you should start by reading `Getting started with django-celery`_ Special note for mod_wsgi users ------------------------------- If you're using ``mod_wsgi`` to deploy your Django application you need to include the following in your ``.wsgi`` module:: import djcelery djcelery.setup_loader() Documentation ============= The `Celery User Manual`_ contains user guides, tutorials and an API reference. Also the `django-celery documentation`_, contains information about the Django integration. .. _`django-celery documentation`: http://django-celery.readthedocs.org/ .. _`Celery User Manual`: http://docs.celeryproject.org/ .. _`Getting started with django-celery`: http://django-celery.readthedocs.org/en/latest/getting-started/first-steps-with-django.html Installation ============= You can install ``django-celery`` either via the Python Package Index (PyPI) or from source. To install using ``pip``,:: $ pip install django-celery To install using ``easy_install``,:: $ easy_install django-celery You will then want to create the necessary tables. If you are using south_ for schema migrations, you'll want to:: $ python manage.py migrate djcelery For those who are not using south, a normal :command:`syncdb` will work:: $ python manage.py syncdb .. _south: http://pypi.python.org/pypi/South/ Downloading and installing from source -------------------------------------- Download the latest version of ``django-celery`` from http://pypi.python.org/pypi/django-celery/ You can install it by doing the following,:: $ tar xvfz django-celery-0.0.0.tar.gz $ cd django-celery-0.0.0 # python setup.py install # as root Using the development version ------------------------------ You can clone the git repository by doing the following:: $ git clone git://github.com/ask/django-celery.git Getting Help ============ Mailing list ------------ For discussions about the usage, development, and future of celery, please join the `celery-users`_ mailing list. .. _`celery-users`: http://groups.google.com/group/celery-users/ IRC --- Come chat with us on IRC. The `#celery`_ channel is located at the `Freenode`_ network. .. _`#celery`: irc://irc.freenode.net/celery .. _`Freenode`: http://freenode.net Bug tracker =========== If you have any suggestions, bug reports or annoyances please report them to our issue tracker at http://github.com/ask/django-celery/issues/ Wiki ==== http://wiki.github.com/ask/celery/ Contributing ============ Development of ``django-celery`` happens at Github: http://github.com/ask/django-celery You are highly encouraged to participate in the development. If you don't like Github (for some reason) you're welcome to send regular patches. License ======= This software is licensed under the ``New BSD License``. See the ``LICENSE`` file in the top distribution directory for the full license text. .. # vim: syntax=rst expandtab tabstop=4 shiftwidth=4 shiftround django-celery-2.5.5/docs/Makefile0000644000076500000240000000472111656051573017272 0ustar asksolstaff00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d .build/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html web pickle htmlhelp latex changes linkcheck help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " changes to make an overview over all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" clean: -rm -rf .build/* html: mkdir -p .build/html .build/doctrees $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) .build/html @echo @echo "Build finished. The HTML pages are in .build/html." coverage: mkdir -p .build/coverage .build/doctrees $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) .build/coverage @echo @echo "Build finished." pickle: mkdir -p .build/pickle .build/doctrees $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) .build/pickle @echo @echo "Build finished; now you can process the pickle files." web: pickle json: mkdir -p .build/json .build/doctrees $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) .build/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: mkdir -p .build/htmlhelp .build/doctrees $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) .build/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in .build/htmlhelp." latex: mkdir -p .build/latex .build/doctrees $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) .build/latex @echo @echo "Build finished; the LaTeX files are in .build/latex." @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ "run these through (pdf)latex." changes: mkdir -p .build/changes .build/doctrees $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) .build/changes @echo @echo "The overview file is in .build/changes." linkcheck: mkdir -p .build/linkcheck .build/doctrees $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) .build/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in .build/linkcheck/output.txt." django-celery-2.5.5/docs/reference/0000755000076500000240000000000011744004562017556 5ustar asksolstaff00000000000000django-celery-2.5.5/docs/reference/djcelery.app.rst0000644000076500000240000000031111656051574022672 0ustar asksolstaff00000000000000============================ App - djcelery.app ============================ .. contents:: :local: .. currentmodule:: djcelery.app .. automodule:: djcelery.app :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.backends.cache.rst0000644000076500000240000000041611656051574024734 0ustar asksolstaff00000000000000========================================= Cache Backend - djcelery.backends.cache ========================================= .. contents:: :local: .. currentmodule:: djcelery.backends.cache .. automodule:: djcelery.backends.cache :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.backends.database.rst0000644000076500000240000000044611656051574025440 0ustar asksolstaff00000000000000=============================================== Database Backend - djcelery.backends.database =============================================== .. contents:: :local: .. currentmodule:: djcelery.backends.database .. automodule:: djcelery.backends.database :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.contrib.test_runner.rst0000644000076500000240000000047411656051574026133 0ustar asksolstaff00000000000000===================================================== Contrib: Test Runner - djcelery.contrib.test_runner ===================================================== .. contents:: :local: .. currentmodule:: djcelery.contrib.test_runner .. automodule:: djcelery.contrib.test_runner :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.humanize.rst0000644000076500000240000000036311723671451023736 0ustar asksolstaff00000000000000==================================== Humanize utils - djcelery.humanize ==================================== .. contents:: :local: .. currentmodule:: djcelery.humanize .. automodule:: djcelery.humanize :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.loaders.rst0000644000076500000240000000035611656051574023554 0ustar asksolstaff00000000000000=================================== Celery Loaders - djcelery.loaders =================================== .. contents:: :local: .. currentmodule:: djcelery.loaders .. automodule:: djcelery.loaders :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.managers.rst0000644000076500000240000000034111656051574023712 0ustar asksolstaff00000000000000============================== Managers - djcelery.managers ============================== .. contents:: :local: .. currentmodule:: djcelery.managers .. automodule:: djcelery.managers :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.models.rst0000644000076500000240000000337011656051574023405 0ustar asksolstaff00000000000000=============================== Django Models - celery.models =============================== .. contents:: :local: .. data:: TASK_STATUS_PENDING The string status of a pending task. .. data:: TASK_STATUS_RETRY The string status of a task which is to be retried. .. data:: TASK_STATUS_FAILURE The string status of a failed task. .. data:: TASK_STATUS_DONE The string status of a task that was successfully executed. .. data:: TASK_STATUSES List of possible task statuses. .. data:: TASK_STATUSES_CHOICES Django tuple of possible values for the task statuses, for usage in model/form fields ``choices`` argument. .. class:: TaskMeta Model for storing the result and status of a task. *Note* Only used if you're running the ``database`` backend. .. attribute:: task_id The unique task id. .. attribute:: status The current status for this task. .. attribute:: result The result after successful/failed execution. If the task failed, this contains the execption it raised. .. attribute:: date_done The date this task changed status. .. class:: PeriodicTaskMeta Metadata model for periodic tasks. .. attribute:: name The name of this task, as registered in the task registry. .. attribute:: last_run_at The date this periodic task was last run. Used to find out when it should be run next. .. attribute:: total_run_count The number of times this periodic task has been run. .. attribute:: task The class/function for this task. .. method:: delay() Delay the execution of a periodic task, and increment its total run count. django-celery-2.5.5/docs/reference/djcelery.schedulers.rst0000644000076500000240000000043311656051574024260 0ustar asksolstaff00000000000000================================================ Periodic Task Schedulers - djcelery.schedulers ================================================ .. contents:: :local: .. currentmodule:: djcelery.schedulers .. automodule:: djcelery.schedulers :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.snapshot.rst0000644000076500000240000000036611656051574023763 0ustar asksolstaff00000000000000===================================== Event Snapshots - djcelery.snapshot ===================================== .. contents:: :local: .. currentmodule:: djcelery.snapshot .. automodule:: djcelery.snapshot :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.urls.rst0000644000076500000240000000030111656051574023076 0ustar asksolstaff00000000000000====================== URLs - djcelery.urls ====================== .. contents:: :local: .. currentmodule:: djcelery.urls .. automodule:: djcelery.urls :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.utils.rst0000644000076500000240000000032511656051574023257 0ustar asksolstaff00000000000000============================ Utilities - djcelery.utils ============================ .. contents:: :local: .. currentmodule:: djcelery.utils .. automodule:: djcelery.utils :members: :undoc-members: django-celery-2.5.5/docs/reference/djcelery.views.rst0000644000076500000240000000031111656051574023247 0ustar asksolstaff00000000000000======================== Views - djcelery.views ======================== .. contents:: :local: .. currentmodule:: djcelery.views .. automodule:: djcelery.views :members: :undoc-members: django-celery-2.5.5/docs/reference/index.rst0000644000076500000240000000063311723671451021425 0ustar asksolstaff00000000000000=============== API Reference =============== :Release: |version| :Date: |today| .. toctree:: :maxdepth: 2 djcelery.app djcelery.views djcelery.urls djcelery.models djcelery.managers djcelery.loaders djcelery.schedulers djcelery.snapshot djcelery.backends.database djcelery.backends.cache djcelery.contrib.test_runner djcelery.humanize djcelery.utils django-celery-2.5.5/docs/settings.py0000644000076500000240000000114211721735236020034 0ustar asksolstaff00000000000000# Django settings for docs project. # import source code dir import os import sys sys.path.insert(0, os.getcwd()) sys.path.insert(0, os.path.join(os.getcwd(), os.pardir)) SITE_ID = 303 DEBUG = True TEMPLATE_DEBUG = DEBUG DATABASES = {"default": {"NAME": ":memory:", "ENGINE": "django.db.backends.sqlite3", "USER": '', "PASSWORD": '', "PORT": ''}} INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'djcelery', ) django-celery-2.5.5/examples/0000755000076500000240000000000011744004562016506 5ustar asksolstaff00000000000000django-celery-2.5.5/examples/demoproject/0000755000076500000240000000000011744004562021021 5ustar asksolstaff00000000000000django-celery-2.5.5/examples/demoproject/demoapp/0000755000076500000240000000000011744004562022446 5ustar asksolstaff00000000000000django-celery-2.5.5/examples/demoproject/demoapp/__init__.py0000644000076500000240000000000011741576434024556 0ustar asksolstaff00000000000000django-celery-2.5.5/examples/demoproject/demoapp/models.py0000644000076500000240000000007111743040560024276 0ustar asksolstaff00000000000000from django.db import models # Create your models here. django-celery-2.5.5/examples/demoproject/demoapp/tasks.py0000644000076500000240000000033311742044217024143 0ustar asksolstaff00000000000000from djcelery import celery @celery.task def add(x, y): return x + y @celery.task def sleeptask(i): from time import sleep sleep(i) return i @celery.task def raisetask(): raise KeyError("foo") django-celery-2.5.5/examples/demoproject/demoapp/tests.py0000644000076500000240000000057711742044217024172 0ustar asksolstaff00000000000000""" This file demonstrates writing tests using the unittest module. These will pass when you run "manage.py test". Replace this with more appropriate tests for your application. """ from django.test import TestCase class SimpleTest(TestCase): def test_basic_addition(self): """ Tests that 1 + 1 always equals 2. """ self.assertEqual(1 + 1, 2) django-celery-2.5.5/examples/demoproject/demoapp/views.py0000644000076500000240000000026011742044217024152 0ustar asksolstaff00000000000000# Create your views here. from django.http import HttpResponse from demoapp import tasks def foo(request): r = tasks.add.delay(2, 2) return HttpResponse(r.task_id) django-celery-2.5.5/examples/demoproject/demoproject/0000755000076500000240000000000011744004562023334 5ustar asksolstaff00000000000000django-celery-2.5.5/examples/demoproject/demoproject/__init__.py0000644000076500000240000000000011742044217025432 0ustar asksolstaff00000000000000django-celery-2.5.5/examples/demoproject/demoproject/settings.py0000644000076500000240000001210411742044217025543 0ustar asksolstaff00000000000000# Django settings for demoproject project. # -- Celery related configuration import djcelery djcelery.setup_loader() BROKER_URL = "amqp://guest:guest@localhost:5672//" CELERY_RESULT_BACKEND = "database" # -- other necessary settings. DATABASES = {"default": {"NAME": "testdb.sqlite", "ENGINE": "django.db.backends.sqlite3", "USER": '', "PASSWORD": '', "HOST": '', "PORT": ''}} INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.admin', 'djcelery', 'demoapp', ) # -- rest is from Django 1.4 template DEBUG = True TEMPLATE_DEBUG = DEBUG ADMINS = ( # ('Your Name', 'your_email@example.com'), ) MANAGERS = ADMINS # Local time zone for this installation. Choices can be found here: # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name # although not all choices may be available on all operating systems. # On Unix systems, a value of None will cause Django to use the same # timezone as the operating system. # If running in a Windows environment this must be set to the same as your # system time zone. TIME_ZONE = 'America/Chicago' # Language code for this installation. All choices can be found here: # http://www.i18nguy.com/unicode/language-identifiers.html LANGUAGE_CODE = 'en-us' SITE_ID = 1 # If you set this to False, Django will make some optimizations so as not # to load the internationalization machinery. USE_I18N = True # If you set this to False, Django will not format dates, numbers and # calendars according to the current locale. USE_L10N = True # If you set this to False, Django will not use timezone-aware datetimes. USE_TZ = True # Absolute filesystem path to the directory that will hold user-uploaded files. # Example: "/home/media/media.lawrence.com/media/" MEDIA_ROOT = '' # URL that handles the media served from MEDIA_ROOT. Make sure to use a # trailing slash. # Examples: "http://media.lawrence.com/media/", "http://example.com/media/" MEDIA_URL = '' # Absolute path to the directory static files should be collected to. # Don't put anything in this directory yourself; store your static files # in apps' "static/" subdirectories and in STATICFILES_DIRS. # Example: "/home/media/media.lawrence.com/static/" STATIC_ROOT = '' # URL prefix for static files. # Example: "http://media.lawrence.com/static/" STATIC_URL = '/static/' # Additional locations of static files STATICFILES_DIRS = ( # Put strings here, like "/home/html/static" or "C:/www/django/static". # Always use forward slashes, even on Windows. # Don't forget to use absolute paths, not relative paths. ) # List of finder classes that know how to find static files in # various locations. STATICFILES_FINDERS = ( 'django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', # 'django.contrib.staticfiles.finders.DefaultStorageFinder', ) # Make this unique, and don't share it with anybody. SECRET_KEY = 'utpl%umodw@pb=t3v8e6c8+)f&ra41gon9l&5+3566nr88%b3y' # List of callables that know how to import templates from various sources. TEMPLATE_LOADERS = ( 'django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader', # 'django.template.loaders.eggs.Loader', ) MIDDLEWARE_CLASSES = ( 'django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', # Uncomment the next line for simple clickjacking protection: # 'django.middleware.clickjacking.XFrameOptionsMiddleware', ) ROOT_URLCONF = 'demoproject.urls' # Python dotted path to the WSGI application used by Django's runserver. WSGI_APPLICATION = 'demoproject.wsgi.application' TEMPLATE_DIRS = ( # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates". # Always use forward slashes, even on Windows. # Don't forget to use absolute paths, not relative paths. ) # A sample logging configuration. The only tangible logging # performed by this configuration is to send an email to # the site admins on every HTTP 500 error when DEBUG=False. # See http://docs.djangoproject.com/en/dev/topics/logging for # more details on how to customize your logging configuration. LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'filters': { 'require_debug_false': { '()': 'django.utils.log.RequireDebugFalse' } }, 'handlers': { 'mail_admins': { 'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler' } }, 'loggers': { 'django.request': { 'handlers': ['mail_admins'], 'level': 'ERROR', 'propagate': True, }, } } django-celery-2.5.5/examples/demoproject/demoproject/urls.py0000644000076500000240000000114511742044217024673 0ustar asksolstaff00000000000000from django.conf.urls.defaults import patterns, include from demoapp import views # Uncomment the next two lines to enable the admin: from django.contrib import admin admin.autodiscover() urlpatterns = patterns('', # Example: # (r'^demoproject/', include('demoproject.foo.urls')), # Uncomment the admin/doc line below and add 'django.contrib.admindocs' # to INSTALLED_APPS to enable admin documentation: (r'^admin/doc/', include('django.contrib.admindocs.urls')), # Uncomment the next line to enable the admin: (r'^admin/', include(admin.site.urls)), (r'^foo/', views.foo), ) django-celery-2.5.5/examples/demoproject/demoproject/wsgi.py0000644000076500000240000000217011742044217024656 0ustar asksolstaff00000000000000""" WSGI config for demoproject project. This module contains the WSGI application used by Django's development server and any production WSGI deployments. It should expose a module-level variable named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover this application via the ``WSGI_APPLICATION`` setting. Usually you will have the standard Django WSGI application here, but it also might make sense to replace the whole Django WSGI application with a custom one that later delegates to the Django one. For example, you could introduce WSGI middleware here, or combine a Django application with an application of another framework. """ import os os.environ.setdefault("DJANGO_SETTINGS_MODULE", "demoproject.settings") # This application object is used by any WSGI server configured to use this # file. This includes Django's development server, if the WSGI_APPLICATION # setting points here. from django.core.wsgi import get_wsgi_application application = get_wsgi_application() # Apply WSGI middleware here. # from helloworld.wsgi import HelloWorldApplication # application = HelloWorldApplication(application) django-celery-2.5.5/examples/demoproject/manage.py0000644000076500000240000000037611742044217022630 0ustar asksolstaff00000000000000#!/usr/bin/env python import os import sys if __name__ == "__main__": os.environ.setdefault("DJANGO_SETTINGS_MODULE", "demoproject.settings") from django.core.management import execute_from_command_line execute_from_command_line(sys.argv) django-celery-2.5.5/FAQ0000644000076500000240000000000011656051572022635 1django-celery-2.5.5/docs/faq.rstustar asksolstaff00000000000000django-celery-2.5.5/INSTALL0000644000076500000240000000066111656051572015731 0ustar asksolstaff00000000000000Installing django-celery ======================== You can install ``django-celery`` either via the Python Package Index (PyPI) or from source. To install using ``pip``,:: $ pip install django-celery To install using ``easy_install``,:: $ easy_install django-celery If you have downloaded a source tarball you can install it by doing the following,:: $ python setup.py build # python setup.py install # as root django-celery-2.5.5/LICENSE0000644000076500000240000000272711656051572015712 0ustar asksolstaff00000000000000Copyright (c) 2009-2011, Ask Solem All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of Ask Solem nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. django-celery-2.5.5/locale/0000755000076500000240000000000011744004562016127 5ustar asksolstaff00000000000000django-celery-2.5.5/locale/en/0000755000076500000240000000000011744004562016531 5ustar asksolstaff00000000000000django-celery-2.5.5/locale/en/LC_MESSAGES/0000755000076500000240000000000011744004562020316 5ustar asksolstaff00000000000000django-celery-2.5.5/locale/en/LC_MESSAGES/django.po0000644000076500000240000001405711656051574022136 0ustar asksolstaff00000000000000# SOME DESCRIPTIVE TITLE. # Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER # This file is distributed under the same license as the PACKAGE package. # FIRST AUTHOR , YEAR. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: PACKAGE VERSION\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2010-11-25 11:55+0100\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language-Team: LANGUAGE \n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" #: djcelery/admin.py:62 djcelery/admin.py:69 djcelery/models.py:258 msgid "state" msgstr "" #: djcelery/admin.py:76 djcelery/models.py:268 msgid "ETA" msgstr "" #: djcelery/admin.py:83 msgid "when" msgstr "" #: djcelery/admin.py:89 djcelery/models.py:163 djcelery/models.py:263 msgid "name" msgstr "" #: djcelery/admin.py:141 msgid "Task detail" msgstr "" #: djcelery/admin.py:142 msgid "Tasks" msgstr "" #: djcelery/admin.py:156 djcelery/models.py:261 msgid "UUID" msgstr "" #: djcelery/admin.py:172 msgid "Revoke selected tasks" msgstr "" #: djcelery/admin.py:181 djcelery/templates/djcelery/confirm_rate_limit.html:9 msgid "Rate limit selected tasks" msgstr "" #: djcelery/admin.py:197 msgid "Rate limit selection" msgstr "" #: djcelery/admin.py:221 msgid "Node detail" msgstr "" #: djcelery/admin.py:222 msgid "Worker Nodes" msgstr "" #: djcelery/admin.py:229 msgid "Shutdown selected worker nodes" msgstr "" #: djcelery/admin.py:233 msgid "Enable event mode for selected nodes." msgstr "" #: djcelery/admin.py:238 msgid "Disable event mode for selected nodes." msgstr "" #: djcelery/admin.py:267 msgid "Task (registered)" msgstr "" #: djcelery/admin.py:269 msgid "Task (custom)" msgstr "" #: djcelery/admin.py:281 msgid "Need name of task" msgstr "" #: djcelery/models.py:26 djcelery/models.py:54 msgid "task id" msgstr "" #: djcelery/models.py:27 msgid "task status" msgstr "" #: djcelery/models.py:30 djcelery/models.py:56 msgid "done at" msgstr "" #: djcelery/models.py:31 djcelery/models.py:272 msgid "traceback" msgstr "" #: djcelery/models.py:37 djcelery/models.py:38 msgid "task meta" msgstr "" #: djcelery/models.py:62 djcelery/models.py:63 msgid "taskset meta" msgstr "" #: djcelery/models.py:75 msgid "Days" msgstr "" #: djcelery/models.py:76 msgid "Hours" msgstr "" #: djcelery/models.py:77 msgid "Minutes" msgstr "" #: djcelery/models.py:78 msgid "Seconds" msgstr "" #: djcelery/models.py:79 msgid "Microseconds" msgstr "" #: djcelery/models.py:83 msgid "every" msgstr "" #: djcelery/models.py:84 msgid "period" msgstr "" #: djcelery/models.py:88 djcelery/models.py:167 msgid "interval" msgstr "" #: djcelery/models.py:89 msgid "intervals" msgstr "" #: djcelery/models.py:102 #, python-format msgid "every %(period)s" msgstr "" #: djcelery/models.py:103 #, python-format msgid "every $(every)s %(period)s" msgstr "" #: djcelery/models.py:108 msgid "minute" msgstr "" #: djcelery/models.py:111 msgid "hour" msgstr "" #: djcelery/models.py:114 msgid "day of week" msgstr "" #: djcelery/models.py:119 djcelery/models.py:169 msgid "crontab" msgstr "" #: djcelery/models.py:120 msgid "crontabs" msgstr "" #: djcelery/models.py:164 msgid "Useful description" msgstr "" #: djcelery/models.py:165 msgid "task name" msgstr "" #: djcelery/models.py:170 msgid "Use one of interval/crontab" msgstr "" #: djcelery/models.py:171 djcelery/models.py:266 msgid "Arguments" msgstr "" #: djcelery/models.py:173 msgid "JSON encoded positional arguments" msgstr "" #: djcelery/models.py:174 djcelery/models.py:267 msgid "Keyword arguments" msgstr "" #: djcelery/models.py:176 msgid "JSON encoded keyword arguments" msgstr "" #: djcelery/models.py:177 msgid "queue" msgstr "" #: djcelery/models.py:180 msgid "Queue defined in CELERY_QUEUES" msgstr "" #: djcelery/models.py:181 msgid "exchange" msgstr "" #: djcelery/models.py:184 msgid "routing key" msgstr "" #: djcelery/models.py:187 djcelery/models.py:270 msgid "expires" msgstr "" #: djcelery/models.py:189 msgid "enabled" msgstr "" #: djcelery/models.py:200 msgid "periodic task" msgstr "" #: djcelery/models.py:201 msgid "periodic tasks" msgstr "" #: djcelery/models.py:228 msgid "hostname" msgstr "" #: djcelery/models.py:229 msgid "last heartbeat" msgstr "" #: djcelery/models.py:236 djcelery/models.py:277 msgid "worker" msgstr "" #: djcelery/models.py:237 msgid "workers" msgstr "" #: djcelery/models.py:265 msgid "event received at" msgstr "" #: djcelery/models.py:271 msgid "result" msgstr "" #: djcelery/models.py:273 msgid "execution time" msgstr "" #: djcelery/models.py:274 msgid "in seconds if task successful" msgstr "" #: djcelery/models.py:275 msgid "number of retries" msgstr "" #: djcelery/models.py:284 msgid "task" msgstr "" #: djcelery/models.py:285 msgid "tasks" msgstr "" #: djcelery/utils.py:5 msgid "just now" msgstr "" #: djcelery/utils.py:6 #, python-format msgid "%(seconds)d second ago" msgstr "" #: djcelery/utils.py:6 #, python-format msgid "%(seconds)d seconds ago" msgstr "" #: djcelery/utils.py:7 #, python-format msgid "%(minutes)d minute ago" msgstr "" #: djcelery/utils.py:7 #, python-format msgid "%(minutes)d minutes ago" msgstr "" #: djcelery/utils.py:8 #, python-format msgid "%(hours)d hour ago" msgstr "" #: djcelery/utils.py:8 #, python-format msgid "%(hours)d hours ago" msgstr "" #: djcelery/utils.py:9 #, python-format msgid "yesterday at %(time)s" msgstr "" #: djcelery/utils.py:10 msgid "year" msgstr "" #: djcelery/utils.py:10 msgid "years" msgstr "" #: djcelery/utils.py:11 msgid "month" msgstr "" #: djcelery/utils.py:11 msgid "months" msgstr "" #: djcelery/utils.py:12 msgid "week" msgstr "" #: djcelery/utils.py:12 msgid "weeks" msgstr "" #: djcelery/utils.py:13 msgid "day" msgstr "" #: djcelery/utils.py:13 msgid "days" msgstr "" #: djcelery/utils.py:20 #, python-format msgid "%(number)d %(type)s ago" msgstr "" #: djcelery/templates/admin/djcelery/change_list.html:7 #: djcelery/templates/djcelery/confirm_rate_limit.html:6 msgid "Home" msgstr "" #: djcelery/templates/djcelery/confirm_rate_limit.html:22 msgid "Rate limit" msgstr "" django-celery-2.5.5/MANIFEST.in0000644000076500000240000000110011656051572016423 0ustar asksolstaff00000000000000include AUTHORS include Changelog include FAQ include INSTALL include LICENSE include MANIFEST.in include README include README.rst include THANKS include TODO include setup.cfg recursive-include bin * recursive-include contrib * recursive-include locale * recursive-include djcelery *.py recursive-include djcelery/templates * recursive-include docs * recursive-include examples * recursive-include tests * recursive-include requirements *.txt prune bin/*.pyc prune tests/*.pyc prune docs/*.pyc prune contrib/*.pyc prune djcelery/*.pyc prune docs/.build prune examples/*.pyc django-celery-2.5.5/PKG-INFO0000644000076500000240000001564211744004562015775 0ustar asksolstaff00000000000000Metadata-Version: 1.0 Name: django-celery Version: 2.5.5 Summary: Django Celery Integration. Home-page: http://celeryproject.org Author: Ask Solem Author-email: ask@celeryproject.org License: BSD Description: =============================================== django-celery - Celery Integration for Django =============================================== .. image:: http://cloud.github.com/downloads/ask/celery/celery_128.png :Version: 2.5.5 :Web: http://celeryproject.org/ :Download: http://pypi.python.org/pypi/django-celery/ :Source: http://github.com/ask/django-celery/ :Keywords: celery, task queue, job queue, asynchronous, rabbitmq, amqp, redis, python, django, webhooks, queue, distributed -- django-celery provides Celery integration for Django; Using the Django ORM and cache backend for storing results, autodiscovery of task modules for applications listed in ``INSTALLED_APPS``, and more. .. contents:: :local: Using django-celery =================== To enable ``django-celery`` for your project you need to add ``djcelery`` to ``INSTALLED_APPS``:: INSTALLED_APPS += ("djcelery", ) then add the following lines to your ``settings.py``:: import djcelery djcelery.setup_loader() Everything works the same as described in the `Celery User Manual`_, except you need to invoke the programs through ``manage.py``: ===================================== ===================================== **Program** **Replace with** ===================================== ===================================== ``celeryd`` ``python manage.py celeryd`` ``celeryctl`` ``python manage.py celeryctl`` ``celerybeat`` ``python manage.py celerybeat`` ``camqadm`` ``python manage.py camqadm`` ``celeryev`` ``python manage.py celeryev`` ``celeryd-multi`` ``python manage.py celeryd_multi`` ===================================== ===================================== The other main difference is that configuration values are stored in your Django projects' ``settings.py`` module rather than in ``celeryconfig.py``. If you're trying celery for the first time you should start by reading `Getting started with django-celery`_ Special note for mod_wsgi users ------------------------------- If you're using ``mod_wsgi`` to deploy your Django application you need to include the following in your ``.wsgi`` module:: import djcelery djcelery.setup_loader() Documentation ============= The `Celery User Manual`_ contains user guides, tutorials and an API reference. Also the `django-celery documentation`_, contains information about the Django integration. .. _`django-celery documentation`: http://django-celery.readthedocs.org/ .. _`Celery User Manual`: http://docs.celeryproject.org/ .. _`Getting started with django-celery`: http://django-celery.readthedocs.org/en/latest/getting-started/first-steps-with-django.html Installation ============= You can install ``django-celery`` either via the Python Package Index (PyPI) or from source. To install using ``pip``,:: $ pip install django-celery To install using ``easy_install``,:: $ easy_install django-celery You will then want to create the necessary tables. If you are using south_ for schema migrations, you'll want to:: $ python manage.py migrate djcelery For those who are not using south, a normal ``syncdb`` will work:: $ python manage.py syncdb .. _south: http://pypi.python.org/pypi/South/ Downloading and installing from source -------------------------------------- Download the latest version of ``django-celery`` from http://pypi.python.org/pypi/django-celery/ You can install it by doing the following,:: $ tar xvfz django-celery-0.0.0.tar.gz $ cd django-celery-0.0.0 # python setup.py install # as root Using the development version ------------------------------ You can clone the git repository by doing the following:: $ git clone git://github.com/ask/django-celery.git Getting Help ============ Mailing list ------------ For discussions about the usage, development, and future of celery, please join the `celery-users`_ mailing list. .. _`celery-users`: http://groups.google.com/group/celery-users/ IRC --- Come chat with us on IRC. The `#celery`_ channel is located at the `Freenode`_ network. .. _`#celery`: irc://irc.freenode.net/celery .. _`Freenode`: http://freenode.net Bug tracker =========== If you have any suggestions, bug reports or annoyances please report them to our issue tracker at http://github.com/ask/django-celery/issues/ Wiki ==== http://wiki.github.com/ask/celery/ Contributing ============ Development of ``django-celery`` happens at Github: http://github.com/ask/django-celery You are highly encouraged to participate in the development. If you don't like Github (for some reason) you're welcome to send regular patches. License ======= This software is licensed under the ``New BSD License``. See the ``LICENSE`` file in the top distribution directory for the full license text. .. # vim: syntax=rst expandtab tabstop=4 shiftwidth=4 shiftround Platform: any Classifier: Development Status :: 5 - Production/Stable Classifier: Framework :: Django Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: POSIX Classifier: Topic :: Communications Classifier: Topic :: System :: Distributed Computing Classifier: Topic :: Software Development :: Libraries :: Python Modules django-celery-2.5.5/README0000644000076500000240000001162111744004536015552 0ustar asksolstaff00000000000000=============================================== django-celery - Celery Integration for Django =============================================== .. image:: http://cloud.github.com/downloads/ask/celery/celery_128.png :Version: 2.5.5 :Web: http://celeryproject.org/ :Download: http://pypi.python.org/pypi/django-celery/ :Source: http://github.com/ask/django-celery/ :Keywords: celery, task queue, job queue, asynchronous, rabbitmq, amqp, redis, python, django, webhooks, queue, distributed -- django-celery provides Celery integration for Django; Using the Django ORM and cache backend for storing results, autodiscovery of task modules for applications listed in ``INSTALLED_APPS``, and more. .. contents:: :local: Using django-celery =================== To enable ``django-celery`` for your project you need to add ``djcelery`` to ``INSTALLED_APPS``:: INSTALLED_APPS += ("djcelery", ) then add the following lines to your ``settings.py``:: import djcelery djcelery.setup_loader() Everything works the same as described in the `Celery User Manual`_, except you need to invoke the programs through ``manage.py``: ===================================== ===================================== **Program** **Replace with** ===================================== ===================================== ``celeryd`` ``python manage.py celeryd`` ``celeryctl`` ``python manage.py celeryctl`` ``celerybeat`` ``python manage.py celerybeat`` ``camqadm`` ``python manage.py camqadm`` ``celeryev`` ``python manage.py celeryev`` ``celeryd-multi`` ``python manage.py celeryd_multi`` ===================================== ===================================== The other main difference is that configuration values are stored in your Django projects' ``settings.py`` module rather than in ``celeryconfig.py``. If you're trying celery for the first time you should start by reading `Getting started with django-celery`_ Special note for mod_wsgi users ------------------------------- If you're using ``mod_wsgi`` to deploy your Django application you need to include the following in your ``.wsgi`` module:: import djcelery djcelery.setup_loader() Documentation ============= The `Celery User Manual`_ contains user guides, tutorials and an API reference. Also the `django-celery documentation`_, contains information about the Django integration. .. _`django-celery documentation`: http://django-celery.readthedocs.org/ .. _`Celery User Manual`: http://docs.celeryproject.org/ .. _`Getting started with django-celery`: http://django-celery.readthedocs.org/en/latest/getting-started/first-steps-with-django.html Installation ============= You can install ``django-celery`` either via the Python Package Index (PyPI) or from source. To install using ``pip``,:: $ pip install django-celery To install using ``easy_install``,:: $ easy_install django-celery You will then want to create the necessary tables. If you are using south_ for schema migrations, you'll want to:: $ python manage.py migrate djcelery For those who are not using south, a normal ``syncdb`` will work:: $ python manage.py syncdb .. _south: http://pypi.python.org/pypi/South/ Downloading and installing from source -------------------------------------- Download the latest version of ``django-celery`` from http://pypi.python.org/pypi/django-celery/ You can install it by doing the following,:: $ tar xvfz django-celery-0.0.0.tar.gz $ cd django-celery-0.0.0 # python setup.py install # as root Using the development version ------------------------------ You can clone the git repository by doing the following:: $ git clone git://github.com/ask/django-celery.git Getting Help ============ Mailing list ------------ For discussions about the usage, development, and future of celery, please join the `celery-users`_ mailing list. .. _`celery-users`: http://groups.google.com/group/celery-users/ IRC --- Come chat with us on IRC. The `#celery`_ channel is located at the `Freenode`_ network. .. _`#celery`: irc://irc.freenode.net/celery .. _`Freenode`: http://freenode.net Bug tracker =========== If you have any suggestions, bug reports or annoyances please report them to our issue tracker at http://github.com/ask/django-celery/issues/ Wiki ==== http://wiki.github.com/ask/celery/ Contributing ============ Development of ``django-celery`` happens at Github: http://github.com/ask/django-celery You are highly encouraged to participate in the development. If you don't like Github (for some reason) you're welcome to send regular patches. License ======= This software is licensed under the ``New BSD License``. See the ``LICENSE`` file in the top distribution directory for the full license text. .. # vim: syntax=rst expandtab tabstop=4 shiftwidth=4 shiftround django-celery-2.5.5/README.rst0000644000076500000240000000000011744004536022415 1django-celery-2.5.5/READMEustar asksolstaff00000000000000django-celery-2.5.5/requirements/0000755000076500000240000000000011744004562017413 5ustar asksolstaff00000000000000django-celery-2.5.5/requirements/default.txt0000644000076500000240000000005011742044217021572 0ustar asksolstaff00000000000000django-picklefield>=0.2.0 celery>=2.5.2 django-celery-2.5.5/requirements/docs.txt0000644000076500000240000000007211721712706021104 0ustar asksolstaff00000000000000Django Sphinx sphinxcontrib-issuetracker python-memcached django-celery-2.5.5/requirements/test.txt0000644000076500000240000000013111656051574021135 0ustar asksolstaff00000000000000unittest2>=0.4.0 coverage>=3.0 nose nose-cover3 mock django django-nose python-memcached django-celery-2.5.5/setup.cfg0000644000076500000240000000037011744004562016511 0ustar asksolstaff00000000000000[build_sphinx] source-dir = docs/ build-dir = docs/.build all_files = 1 [upload_sphinx] upload-dir = docs/.build/html [bdist_rpm] requires = django-picklefield >= 0.2.0 celery >= 2.5.2 [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 django-celery-2.5.5/setup.py0000755000076500000240000001256111743053521016410 0ustar asksolstaff00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- import os import sys import codecs try: from setuptools import setup, Command except ImportError: from ez_setup import use_setuptools use_setuptools() from setuptools import setup, Command # noqa from distutils.command.install import INSTALL_SCHEMES # -*- Distribution Meta -*- NAME = "django-celery" import re re_meta = re.compile(r'__(\w+?)__\s*=\s*(.*)') re_vers = re.compile(r'VERSION\s*=\s*\((.*?)\)') re_doc = re.compile(r'^"""(.+?)"""') rq = lambda s: s.strip("\"'") def add_default(m): attr_name, attr_value = m.groups() return ((attr_name, rq(attr_value)), ) def add_version(m): v = list(map(rq, m.groups()[0].split(", "))) return (("VERSION", ".".join(v[0:3]) + "".join(v[3:])), ) def add_doc(m): return (("doc", m.groups()[0]), ) pats = {re_meta: add_default, re_vers: add_version, re_doc: add_doc} here = os.path.abspath(os.path.dirname(__file__)) meta_fh = open(os.path.join(here, "djcelery/__init__.py")) try: meta = {} for line in meta_fh: if line.strip() == '# -eof meta-': break for pattern, handler in pats.items(): m = pattern.match(line.strip()) if m: meta.update(handler(m)) finally: meta_fh.close() packages, data_files = [], [] root_dir = os.path.dirname(__file__) if root_dir != '': os.chdir(root_dir) src_dir = "djcelery" def fullsplit(path, result=None): if result is None: result = [] head, tail = os.path.split(path) if head == '': return [tail] + result if head == path: return result return fullsplit(head, [tail] + result) for scheme in INSTALL_SCHEMES.values(): scheme['data'] = scheme['purelib'] SKIP_EXTENSIONS = [".pyc", ".pyo", ".swp", ".swo"] def is_unwanted_file(filename): for skip_ext in SKIP_EXTENSIONS: if filename.endswith(skip_ext): return True return False for dirpath, dirnames, filenames in os.walk(src_dir): # Ignore dirnames that start with '.' for i, dirname in enumerate(dirnames): if dirname.startswith("."): del dirnames[i] for filename in filenames: if filename.endswith(".py"): packages.append('.'.join(fullsplit(dirpath))) elif is_unwanted_file(filename): pass else: data_files.append([dirpath, [os.path.join(dirpath, f) for f in filenames]]) class RunTests(Command): description = "Run the django test suite from the tests dir." user_options = [] extra_env = {} extra_args = [] def run(self): for env_name, env_value in self.extra_env.items(): os.environ[env_name] = str(env_value) this_dir = os.getcwd() testproj_dir = os.path.join(this_dir, "tests") os.chdir(testproj_dir) sys.path.append(testproj_dir) from django.core.management import execute_manager os.environ["DJANGO_SETTINGS_MODULE"] = os.environ.get( "DJANGO_SETTINGS_MODULE", "settings") settings_file = os.environ["DJANGO_SETTINGS_MODULE"] settings_mod = __import__(settings_file, {}, {}, ['']) prev_argv = list(sys.argv) try: sys.argv = [__file__, "test"] + self.extra_args execute_manager(settings_mod, argv=sys.argv) finally: sys.argv = prev_argv def initialize_options(self): pass def finalize_options(self): pass class QuickRunTests(RunTests): extra_env = dict(SKIP_RLIMITS=1, QUICKTEST=1) class CIRunTests(RunTests): @property def extra_args(self): toxinidir = os.environ.get("TOXINIDIR", "") return [ "--with-coverage3", "--cover3-xml", "--cover3-xml-file=%s" % ( os.path.join(toxinidir, "coverage.xml"), ), "--with-xunit", "--xunit-file=%s" % ( os.path.join(toxinidir, "nosetests.xml"), ), "--cover3-html", "--cover3-html-dir=%s" % ( os.path.join(toxinidir, "cover"), ), ] if os.path.exists("README.rst"): long_description = codecs.open("README.rst", "r", "utf-8").read() else: long_description = "See http://github.com/ask/django-celery" setup( name=NAME, version=meta["VERSION"], description=meta["doc"], author=meta["author"], author_email=meta["contact"], url=meta["homepage"], platforms=["any"], license="BSD", packages=packages, data_files=data_files, scripts=["bin/djcelerymon"], zip_safe=False, install_requires=[ "django-picklefield>=0.2.0", "celery>=2.5.2", ], cmdclass={"test": RunTests, "quicktest": QuickRunTests, "citest": CIRunTests}, classifiers=[ "Development Status :: 5 - Production/Stable", "Framework :: Django", "Operating System :: OS Independent", "Programming Language :: Python", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: POSIX", "Topic :: Communications", "Topic :: System :: Distributed Computing", "Topic :: Software Development :: Libraries :: Python Modules", ], entry_points={ "console_scripts": ["djcelerymon = djcelery.mon:main"], }, long_description=long_description, ) django-celery-2.5.5/tests/0000755000076500000240000000000011744004562016032 5ustar asksolstaff00000000000000django-celery-2.5.5/tests/__init__.py0000644000076500000240000000000011656051574020140 0ustar asksolstaff00000000000000django-celery-2.5.5/tests/manage.py0000755000076500000240000000115111656051574017644 0ustar asksolstaff00000000000000#!/usr/bin/env python from django.core.management import execute_manager try: import settings # Assumed to be in the same directory. except ImportError: import sys sys.stderr.write( "Error: Can't find the file 'settings.py' in the directory \ containing %r. It appears you've customized things.\n\ You'll have to run django-admin.py, passing it your settings\ module.\n(If the file settings.py does indeed exist, it's\ causing an ImportError somehow.)\n" % __file__) sys.exit(1) if __name__ == "__main__": execute_manager(settings) django-celery-2.5.5/tests/settings.py0000644000076500000240000000373711741563364020265 0ustar asksolstaff00000000000000# Django settings for testproj project. import warnings warnings.filterwarnings( 'error', r"DateTimeField received a naive datetime", RuntimeWarning, r'django\.db\.models\.fields') import os import sys # import source code dir sys.path.insert(0, os.getcwd()) sys.path.insert(0, os.path.join(os.getcwd(), os.pardir)) NO_NOSE = os.environ.get("DJCELERY_NO_NOSE", False) SITE_ID = 300 DEBUG = True TEMPLATE_DEBUG = DEBUG ROOT_URLCONF = "tests.urls" ADMINS = ( # ('Your Name', 'your_email@domain.com'), ) if not NO_NOSE: TEST_RUNNER = "django_nose.run_tests" here = os.path.abspath(os.path.dirname(__file__)) COVERAGE_EXCLUDE_MODULES = ("djcelery", "djcelery.tests.*", "djcelery.management.*", "djcelery.contrib.*", ) NOSE_ARGS = [os.path.join(here, os.pardir, "djcelery", "tests"), os.environ.get("NOSE_VERBOSE") and "--verbose" or "", "--cover3-package=djcelery", "--cover3-branch", "--cover3-exclude=%s" % ",".join(COVERAGE_EXCLUDE_MODULES)] BROKER_HOST = "localhost" BROKER_PORT = 5672 BROKER_VHOST = "/" BROKER_USER = "guest" BROKER_PASSWORD = "guest" TT_HOST = "localhost" TT_PORT = 1978 CELERY_DEFAULT_EXCHANGE = "testcelery" CELERY_DEFAULT_ROUTING_KEY = "testcelery" CELERY_DEFAULT_QUEUE = "testcelery" CELERY_QUEUES = {"testcelery": {"binding_key": "testcelery"}} MANAGERS = ADMINS DATABASES = {"default": {"NAME": "djcelery-test-db", "ENGINE": "django.db.backends.sqlite3", "USER": '', "PASSWORD": '', "PORT": ''}} INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'djcelery', 'someapp', 'someappwotask', ) if not NO_NOSE: INSTALLED_APPS = INSTALLED_APPS + ("django_nose", ) CELERY_SEND_TASK_ERROR_EMAILS = False USE_TZ = True django-celery-2.5.5/tests/someapp/0000755000076500000240000000000011744004562017476 5ustar asksolstaff00000000000000django-celery-2.5.5/tests/someapp/__init__.py0000644000076500000240000000000011656051574021604 0ustar asksolstaff00000000000000django-celery-2.5.5/tests/someapp/models.py0000644000076500000240000000015511741563541021340 0ustar asksolstaff00000000000000from django.db import models # noqa class Thing(models.Model): name = models.CharField(max_length=10) django-celery-2.5.5/tests/someapp/tasks.py0000644000076500000240000000047211741563541021204 0ustar asksolstaff00000000000000from celery.task import task from django.db.models import get_model @task(name="c.unittest.SomeAppTask") def SomeAppTask(**kwargs): return 42 @task(name="c.unittest.SomeModelTask") def SomeModelTask(pk): model = get_model("someapp", "Thing") thing = model.objects.get(pk=pk) return thing.name django-celery-2.5.5/tests/someapp/tests.py0000644000076500000240000000117511741563541021222 0ustar asksolstaff00000000000000from __future__ import absolute_import from django.test.testcases import TestCase as DjangoTestCase from someapp.models import Thing from someapp.tasks import SomeModelTask class SimpleTest(DjangoTestCase): def setUp(self): self.thing = Thing.objects.create(name=u"Foo") def test_apply_task(self): "Apply task function." result = SomeModelTask.apply(kwargs={'pk': self.thing.pk}) self.assertEqual(result.get(), self.thing.name) def test_task_function(self): "Run task function." result = SomeModelTask(pk=self.thing.pk) self.assertEqual(result, self.thing.name) django-celery-2.5.5/tests/someapp/views.py0000644000076500000240000000003211656051574021207 0ustar asksolstaff00000000000000# Create your views here. django-celery-2.5.5/tests/someappwotask/0000755000076500000240000000000011744004562020727 5ustar asksolstaff00000000000000django-celery-2.5.5/tests/someappwotask/__init__.py0000644000076500000240000000000011656051574023035 0ustar asksolstaff00000000000000django-celery-2.5.5/tests/someappwotask/models.py0000644000076500000240000000010111656051574022563 0ustar asksolstaff00000000000000from django.db import models # noqa # Create your models here. django-celery-2.5.5/tests/someappwotask/views.py0000644000076500000240000000003211656051574022440 0ustar asksolstaff00000000000000# Create your views here. django-celery-2.5.5/tests/urls.py0000644000076500000240000000141211656051574017376 0ustar asksolstaff00000000000000from django.conf.urls.defaults import (patterns, url, include, # noqa handler500, handler404) from djcelery.views import apply # Uncomment the next two lines to enable the admin: # from django.contrib import admin # admin.autodiscover() urlpatterns = patterns('', # Example: # (r'^tests/', include('tests.foo.urls')), # Uncomment the admin/doc line below and add 'django.contrib.admindocs' # to INSTALLED_APPS to enable admin documentation: # (r'^admin/doc/', include('django.contrib.admindocs.urls')), # Uncomment the next line to enable the admin: # (r'^admin/(.*)', admin.site.root), url(r"^apply/(?P.+?)/", apply, name="celery-apply"), url(r"^celery/", include("djcelery.urls")), ) django-celery-2.5.5/THANKS0000644000076500000240000000055511656051573015616 0ustar asksolstaff00000000000000Thanks to Rune Halvorsen for the name. Thanks to Anton Tsigularov for the previous name (crunchy) which we had to abandon because of an existing project with that name. Thanks to Martin Mahner for the Sphinx theme. Thanks to Brian K. Jones for bunny.py (http://github.com/bkjones/bunny), the tool that inspired camqadm. django-celery-2.5.5/TODO0000644000076500000240000000012711656051573015366 0ustar asksolstaff00000000000000Please see our Issue Tracker at GitHub: http://github.com/ask/django-celery/issues