././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1621464038.2324305 haystack-redis-0.0.1/0000755000076500000240000000000000000000000014134 5ustar00steinbrostaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621463817.0 haystack-redis-0.0.1/LICENSE0000644000076500000240000000242100000000000015140 0ustar00steinbrostaffCopyright (c) 2012, Jökull Sólberg Auðunsson All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1621464038.2317104 haystack-redis-0.0.1/PKG-INFO0000644000076500000240000000317300000000000015235 0ustar00steinbrostaffMetadata-Version: 2.1 Name: haystack-redis Version: 0.0.1 Summary: Use redis as a persistence layer for Whoosh and Haystack Home-page: https://github.com/steinbro/haystack-redis Author: Jökull Sólberg Auðunsson Author-email: jokull@solberg.is Maintainer: Daniel W. Steinbrook Maintainer-email: steinbro@post.harvard.edu License: BSD Description: haystack-redis ~~~~~~~~~~~~~~~~~~~~~~ A Whoosh storage engine using redis for persistence. A Haystack ``SearchBackend`` subclass is also provided. Normally the ``STORAGE`` key could just be set but Haystack 2.0.0alpha is only aware of ``file`` and ``ram`` backends. This is especially useful for small sites hosted on Heroku, which does not allow writing to local disk. The ``REDISTOGO_URL`` environment variable is read, falling back to the localhost default port. Usage ----- Configure your Haystack connections in ``settings.py``:: import tempfile HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack_redis.RedisEngine', 'PATH': tempfile.gettempdir(), }, } Platform: any Classifier: Environment :: Web Environment Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Software Development :: Libraries :: Python Modules ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621463817.0 haystack-redis-0.0.1/README.rst0000644000076500000240000000170000000000000015621 0ustar00steinbrostaffhaystack-redis ~~~~~~~~~~~~~~ A Whoosh storage engine using redis for persistence. A Haystack ``SearchBackend`` subclass is also provided. Normally the ``STORAGE`` key could just be set but Haystack 2.0.0beta is only aware of ``file`` and ``ram`` backends. This is especially useful for small sites hosted on Heroku, which does not allow writing to local disk. The ``REDISTOGO_URL`` environment variable is read, falling back to the localhost default port. Code is based on maxpert_’s snippet (see blog post_) .. _maxpert: https://github.com/maxpert .. _post: http://blog.creapptives.com/post/32262168370/python-whoosh-with-redis-storage Usage ----- Configure your Haystack connections in ``settings.py``:: import tempfile HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack_redis.RedisEngine', 'PATH': tempfile.gettempdir(), }, } Installation ------------ $ pip install haystack-redis ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1621464038.2309554 haystack-redis-0.0.1/haystack_redis.egg-info/0000755000076500000240000000000000000000000020623 5ustar00steinbrostaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621464038.0 haystack-redis-0.0.1/haystack_redis.egg-info/PKG-INFO0000644000076500000240000000317300000000000021724 0ustar00steinbrostaffMetadata-Version: 2.1 Name: haystack-redis Version: 0.0.1 Summary: Use redis as a persistence layer for Whoosh and Haystack Home-page: https://github.com/steinbro/haystack-redis Author: Jökull Sólberg Auðunsson Author-email: jokull@solberg.is Maintainer: Daniel W. Steinbrook Maintainer-email: steinbro@post.harvard.edu License: BSD Description: haystack-redis ~~~~~~~~~~~~~~~~~~~~~~ A Whoosh storage engine using redis for persistence. A Haystack ``SearchBackend`` subclass is also provided. Normally the ``STORAGE`` key could just be set but Haystack 2.0.0alpha is only aware of ``file`` and ``ram`` backends. This is especially useful for small sites hosted on Heroku, which does not allow writing to local disk. The ``REDISTOGO_URL`` environment variable is read, falling back to the localhost default port. Usage ----- Configure your Haystack connections in ``settings.py``:: import tempfile HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack_redis.RedisEngine', 'PATH': tempfile.gettempdir(), }, } Platform: any Classifier: Environment :: Web Environment Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Software Development :: Libraries :: Python Modules ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621464038.0 haystack-redis-0.0.1/haystack_redis.egg-info/SOURCES.txt0000644000076500000240000000041700000000000022511 0ustar00steinbrostaffLICENSE README.rst haystack_redis.py setup.py haystack_redis.egg-info/PKG-INFO haystack_redis.egg-info/SOURCES.txt haystack_redis.egg-info/dependency_links.txt haystack_redis.egg-info/not-zip-safe haystack_redis.egg-info/requires.txt haystack_redis.egg-info/top_level.txt././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621464038.0 haystack-redis-0.0.1/haystack_redis.egg-info/dependency_links.txt0000644000076500000240000000000100000000000024671 0ustar00steinbrostaff ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621463932.0 haystack-redis-0.0.1/haystack_redis.egg-info/not-zip-safe0000644000076500000240000000000100000000000023051 0ustar00steinbrostaff ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621464038.0 haystack-redis-0.0.1/haystack_redis.egg-info/requires.txt0000644000076500000240000000003500000000000023221 0ustar00steinbrostaffdjango-haystack redis whoosh ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621464038.0 haystack-redis-0.0.1/haystack_redis.egg-info/top_level.txt0000644000076500000240000000001700000000000023353 0ustar00steinbrostaffhaystack_redis ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621463817.0 haystack-redis-0.0.1/haystack_redis.py0000644000076500000240000000735400000000000017514 0ustar00steinbrostaffimport os from io import BytesIO import tempfile from redis import from_url as redis from whoosh.index import _DEF_INDEX_NAME, EmptyIndexError from whoosh.qparser import QueryParser from whoosh.filedb.structfile import StructFile from whoosh.filedb.filestore import Storage, FileStorage from whoosh.util import random_name from whoosh.util.filelock import FileLock from haystack.backends.whoosh_backend import WhooshSearchBackend, WhooshEngine class RedisSearchBackend(WhooshSearchBackend): def setup(self): """ Defers loading until needed. """ from haystack import connections self.storage = RedisStorage(self.path) self.content_field_name, self.schema = self.build_schema(connections[self.connection_alias].get_unified_index().all_searchfields()) self.parser = QueryParser(self.content_field_name, schema=self.schema) try: self.index = self.storage.open_index(schema=self.schema) except EmptyIndexError: self.index = self.storage.create_index(self.schema) self.setup_complete = True class RedisEngine(WhooshEngine): backend = RedisSearchBackend class RedisStorage(Storage): """Storage object that keeps the index in redis. """ supports_mmap = False def __file(self, name): return self.redis.hget("RedisStore:%s" % self.folder, name) def __init__(self, redis_url, namespace='whoosh'): self.folder = namespace self.redis = redis(redis_url) self.locks = {} def file_modified(self, name): return -1 def list(self): return [x.decode('utf-8') for x in self.redis.hkeys("RedisStore:%s" % self.folder)] def clean(self): self.redis.delete("RedisStore:%s" % self.folder) def total_size(self): return sum(self.file_length(f) for f in self.list()) def file_exists(self, name): return self.redis.hexists("RedisStore:%s" % self.folder, name) def file_length(self, name): if not self.file_exists(name): raise NameError return len(self.__file(name)) def delete_file(self, name): if not self.file_exists(name): raise NameError self.redis.hdel("RedisStore:%s" % self.folder, name) def rename_file(self, name, newname, safe=False): if not self.file_exists(name): raise NameError("File %r does not exist" % name) if safe and self.file_exists(newname): raise NameError("File %r exists" % newname) content = self.__file(name) pl = self.redis.pipeline() pl.hdel("RedisStore:%s" % self.folder, name) pl.hset("RedisStore:%s" % self.folder, newname, content) pl.execute() def create_file(self, name, **kwargs): def onclose_fn(sfile): self.redis.hset("RedisStore:%s" % self.folder, name, sfile.file.getvalue()) f = StructFile(BytesIO(), name=name, onclose=onclose_fn) return f def open_file(self, name, *args, **kwargs): if not self.file_exists(name): raise NameError("No such file %r" % name) def onclose_fn(sfile): self.redis.hset("RedisStore:%s" % self.folder, name, sfile.file.getvalue()) #print "Opened file %s %s " % (name, self.__file(name)) return StructFile(BytesIO(self.__file(name)), name=name, onclose=onclose_fn, *args, **kwargs) def lock(self, name): tdir = tempfile.gettempdir() name = "%s.lock" % name path = os.path.join(tdir, name) return FileLock(path) def temp_storage(self, name=None): tdir = tempfile.gettempdir() name = name or "%s.tmp" % random_name() path = os.path.join(tdir, name) tempstore = FileStorage(path) return tempstore.create() ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1621464038.2326689 haystack-redis-0.0.1/setup.cfg0000644000076500000240000000004600000000000015755 0ustar00steinbrostaff[egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1621463977.0 haystack-redis-0.0.1/setup.py0000644000076500000240000000331300000000000015646 0ustar00steinbrostaff# -*- coding: utf-8 -*- """ haystack-redis ~~~~~~~~~~~~~~~~~~~~~~ A Whoosh storage engine using redis for persistence. A Haystack ``SearchBackend`` subclass is also provided. Normally the ``STORAGE`` key could just be set but Haystack 2.0.0alpha is only aware of ``file`` and ``ram`` backends. This is especially useful for small sites hosted on Heroku, which does not allow writing to local disk. The ``REDISTOGO_URL`` environment variable is read, falling back to the localhost default port. Usage ----- Configure your Haystack connections in ``settings.py``:: import tempfile HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack_redis.RedisEngine', 'PATH': tempfile.gettempdir(), }, } """ from setuptools import setup setup( name='haystack-redis', version='0.0.1', url='https://github.com/steinbro/haystack-redis', license='BSD', author=u'Jökull Sólberg Auðunsson', author_email='jokull@solberg.is', maintainer='Daniel W. Steinbrook', maintainer_email='steinbro@post.harvard.edu', description='Use redis as a persistence layer for Whoosh and Haystack', long_description=__doc__, py_modules=['haystack_redis'], zip_safe=False, include_package_data=True, platforms='any', install_requires=['django-haystack', 'redis', 'whoosh'], classifiers=[ 'Environment :: Web Environment', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Topic :: Internet :: WWW/HTTP :: Dynamic Content', 'Topic :: Software Development :: Libraries :: Python Modules' ] )