pax_global_header00006660000000000000000000000064140000020170014472gustar00rootroot0000000000000052 comment=70bc59493906093dfb2309120f1e07870cd465d6 afew-3.0.1/000077500000000000000000000000001400000201700124155ustar00rootroot00000000000000afew-3.0.1/.github/000077500000000000000000000000001400000201700137555ustar00rootroot00000000000000afew-3.0.1/.github/workflows/000077500000000000000000000000001400000201700160125ustar00rootroot00000000000000afew-3.0.1/.github/workflows/build.yml000066400000000000000000000031571400000201700176420ustar00rootroot00000000000000name: CI # Trigger the workflow on push or pull request on: [push, pull_request] jobs: build-ubuntu: strategy: matrix: python: [3.6, 3.7, 3.8] name: Build (Python ${{ matrix.python }}) runs-on: ubuntu-18.04 steps: - uses: actions/checkout@v2 - name: Set up Python uses: actions/setup-python@v1 with: python-version: '${{ matrix.python }}' - name: Install dependencies shell: bash run: | sudo apt-get update sudo apt-get install -y notmuch python3-notmuch python3-venv python3 -m venv env source ./env/bin/activate pip install setuptools pytest dkimpy ln -s /usr/lib/python3/dist-packages/notmuch ./env/lib/python*/site-packages - name: Tests run: | source ./env/bin/activate pip install freezegun pytest - name: build run: | source ./env/bin/activate python setup.py build - name: install run: | source ./env/bin/activate python setup.py install - name: Generate coverage report run: | source ./env/bin/activate pip install pytest-cov pytest --cov=./ --cov-report=xml - name: Upload coverage to Codecov uses: codecov/codecov-action@v1 with: file: ./coverage.xml flags: unittests name: codecov-umbrella fail_ci_if_error: true - name: Docs run: | source ./env/bin/activate pip install sphinx python setup.py build_sphinx -b html,man afew-3.0.1/.gitignore000066400000000000000000000001171400000201700144040ustar00rootroot00000000000000*.py[co] /dist /build bin/ include/ lib/ /afew.egg-info afew/version.py /.eggs afew-3.0.1/LICENSE000066400000000000000000000013461400000201700134260ustar00rootroot00000000000000ISC License Copyright (c) afewmail project Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. afew-3.0.1/NEWS.md000066400000000000000000000103711400000201700135150ustar00rootroot00000000000000afew 3.0.0 (2020-03-10) ======================= MailMover: many fixes Previously, MailMover didn't properly preserve flags when renaming files, and moved all mail to `cur`. This was fixed. Also, MailMover gained a test suite. New filters: PropagateTags[ByRegex]InThreadFilter These filters allow propagating tags set to a message to the whole thread. New command line argument: --notmuch-args= in move mode In move mode, afew calls `notmuch new` after moving mails around. This prevents `afew -m` from being used in a pre-new hook in `notmuch`. Now it's possible to specify notmuch args, so something like `afew -m --notmuch-args=--no-hooks` can live happily in a pre-new hook. Python 3.4 and 3.5 support dropped afew stopped supporting the older python versions 3.4 and 3.5, and removed some more Python 2 compatibility code. (`from __future__ import …`, utf-8 headers, relative imports, …) afew 2.0.0 (2019-06-16) ======================= Python 2 support removed afew doesn't support Python 2 anymore, and all Python 2 specific compat hacks were removed. Better support for whitespaces and quotes in folder names Previously, afew failed with folders containing quotes or namespaces. These are now properly escaped internally. Support `MAILDIR` as fallback for database location In addition to reading notmuch databse location from notmuch config, afew now supports reading from the `MAILDIR` environment variable, like notmuch CLI does, too. Support relative path for database location As of notmuch 0.28, a relative path may be provided for the database location. notmuch prepends `$HOME/` to the relative path. For feature parity, afew now supports the same methodology of prepending `$HOME/` if a relative path is provided. Support for removing unread and read tags in filters In a filter rule, it was possible to add "unread" and "read" tags but not to remove them. afew 1.3.0 (2018-02-06) ======================= MeFilter added Add filter tagging mail sent directly to any of addresses defined in Notmuch config file: `primary_email` or `other_email`. Default tag is `to-me` and can be customized with `me_tag` option. License comments replaced with SPDX-License-Identifier Where possible, license boilerplate comments were changed to just the SPDX-License-Identifier, while adding the license to the repo and referencing it in `setup.py`, too. DMARCReportInspectionFilter added DMARC reports usually come in ZIP files. To check the report you have to unpack and search thru XML document which is very tedious. The filter tags the message as follows: if there's any SPF failure in any attachment, tag the message with "dmarc-spf-fail" tag, otherwise tag with "dmarc-spf-ok" if there's any DKIM failure in any attachment, tag the message with "dmarc-dkim-fail" tag, otherwise tag with "dmarc-dkim-ok" DKIMValidityFilter added This filter verifies DKIM signatures of E-Mails with DKIM header, and adds `dkin-ok` or `dkin-fail` tags. afew 1.2.0 (2017-08-07) ======================= FolderNameFilter supporting mails in multiple directories FolderNameFilter now looks at all folders that a message is in when adding tags to it. afew 1.1.0 (2017-06-12) ======================= Classification system removed As of commit 86d881d948c6ff00a6475dee97551ea092e526a1, the classification system (--learn) was removed, as it was really broken. If someone wants to implement it properly in the future it would be much simpler to start from scratch. afew 1.0.0 (2017-02-13) ===================== Filter behaviour change As of commit d98a0cd0d1f37ee64d03be75e75556cff9f32c29, the ListMailsFilter does not add a tag named `list-id `anymore, but a new one called `lists/`. afew 0.1pre (2012-02-10) ======================== Configuration format change Previously the values for configuration entries with the key `tags` were interpreted as a whitespace delimited list of strings. As of commit e4ec3ced16cc90c3e9c738630bf0151699c4c087 those entries are split at semicolons (';'). This changes the semantic of the configuration file and affects everyone who uses filter rules that set or remove more than one tag at once. Please inspect your configuration files and adjust them if necessary. afew-3.0.1/PKG-INFO000066400000000000000000000012011400000201700135040ustar00rootroot00000000000000Metadata-Version: 1.1 Name: afew Version: 3.0.1 Summary: An initial tagging script for notmuch mail Home-page: https://github.com/afewmail/afew Author: UNKNOWN Author-email: UNKNOWN License: ISC Description: UNKNOWN Platform: UNKNOWN Classifier: License :: OSI Approved :: ISC License (ISCL) Classifier: Development Status :: 4 - Beta Classifier: Environment :: Console Classifier: Intended Audience :: End Users/Desktop Classifier: Programming Language :: Python Classifier: Topic :: Communications :: Email Classifier: Topic :: Communications :: Email :: Filters Classifier: Topic :: Utilities Classifier: Topic :: Database Provides: afew afew-3.0.1/README.rst000066400000000000000000000041501400000201700141040ustar00rootroot00000000000000==== afew ==== |GithubTag| |CodeCov| |CI Status| About ----- afew is an initial tagging script for notmuch mail: * http://notmuchmail.org/ * http://notmuchmail.org/initial_tagging/ Its basic task is to provide automatic tagging each time new mail is registered with notmuch. In a classic setup, you might call it after 'notmuch new' in an offlineimap post sync hook. It can do basic thing such as adding tags based on email headers or maildir folders, handling killed threads and spam. In move mode, afew will move mails between maildir folders according to configurable rules that can contain arbitrary notmuch queries to match against any searchable attributes. fyi: afew plays nicely with alot, a GUI for notmuch mail ;) * https://github.com/pazz/alot IRC --- Feel free to ask your questions and discuss usage in the `#afewmail IRC Channel`_ on freenode. .. _#afewmail IRC Channel: http://webchat.freenode.net/?channels=#afewmail Features -------- * spam handling (flush all tags, add spam) * killed thread handling * automatic propagation of tags to whole thread * tags posts to lists with ``lists``, ``$list-id`` * autoarchives mails sent from you * catchall -> remove ``new``, add ``inbox`` * can operate on new messages [default], ``--all`` messages or on custom query results * can move mails based on arbitrary notmuch queries, so your sorting may show on your traditional mail client (well, almost ;)) * has a ``--dry-run`` mode for safe testing * works with python 3.6+ Installation and Usage ---------------------- Full documentation is available in the `docs/`_ directory and in rendered form at afew.readthedocs.io_. .. _afew.readthedocs.io: https://afew.readthedocs.io/en/latest/ .. _docs/: docs/ Have fun :) .. |GithubTag| image:: https://img.shields.io/github/tag/afewmail/afew.svg :target: https://github.com/afewmail/afew/releases .. |CodeCov| image:: https://codecov.io/gh/afewmail/afew/branch/master/graph/badge.svg :target: https://codecov.io/gh/afewmail/afew .. |CI Status| image:: https://github.com/afewmail/afew/workflows/CI/badge.svg :target: https://github.com/afewmail/afew/actions afew-3.0.1/afew.egg-info/000077500000000000000000000000001400000201700150315ustar00rootroot00000000000000afew-3.0.1/afew.egg-info/PKG-INFO000066400000000000000000000012011400000201700161200ustar00rootroot00000000000000Metadata-Version: 1.1 Name: afew Version: 3.0.1 Summary: An initial tagging script for notmuch mail Home-page: https://github.com/afewmail/afew Author: UNKNOWN Author-email: UNKNOWN License: ISC Description: UNKNOWN Platform: UNKNOWN Classifier: License :: OSI Approved :: ISC License (ISCL) Classifier: Development Status :: 4 - Beta Classifier: Environment :: Console Classifier: Intended Audience :: End Users/Desktop Classifier: Programming Language :: Python Classifier: Topic :: Communications :: Email Classifier: Topic :: Communications :: Email :: Filters Classifier: Topic :: Utilities Classifier: Topic :: Database Provides: afew afew-3.0.1/afew.egg-info/SOURCES.txt000066400000000000000000000025261400000201700167220ustar00rootroot00000000000000.gitignore LICENSE NEWS.md README.rst setup.py .github/workflows/build.yml afew/Database.py afew/FilterRegistry.py afew/MailMover.py afew/NotmuchSettings.py afew/Settings.py afew/__init__.py afew/__main__.py afew/commands.py afew/configparser.py afew/files.py afew/main.py afew/utils.py afew/version.py afew.egg-info/PKG-INFO afew.egg-info/SOURCES.txt afew.egg-info/dependency_links.txt afew.egg-info/entry_points.txt afew.egg-info/requires.txt afew.egg-info/top_level.txt afew/defaults/afew.config afew/filters/ArchiveSentMailsFilter.py afew/filters/BaseFilter.py afew/filters/DKIMValidityFilter.py afew/filters/DMARCReportInspectionFilter.py afew/filters/FolderNameFilter.py afew/filters/HeaderMatchingFilter.py afew/filters/InboxFilter.py afew/filters/KillThreadsFilter.py afew/filters/ListMailsFilter.py afew/filters/MeFilter.py afew/filters/PropagateTagsByRegexInThreadFilter.py afew/filters/PropagateTagsInThreadFilter.py afew/filters/SentMailsFilter.py afew/filters/SpamFilter.py afew/filters/__init__.py afew/tests/__init__.py afew/tests/test_dkimvalidityfilter.py afew/tests/test_mailmover.py afew/tests/test_settings.py afew/tests/test_utils.py docs/commandline.rst docs/conf.py docs/configuration.rst docs/extending.rst docs/filters.rst docs/implementation.rst docs/index.rst docs/installation.rst docs/move_mode.rst docs/quickstart.rst docs/_static/.keepafew-3.0.1/afew.egg-info/dependency_links.txt000066400000000000000000000000011400000201700210770ustar00rootroot00000000000000 afew-3.0.1/afew.egg-info/entry_points.txt000066400000000000000000000020361400000201700203300ustar00rootroot00000000000000[afew.filter] ArchiveSentMailsFilter = afew.filters.ArchiveSentMailsFilter:ArchiveSentMailsFilter DKIMValidityFilter = afew.filters.DKIMValidityFilter:DKIMValidityFilter DMARCReportInspectionFilter = afew.filters.DMARCReportInspectionFilter:DMARCReportInspectionFilter Filter = afew.filters.BaseFilter:Filter FolderNameFilter = afew.filters.FolderNameFilter:FolderNameFilter HeaderMatchingFilter = afew.filters.HeaderMatchingFilter:HeaderMatchingFilter InboxFilter = afew.filters.InboxFilter:InboxFilter KillThreadsFilter = afew.filters.KillThreadsFilter:KillThreadsFilter ListMailsFilter = afew.filters.ListMailsFilter:ListMailsFilter MeFilter = afew.filters.MeFilter:MeFilter PropagateTagsByRegexInThreadFilter = afew.filters.PropagateTagsByRegexInThreadFilter:PropagateTagsByRegexInThreadFilter PropagateTagsInThreadFilter = afew.filters.PropagateTagsInThreadFilter:PropagateTagsInThreadFilter SentMailsFilter = afew.filters.SentMailsFilter:SentMailsFilter SpamFilter = afew.filters.SpamFilter:SpamFilter [console_scripts] afew = afew.commands:main afew-3.0.1/afew.egg-info/requires.txt000066400000000000000000000000271400000201700174300ustar00rootroot00000000000000notmuch chardet dkimpy afew-3.0.1/afew.egg-info/top_level.txt000066400000000000000000000000051400000201700175560ustar00rootroot00000000000000afew afew-3.0.1/afew/000077500000000000000000000000001400000201700133375ustar00rootroot00000000000000afew-3.0.1/afew/Database.py000066400000000000000000000144371400000201700154260ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import os import time import logging import notmuch from afew.NotmuchSettings import notmuch_settings, get_notmuch_new_tags class Database: """ Convenience wrapper around `notmuch`. """ def __init__(self): self.db_path = self._calculate_db_path() self.handle = None def _calculate_db_path(self): """ Calculates the path to use for the database. Supports notmuch's methodology including falling back to $MAILDIR or $HOME/mail if a path is not specified and using $HOME/ if path is relative. """ default_path = os.environ.get('MAILDIR', '{}/mail'.format(os.environ.get('HOME'))) db_path = notmuch_settings.get('database', 'path', fallback=default_path) # If path is relative, notmuch prepends $HOME in front if not os.path.isabs(db_path): db_path = '{}/{}'.format(os.environ.get('HOME'), db_path) return db_path def __enter__(self): """ Implements the context manager protocol. """ return self def __exit__(self, exc_type, exc_value, traceback): """ Implements the context manager protocol. """ self.close() def open(self, rw=False, retry_for=180, retry_delay=1, create=False): if rw: if self.handle and self.handle.mode == notmuch.Database.MODE.READ_WRITE: return self.handle start_time = time.time() while True: try: self.handle = notmuch.Database(self.db_path, mode=notmuch.Database.MODE.READ_WRITE, create=create) break except notmuch.NotmuchError: time_left = int(retry_for - (time.time() - start_time)) if time_left <= 0: raise if time_left % 15 == 0: logging.debug( 'Opening the database failed. Will keep trying for another {} seconds'.format(time_left)) time.sleep(retry_delay) else: if not self.handle: self.handle = notmuch.Database(self.db_path, create=create) return self.handle def close(self): """ Closes the notmuch database if it has been opened. """ if self.handle: self.handle.close() self.handle = None def do_query(self, query): """ Executes a notmuch query. :param query: the query to execute :type query: str :returns: the query result :rtype: :class:`notmuch.Query` """ logging.debug('Executing query %r' % query) return notmuch.Query(self.open(), query) def get_messages(self, query, full_thread=False): """ Get all messages mathing the given query. :param query: the query to execute using :func:`Database.do_query` :type query: str :param full_thread: return all messages from mathing threads :type full_thread: bool :returns: an iterator over :class:`notmuch.Message` objects """ if not full_thread: for message in self.do_query(query).search_messages(): yield message else: for thread in self.do_query(query).search_threads(): for message in self.walk_thread(thread): yield message def walk_replies(self, message): """ Returns all replies to the given message. :param message: the message to start from :type message: :class:`notmuch.Message` :returns: an iterator over :class:`notmuch.Message` objects """ yield message # TODO: bindings are *very* unpythonic here... iterator *or* None # is a nono replies = message.get_replies() if replies != None: for message in replies: # TODO: yield from for message in self.walk_replies(message): yield message def walk_thread(self, thread): """ Returns all messages in the given thread. :param thread: the tread you are interested in :type thread: :class:`notmuch.Thread` :returns: an iterator over :class:`notmuch.Message` objects """ for message in thread.get_toplevel_messages(): # TODO: yield from for message in self.walk_replies(message): yield message def add_message(self, path, sync_maildir_flags=False, new_mail_handler=None): """ Adds the given message to the notmuch index. :param path: path to the message :type path: str :param sync_maildir_flags: if `True` notmuch converts the standard maildir flags to tags :type sync_maildir_flags: bool :param new_mail_handler: callback for new messages :type new_mail_handler: a function that is called with a :class:`notmuch.Message` object as its only argument :raises: :class:`notmuch.NotmuchError` if adding the message fails :returns: a :class:`notmuch.Message` object """ # TODO: it would be nice to update notmuchs directory index here handle = self.open(rw=True) if hasattr(notmuch.Database, 'index_file'): message, status = handle.index_file(path, sync_maildir_flags=sync_maildir_flags) else: message, status = handle.add_message(path, sync_maildir_flags=sync_maildir_flags) if status != notmuch.STATUS.DUPLICATE_MESSAGE_ID: logging.info('Found new mail in {}'.format(path)) for tag in get_notmuch_new_tags(): message.add_tag(tag) if new_mail_handler: new_mail_handler(message) return message def remove_message(self, path): """ Remove the given message from the notmuch index. :param path: path to the message :type path: str """ self.open(rw=True).remove_message(path) afew-3.0.1/afew/FilterRegistry.py000066400000000000000000000026531400000201700166750ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import pkg_resources RAISEIT = object() class FilterRegistry: """ The FilterRegistry is responsible for returning filters by key. Filters get registered via entry points. To avoid any circular dependencies, the registry loads the Filters lazily """ def __init__(self, filters): self._filteriterator = filters @property def filter(self): if not hasattr(self, '_filter'): self._filter = {} for f in self._filteriterator: self._filter[f.name] = f.load() return self._filter def get(self, key, default=RAISEIT): if default == RAISEIT: return self.filter[key] else: return self.filter.get(key, default) def __getitem__(self, key): return self.get(key) def __setitem__(self, key, value): self.filter[key] = value def __delitem__(self, key): del self.filter[key] def keys(self): return self.filter.keys() def values(self): return self.filter.values() def items(self): return self.filter.items() all_filters = FilterRegistry(pkg_resources.iter_entry_points('afew.filter')) def register_filter(klass): '''Decorator function for registering a class as a filter.''' all_filters[klass.__name__] = klass return klass afew-3.0.1/afew/MailMover.py000066400000000000000000000121561400000201700156110ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) dtk import logging import os import shutil import uuid from datetime import date, datetime, timedelta from subprocess import check_call, CalledProcessError, DEVNULL import notmuch from afew.Database import Database from afew.utils import get_message_summary class MailMover(Database): """ Move mail files matching a given notmuch query into a target maildir folder. """ def __init__(self, max_age=0, rename=False, dry_run=False, notmuch_args='', quiet=False): super().__init__() self.db = notmuch.Database(self.db_path) self.query = 'folder:"{folder}" AND {subquery}' if max_age: days = timedelta(int(max_age)) start = date.today() - days now = datetime.now() self.query += ' AND {start}..{now}'.format(start=start.strftime('%s'), now=now.strftime('%s')) self.dry_run = dry_run self.rename = rename self.notmuch_args = notmuch_args self.quiet = quiet def get_new_name(self, fname, destination): basename = os.path.basename(fname) submaildir = os.path.split(os.path.split(fname)[0])[1] if self.rename: parts = basename.split(':') if len(parts) > 1: flagpart = ':' + parts[-1] else: flagpart = '' # construct a new filename, composed of a made-up ID and the flags part # of the original filename. basename = str(uuid.uuid1()) + flagpart return os.path.join(destination, submaildir, basename) def move(self, maildir, rules): """ Move mails in folder maildir according to the given rules. """ # identify and move messages logging.info("checking mails in '{}'".format(maildir)) to_delete_fnames = [] moved = False for query in rules.keys(): destination = '{}/{}/'.format(self.db_path, rules[query]) main_query = self.query.format( folder=maildir.replace("\"", "\\\""), subquery=query) logging.debug("query: {}".format(main_query)) messages = notmuch.Query(self.db, main_query).search_messages() for message in messages: # a single message (identified by Message-ID) can be in several # places; only touch the one(s) that exists in this maildir all_message_fnames = message.get_filenames() to_move_fnames = [name for name in all_message_fnames if maildir in name] if not to_move_fnames: continue moved = True self.__log_move_action(message, maildir, rules[query], self.dry_run) for fname in to_move_fnames: if self.dry_run: continue try: shutil.copy2(fname, self.get_new_name(fname, destination)) to_delete_fnames.append(fname) except shutil.SameFileError: logging.warn("trying to move '{}' onto itself".format(fname)) continue except shutil.Error as e: # this is ugly, but shutil does not provide more # finely individuated errors if str(e).endswith("already exists"): continue else: raise # remove mail from source locations only after all copies are finished for fname in set(to_delete_fnames): os.remove(fname) # update notmuch database if not self.dry_run: if moved: logging.info("updating database") self.__update_db(maildir) else: logging.info("Would update database") def __update_db(self, maildir): """ Update the database after mail files have been moved in the filesystem. """ try: if self.quiet: check_call(['notmuch', 'new'] + self.notmuch_args.split(), stdout=DEVNULL, stderr=DEVNULL) else: check_call(['notmuch', 'new'] + self.notmuch_args.split()) except CalledProcessError as err: logging.error("Could not update notmuch database " "after syncing maildir '{}': {}".format(maildir, err)) raise SystemExit def __log_move_action(self, message, source, destination, dry_run): ''' Report which mails have been identified for moving. ''' if not dry_run: level = logging.DEBUG prefix = 'moving mail' else: level = logging.INFO prefix = 'I would move mail' logging.log(level, prefix) logging.log(level, " {}".format(get_message_summary(message).encode('utf8'))) logging.log(level, "from '{}' to '{}'".format(source, destination)) afew-3.0.1/afew/NotmuchSettings.py000066400000000000000000000015601400000201700170510ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import os from afew.configparser import RawConfigParser notmuch_settings = RawConfigParser() def read_notmuch_settings(path=None): if path == None: path = os.environ.get('NOTMUCH_CONFIG', os.path.expanduser('~/.notmuch-config')) with open(path) as fp: notmuch_settings.read_file(fp) def write_notmuch_settings(path = None): if path == None: path = os.environ.get('NOTMUCH_CONFIG', os.path.expanduser('~/.notmuch-config')) with open(path, 'w+') as fp: notmuch_settings.write(fp) def get_notmuch_new_tags(): # see issue 158 return filter(lambda x: x != 'unread', notmuch_settings.get_list('new', 'tags')) def get_notmuch_new_query(): return '(%s)' % ' AND '.join('tag:%s' % tag for tag in get_notmuch_new_tags()) afew-3.0.1/afew/Settings.py000066400000000000000000000070461400000201700155200ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import os import re import collections import shlex from afew.configparser import ConfigParser from afew.FilterRegistry import all_filters user_config_dir = os.path.join(os.environ.get('XDG_CONFIG_HOME', os.path.expanduser('~/.config')), 'afew') user_config_dir = os.path.expandvars(user_config_dir) settings = ConfigParser() # preserve the capitalization of the keys. settings.optionxform = str settings.readfp(open(os.path.join(os.path.dirname(__file__), 'defaults', 'afew.config'))) settings.read(os.path.join(user_config_dir, 'config')) # All the values for keys listed here are interpreted as ;-delimited lists value_is_a_list = ['tags', 'tags_blacklist'] mail_mover_section = 'MailMover' section_re = re.compile(r'^(?P[a-z_][a-z0-9_]*)(\((?P[a-z_][a-z0-9_]*)\)|\.(?P\d+))?$', re.I) def get_filter_chain(database): filter_chain = [] for section in settings.sections(): if section == 'global' or section == mail_mover_section: continue match = section_re.match(section) if not match: raise SyntaxError('Malformed section title %r.' % section) kwargs = dict( (key, settings.get(section, key)) if key not in value_is_a_list else (key, settings.get_list(section, key)) for key in settings.options(section) ) if match.group('parent_class'): try: parent_class = all_filters[match.group('parent_class')] except KeyError: raise NameError( 'Parent class %r not found in filter type definition %r.' % (match.group('parent_class'), section)) new_type = type(match.group('name'), (parent_class,), kwargs) all_filters[match.group('name')] = new_type else: try: klass = all_filters[match.group('name')] except KeyError: raise NameError('Filter type %r not found.' % match.group('name')) filter_chain.append(klass(database, **kwargs)) return filter_chain def get_mail_move_rules(): rule_pattern = re.compile(r"'(.+?)':((?P['\"])(.*?)(?P=quote)|\S+)") if settings.has_option(mail_mover_section, 'folders'): all_rules = collections.OrderedDict() for folder in shlex.split(settings.get(mail_mover_section, 'folders')): if settings.has_option(mail_mover_section, folder): rules = collections.OrderedDict() raw_rules = re.findall(rule_pattern, settings.get(mail_mover_section, folder)) for rule in raw_rules: query = rule[0] destination = rule[3] or rule[1] rules[query] = destination all_rules[folder] = rules else: raise NameError("No rules specified for maildir '{}'.".format(folder)) return all_rules else: raise NameError("No folders defined to move mails from.") def get_mail_move_age(): max_age = 0 if settings.has_option(mail_mover_section, 'max_age'): max_age = settings.get(mail_mover_section, 'max_age') return max_age def get_mail_move_rename(): rename = False if settings.has_option(mail_mover_section, 'rename'): rename = settings.get(mail_mover_section, 'rename').lower() == 'true' return rename afew-3.0.1/afew/__init__.py000066400000000000000000000001411400000201700154440ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> afew-3.0.1/afew/__main__.py000066400000000000000000000001461400000201700154320ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Lucas Hoffmann from afew.commands import main main() afew-3.0.1/afew/commands.py000066400000000000000000000121151400000201700155120ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import glob import sys import logging import argparse from afew.Database import Database from afew.main import main as inner_main from afew.FilterRegistry import all_filters from afew.Settings import user_config_dir, get_filter_chain, \ get_mail_move_rules, get_mail_move_age, get_mail_move_rename from afew.NotmuchSettings import read_notmuch_settings, get_notmuch_new_query from afew.version import version parser = argparse.ArgumentParser() parser.add_argument('-V', '--version', action='version', version=version) # the actions action_group = parser.add_argument_group( 'Actions', 'Please specify exactly one action.' ) action_group.add_argument( '-t', '--tag', action='store_true', help='run the tag filters' ) action_group.add_argument( '-w', '--watch', action='store_true', help='continuously monitor the mailbox for new files' ) action_group.add_argument( '-m', '--move-mails', action='store_true', help='move mail files between maildir folders' ) # query modifiers query_modifier_group = parser.add_argument_group( 'Query modifiers', 'Please specify either --all or --new or a query string.' ) query_modifier_group.add_argument( '-a', '--all', action='store_true', help='operate on all messages' ) query_modifier_group.add_argument( '-n', '--new', action='store_true', help='operate on all new messages' ) query_modifier_group.add_argument( 'query', nargs='*', help='a notmuch query to find messages to work on' ) # general options options_group = parser.add_argument_group('General options') # TODO: get config via notmuch api options_group.add_argument( '-C', '--notmuch-config', default=None, help='path to the notmuch configuration file [default: $NOTMUCH_CONFIG or' ' ~/.notmuch-config]' ) options_group.add_argument( '-e', '--enable-filters', help="filter classes to use, separated by ',' [default: filters specified" " in afew's config]" ) options_group.add_argument( '-d', '--dry-run', default=False, action='store_true', help="don't change the db [default: %(default)s]" ) options_group.add_argument( '-R', '--reference-set-size', type=int, default=1000, help='size of the reference set [default: %(default)s]' ) options_group.add_argument( '-T', '--reference-set-timeframe', type=int, default=30, metavar='DAYS', help='do not use mails older than DAYS days [default: %(default)s]' ) options_group.add_argument( '-v', '--verbose', dest='verbosity', action='count', default=0, help='be more verbose, can be given multiple times' ) options_group.add_argument( '-N', '--notmuch-args', default='', help='arguments for notmuch new (in move mode)' ) def main(): if sys.version_info < (3, 6): sys.exit("Python 3.6 or later is required.") args = parser.parse_args() no_actions = len(list(filter(None, ( args.tag, args.watch, args.move_mails )))) if no_actions == 0: sys.exit('You need to specify an action') elif no_actions > 1: sys.exit('Please specify exactly one action') no_query_modifiers = len(list(filter(None, (args.all, args.new, args.query)))) if no_query_modifiers == 0 and not args.watch \ and not args.move_mails: sys.exit('You need to specify one of --new, --all or a query string') elif no_query_modifiers > 1: sys.exit('Please specify either --all, --new or a query string') read_notmuch_settings(args.notmuch_config) if args.new: query_string = get_notmuch_new_query() elif args.all: query_string = '' else: query_string = ' '.join(args.query) loglevel = { 0: logging.WARNING, 1: logging.INFO, 2: logging.DEBUG, }[min(2, args.verbosity)] logging.basicConfig(level=loglevel) sys.path.insert(0, user_config_dir) for file_name in glob.glob1(user_config_dir, '*.py'): logging.info('Importing user filter %r' % (file_name,)) __import__(file_name[:-3], level=0) if args.move_mails: args.mail_move_rules = get_mail_move_rules() args.mail_move_age = get_mail_move_age() args.mail_move_rename = get_mail_move_rename() with Database() as database: configured_filter_chain = get_filter_chain(database) if args.enable_filters: args.enable_filters = args.enable_filters.split(',') all_filters_set = set(all_filters.keys()) enabled_filters_set = set(args.enable_filters) if not all_filters_set.issuperset(enabled_filters_set): sys.exit('Unknown filter(s) selected: %s' % (' '.join( enabled_filters_set.difference(all_filters_set)))) args.enable_filters = [all_filters[filter_name](database) for filter_name in args.enable_filters] else: args.enable_filters = configured_filter_chain inner_main(args, database, query_string) afew-3.0.1/afew/configparser.py000066400000000000000000000012031400000201700163670ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import configparser class GetListMixIn: def get_list(self, section, key, delimiter=';', filter_=lambda value: value.strip(), include_falsish=False): result = (filter_(value) for value in self.get(section, key).split(delimiter)) if include_falsish: return result else: return filter(None, result) class ConfigParser(configparser.ConfigParser, GetListMixIn): pass class RawConfigParser(configparser.RawConfigParser, GetListMixIn): pass afew-3.0.1/afew/defaults/000077500000000000000000000000001400000201700151465ustar00rootroot00000000000000afew-3.0.1/afew/defaults/afew.config000066400000000000000000000017021400000201700172570ustar00rootroot00000000000000# global configuration [global] #[MailMover] #folders = INBOX Junk #max_age = 15 ##rules #INBOX = 'tag:spam':Junk 'NOT tag:inbox':Archive #Junk = 'NOT tag:spam and tag:inbox':INBOX 'NOT tag:spam':Archive # This is the default filter chain #[SpamFilter] #[KillThreadsFilter] #[ListMailsFilter] #[ArchiveSentMailsFilter] #[InboxFilter] # Let's say you like the SpamFilter, but it is way too polite # 1. create an filter object and customize it #[SpamFilter.0] # note the index #message = meh # 2. create a new type and... #[ShitFilter(SpamFilter)] #message = I hatez teh spam! # create an object or two... #[ShitFilter.0] #[ShitFilter.1] #message = Me hatez it too. # 3. drop a custom filter type in ~/.config/afew/ #[MyCustomFilter] # To create a custom generic filter, define it inline with # your above filter chain. E.g.: # ... # [ListMailsFilter] # # [Filter.1] # query = from:boss@office.com # tags = +office # # [ArchiveSentMailsFilter] # ... afew-3.0.1/afew/files.py000066400000000000000000000130061400000201700150130ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import os import re import stat import logging import platform import queue import threading if platform.system() != 'Linux': raise ImportError('Unsupported platform: {!r}'.format(platform.system())) import notmuch import pyinotify class EventHandler(pyinotify.ProcessEvent): def __init__(self, options, database): self.options = options self.database = database super().__init__() ignore_re = re.compile('(/xapian/.*(base.|tmp)$)|(\.lock$)|(/dovecot)') def process_IN_DELETE(self, event): if self.ignore_re.search(event.pathname): return logging.debug("Detected file removal: {!r}".format(event.pathname)) self.database.remove_message(event.pathname) self.database.close() def process_IN_MOVED_TO(self, event): if self.ignore_re.search(event.pathname): return src_pathname = event.src_pathname if hasattr(event, 'src_pathname') else None logging.debug("Detected file rename: {!r} -> {!r}".format(src_pathname, event.pathname)) def new_mail(message): for filter_ in self.options.enable_filters: try: filter_.run('id:"{}"'.format(message.get_message_id())) filter_.commit(self.options.dry_run) except Exception as e: logging.warn('Error processing mail with filter {!r}: {}'.format(filter_.message, e)) try: self.database.add_message(event.pathname, sync_maildir_flags=True, new_mail_handler=new_mail) except notmuch.FileError as e: logging.warn('Error opening mail file: {}'.format(e)) return except notmuch.FileNotEmailError as e: logging.warn('File does not look like an email: {}'.format(e)) return else: if src_pathname: self.database.remove_message(src_pathname) finally: self.database.close() def watch_for_new_files(options, database, paths, daemonize=False): wm = pyinotify.WatchManager() mask = ( pyinotify.IN_DELETE | pyinotify.IN_MOVED_FROM | pyinotify.IN_MOVED_TO) handler = EventHandler(options, database) notifier = pyinotify.Notifier(wm, handler) logging.debug('Registering inotify watch descriptors') wdds = dict() for path in paths: wdds[path] = wm.add_watch(path, mask) # TODO: honor daemonize logging.debug('Running mainloop') notifier.loop() import ctypes import contextlib try: libc = ctypes.CDLL(ctypes.util.find_library("c")) except ImportError as e: raise ImportError('Could not load libc: {}'.format(e)) class Libc: class c_dir(ctypes.Structure): pass c_dir_p = ctypes.POINTER(c_dir) opendir = libc.opendir opendir.argtypes = [ctypes.c_char_p] opendir.restype = c_dir_p closedir = libc.closedir closedir.argtypes = [c_dir_p] closedir.restype = ctypes.c_int @classmethod @contextlib.contextmanager def open_directory(cls, path): handle = cls.opendir(path) yield handle cls.closedir(handle) class c_dirent(ctypes.Structure): ''' man 3 readdir says:: On Linux, the dirent structure is defined as follows: struct dirent { ino_t d_ino; /* inode number */ off_t d_off; /* offset to the next dirent */ unsigned short d_reclen; /* length of this record */ unsigned char d_type; /* type of file; not supported by all file system types */ char d_name[256]; /* filename */ }; ''' _fields_ = ( ('d_ino', ctypes.c_long), ('d_off', ctypes.c_long), ('d_reclen', ctypes.c_ushort), ('d_type', ctypes.c_byte), ('d_name', ctypes.c_char * 4096), ) c_dirent_p = ctypes.POINTER(c_dirent) readdir = libc.readdir readdir.argtypes = [c_dir_p] readdir.restype = c_dirent_p # magic value for directory DT_DIR = 4 blacklist = {'.', '..', 'tmp'} def walk_linux(channel, path): channel.put(path) with Libc.open_directory(path) as handle: while True: dirent_p = Libc.readdir(handle) if not dirent_p: break if dirent_p.contents.d_type == Libc.DT_DIR and \ dirent_p.contents.d_name not in blacklist: walk_linux(channel, os.path.join(path, dirent_p.contents.d_name)) def walk(channel, path): channel.put(path) for child_path in (os.path.join(path, child) for child in os.listdir(path) if child not in blacklist): try: stat_result = os.stat(child_path) except: continue if stat_result.st_mode & stat.S_IFDIR: walk(channel, child_path) def walker(channel, path): walk_linux(channel, path) channel.put(None) def quick_find_dirs_hack(path): results = queue.Queue() walker_thread = threading.Thread(target=walker, args=(results, path)) walker_thread.daemon = True walker_thread.start() while True: result = results.get() if result != None: yield result else: break afew-3.0.1/afew/filters/000077500000000000000000000000001400000201700150075ustar00rootroot00000000000000afew-3.0.1/afew/filters/ArchiveSentMailsFilter.py000066400000000000000000000010711400000201700217270ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> from afew.filters.SentMailsFilter import SentMailsFilter from afew.NotmuchSettings import get_notmuch_new_tags class ArchiveSentMailsFilter(SentMailsFilter): message = 'Archiving all mails sent by myself to others' def __init__(self, database, sent_tag='', to_transforms=''): super().__init__(database, sent_tag) def handle_message(self, message): super().handle_message(message) self.remove_tags(message, *get_notmuch_new_tags()) afew-3.0.1/afew/filters/BaseFilter.py000066400000000000000000000071771400000201700174150ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import collections import logging class Filter: message = 'No message specified for filter' tags = [] tags_blacklist = [] def __init__(self, database, **kwargs): super().__init__() self.log = logging.getLogger('{}.{}'.format( self.__module__, self.__class__.__name__)) self.database = database if 'tags' not in kwargs: kwargs['tags'] = self.tags for key, value in kwargs.items(): setattr(self, key, value) self.flush_changes() self._tags_to_add = [] self._tags_to_remove = [] for tag_action in self.tags: if tag_action[0] not in '+-': raise ValueError('Each tag must be preceded by either + or -') (self._tags_to_add if tag_action[0] == '+' else self._tags_to_remove).append(tag_action[1:]) self._tag_blacklist = set(self.tags_blacklist) def flush_changes(self): ''' (Re)Initializes the data structures that hold the enqueued changes to the notmuch database. ''' self._add_tags = collections.defaultdict(lambda: set()) self._remove_tags = collections.defaultdict(lambda: set()) self._flush_tags = [] def run(self, query): self.log.info(self.message) if getattr(self, 'query', None): if query: query = '(%s) AND (%s)' % (query, self.query) else: query = self.query for message in self.database.get_messages(query): self.handle_message(message) def handle_message(self, message): if not self._tag_blacklist.intersection(message.get_tags()): self.remove_tags(message, *self._tags_to_remove) self.add_tags(message, *self._tags_to_add) def add_tags(self, message, *tags): if tags: self.log.debug('Adding tags %s to id:%s' % (', '.join(tags), message.get_message_id())) self._add_tags[message.get_message_id()].update(tags) def remove_tags(self, message, *tags): if tags: filtered_tags = list(tags) self.log.debug('Removing tags %s from id:%s' % (', '.join(filtered_tags), message.get_message_id())) self._remove_tags[message.get_message_id()].update(filtered_tags) def flush_tags(self, message): self.log.debug('Removing all tags from id:%s' % message.get_message_id()) self._flush_tags.append(message.get_message_id()) def commit(self, dry_run=True): dirty_messages = set() dirty_messages.update(self._flush_tags) dirty_messages.update(self._add_tags.keys()) dirty_messages.update(self._remove_tags.keys()) if not dirty_messages: return if dry_run: self.log.info('I would commit changes to %i messages' % len(dirty_messages)) else: self.log.info('Committing changes to %i messages' % len(dirty_messages)) db = self.database.open(rw=True) for message_id in dirty_messages: message = db.find_message(message_id) if message_id in self._flush_tags: message.remove_all_tags() for tag in self._add_tags.get(message_id, []): message.add_tag(tag) for tag in self._remove_tags.get(message_id, []): message.remove_tag(tag) self.flush_changes() afew-3.0.1/afew/filters/DKIMValidityFilter.py000066400000000000000000000034431400000201700207650ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Amadeusz Zolnowski """ DKIM validator filter. Verifies DKIM signature of an e-mail which has DKIM header. """ import logging import dkim import dns.exception from afew.filters.BaseFilter import Filter class DKIMVerifyError(Exception): """Failed to verify DKIM signature. """ def verify_dkim(path): """ Verify DKIM signature of an e-mail file. :param path: Path to the e-mail file. :returns: Whether DKIM signature is valid or not. """ with open(path, 'rb') as message_file: message_bytes = message_file.read() try: return dkim.verify(message_bytes) except (dns.exception.DNSException, dkim.DKIMException) as exception: raise DKIMVerifyError(str(exception)) from exception class DKIMValidityFilter(Filter): """ Verifies DKIM signature of an e-mail which has DKIM header. """ message = 'Verify DKIM signature' header = 'DKIM-Signature' def __init__(self, database, ok_tag='dkim-ok', fail_tag='dkim-fail'): super().__init__(database) self.dkim_tag = {True: ok_tag, False: fail_tag} self.log = logging.getLogger('{}.{}'.format( self.__module__, self.__class__.__name__)) def handle_message(self, message): if message.get_header(self.header): try: dkim_ok = all(map(verify_dkim, message.get_filenames())) except DKIMVerifyError as verify_error: self.log.warning( "Failed to verify DKIM of '%s': %s " "(marked as 'dkim-fail')", message.get_message_id(), verify_error ) dkim_ok = False self.add_tags(message, self.dkim_tag[dkim_ok]) afew-3.0.1/afew/filters/DMARCReportInspectionFilter.py000066400000000000000000000122501400000201700226050ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Amadeusz Zolnowski """ DMARC report inspection filter. Looks into DMARC report whether all results are successful or any is failing. Add tags 2 of the tags below: - dmarc/dkim-ok - dmarc/dkim-fail - dmarc/spf-ok - dmarc/spf-fail """ import logging import re import tempfile import xml.etree.ElementTree as ET import zipfile from .BaseFilter import Filter class DMARCInspectionError(Exception): """Failed to inspect DMARC report. """ class ReportFilesIterator: """ Iterator over DMARC reports files attached to the e-mail either directly or in ZIP files. Returns content of each document file (as bytes, not as string) which needs to be decoded from charset encoding. """ def __init__(self, message): self.message = message def __iter__(self): for part in self.message.get_message_parts(): if part.get_content_type() == 'application/zip': with tempfile.TemporaryFile(suffix='.zip') as file: file.write(part.get_payload(decode=True)) try: with zipfile.ZipFile(file) as zip_file: for member_file in zip_file.infolist(): if member_file.filename.endswith('.xml'): yield zip_file.read(member_file) except zipfile.BadZipFile as zip_error: raise DMARCInspectionError(str(zip_error)) \ from zip_error elif part.get_content_type() == 'application/xml': yield part.get_payload(decode=True) def and_dict(dict1, dict2): """ Apply logical conjunction between values of dictionaries of the same keys. Keys set must be identical in both dictionaries. Otherwise KeyError exception is raised. :param dict1: Dictionary of bool values. :param dict2: Dictionary of bool values. :returns: A dictionary with the same set of keys but with modified values. """ dict3 = {} for key in dict1.keys(): dict3[key] = dict1[key] & dict2.get(key, False) return dict3 def has_failed(node): """ Check whether status is "failed". To avoid false positives check whether status is one of "pass" or "none". :param node: XML node holding status as text. :returns: Whether the status is reported as "failed". """ return (node.text.strip() not in ['pass', 'none']) def read_auth_results(document): """ Parse DMARC document. Look for results for DKIM and SPF. If there's more than one record, return `True` only and only if all of the records of particular type (DKIM or SPF) are "pass". :returns: Results as a dictionary where keys are: `dkim` and `spf` and values are boolean values. """ try: results = {'dkim': True, 'spf': True} root = ET.fromstring(document) for record in root.findall('record'): auth_results = record.find('auth_results') if auth_results: dkim = auth_results.find('dkim') if dkim: dkim = dkim.find('result') results['dkim'] &= not has_failed(dkim) spf = auth_results.find('spf') if spf: spf = spf.find('result') results['spf'] &= not has_failed(spf) except ET.ParseError as parse_error: raise DMARCInspectionError(str(parse_error)) from parse_error return results class DMARCReportInspectionFilter(Filter): """ Inspect DMARC reports for DKIM and SPF status. """ def __init__(self, # pylint: disable=too-many-arguments database, dkim_ok_tag='dmarc/dkim-ok', dkim_fail_tag='dmarc/dkim-fail', spf_ok_tag='dmarc/spf-ok', spf_fail_tag='dmarc/spf-fail'): super().__init__(database) self.dkim_tag = {True: dkim_ok_tag, False: dkim_fail_tag} self.spf_tag = {True: spf_ok_tag, False: spf_fail_tag} self.dmarc_subject = re.compile(r'^report domain:', flags=re.IGNORECASE) self.log = logging.getLogger('{}.{}'.format( self.__module__, self.__class__.__name__)) def handle_message(self, message): if not self.dmarc_subject.match(message.get_header('Subject')): return auth_results = {'dkim': True, 'spf': True} try: for file_content in ReportFilesIterator(message): document = file_content.decode('UTF-8') auth_results = and_dict(auth_results, read_auth_results(document)) self.add_tags(message, 'dmarc', self.dkim_tag[auth_results['dkim']], self.spf_tag[auth_results['spf']]) except DMARCInspectionError as inspection_error: self.log.error( "Failed to verify DMARC report of '%s': %s (not tagging)", message.get_message_id(), inspection_error ) afew-3.0.1/afew/filters/FolderNameFilter.py000066400000000000000000000061461400000201700205520ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) dtk from afew.filters.BaseFilter import Filter from afew.NotmuchSettings import notmuch_settings import re import shlex class FolderNameFilter(Filter): message = 'Tags all new messages with their folder' def __init__(self, database, folder_blacklist='', folder_transforms='', maildir_separator='.', folder_explicit_list='', folder_lowercases=''): super().__init__(database) self.__filename_pattern = '{mail_root}/(?P.*)/(cur|new)/[^/]+'.format( mail_root=notmuch_settings.get('database', 'path').rstrip('/')) self.__folder_explicit_list = set(shlex.split(folder_explicit_list)) self.__folder_blacklist = set(shlex.split(folder_blacklist)) self.__folder_transforms = self.__parse_transforms(folder_transforms) self.__folder_lowercases = folder_lowercases != '' self.__maildir_separator = maildir_separator def handle_message(self, message): # Find all the dirs in the mail directory that this message # belongs to maildirs = [re.match(self.__filename_pattern, filename) for filename in message.get_filenames()] maildirs = filter(None, maildirs) if maildirs: # Make the folders relative to mail_root and split them. folder_groups = [maildir.group('maildirs').split(self.__maildir_separator) for maildir in maildirs] folders = set([folder for folder_group in folder_groups for folder in folder_group]) self.log.debug('found folders {} for message {!r}'.format( folders, message.get_header('subject'))) # remove blacklisted folders clean_folders = folders - self.__folder_blacklist if self.__folder_explicit_list: # only explicitly listed folders clean_folders &= self.__folder_explicit_list # apply transformations transformed_folders = self.__transform_folders(clean_folders) self.add_tags(message, *transformed_folders) def __transform_folders(self, folders): """ Transforms the given collection of folders according to the transformation rules. """ transformations = set() for folder in folders: if folder in self.__folder_transforms: transformations.add(self.__folder_transforms[folder]) else: transformations.add(folder) if self.__folder_lowercases: rtn = set() for folder in transformations: rtn.add(folder.lower()) return rtn return transformations def __parse_transforms(self, transformation_description): """ Parses the transformation rules specified in the config file. """ transformations = dict() for rule in shlex.split(transformation_description): folder, tag = rule.split(':') transformations[folder] = tag return transformations afew-3.0.1/afew/filters/HeaderMatchingFilter.py000066400000000000000000000023371400000201700213770ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) 2012 Justus Winter <4winter@informatik.uni-hamburg.de> # Copyright (c) 2013 Patrick Gerken # Copyright (c) 2013 Patrick Totzke # Copyright (c) 2014 Lars Kellogg-Stedman from afew.filters.BaseFilter import Filter import re class HeaderMatchingFilter(Filter): message = 'Tagging based on specific header values matching a given RE' header = None pattern = None def __init__(self, database, **kwargs): super().__init__(database, **kwargs) if self.pattern is not None: self.pattern = re.compile(self.pattern, re.I) def handle_message(self, message): if self.header is not None and self.pattern is not None: if not self._tag_blacklist.intersection(message.get_tags()): value = message.get_header(self.header) match = self.pattern.search(value) if match: sub = (lambda tag: tag.format(**match.groupdict()).lower()) self.remove_tags(message, *map(sub, self._tags_to_remove)) self.add_tags(message, *map(sub, self._tags_to_add)) afew-3.0.1/afew/filters/InboxFilter.py000066400000000000000000000013501400000201700176050ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> from afew.filters.BaseFilter import Filter from afew.NotmuchSettings import get_notmuch_new_tags, get_notmuch_new_query class InboxFilter(Filter): message = 'Retags all messages not tagged as junk or killed as inbox' tags = ['+inbox'] tags_blacklist = ['killed', 'spam'] @property def query(self): ''' Need to read the notmuch settings first. Using a property here so that the setting is looked up on demand. ''' return get_notmuch_new_query() def handle_message(self, message): self.remove_tags(message, *get_notmuch_new_tags()) super().handle_message(message) afew-3.0.1/afew/filters/KillThreadsFilter.py000066400000000000000000000007621400000201700207420ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> from afew.filters.BaseFilter import Filter class KillThreadsFilter(Filter): message = 'Looking for messages in killed threads that are not yet killed' query = 'NOT tag:killed' def handle_message(self, message): query = self.database.get_messages('thread:"%s" AND tag:killed' % message.get_thread_id()) if len(list(query)): self.add_tags(message, 'killed') afew-3.0.1/afew/filters/ListMailsFilter.py000066400000000000000000000006321400000201700204310ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> from afew.filters.HeaderMatchingFilter import HeaderMatchingFilter class ListMailsFilter(HeaderMatchingFilter): message = 'Tagging mailing list posts' query = 'NOT tag:lists' pattern = r"<(?P[a-z0-9!#$%&'*+/=?^_`{|}~-]+)\." header = 'List-Id' tags = ['+lists', '+lists/{list_id}'] afew-3.0.1/afew/filters/MeFilter.py000066400000000000000000000020671400000201700170750ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Amadeusz Zolnowski import re from afew.filters.BaseFilter import Filter from afew.NotmuchSettings import notmuch_settings class MeFilter(Filter): message = 'Tagging all mails sent directly to myself' _bare_email_re = re.compile(r"[^<]*<(?P[^@<>]+@[^@<>]+)>") def __init__(self, database, me_tag='to-me', tags_blacklist=[]): super().__init__(database, tags_blacklist=tags_blacklist) my_addresses = set() my_addresses.add(notmuch_settings.get('user', 'primary_email')) if notmuch_settings.has_option('user', 'other_email'): for other_email in notmuch_settings.get_list('user', 'other_email'): my_addresses.add(other_email) self.query = ' OR '.join('to:"%s"' % address for address in my_addresses) self.me_tag = me_tag def handle_message(self, message): if not self._tag_blacklist.intersection(message.get_tags()): self.add_tags(message, self.me_tag) afew-3.0.1/afew/filters/PropagateTagsByRegexInThreadFilter.py000066400000000000000000000054701400000201700242030ustar00rootroot00000000000000# Copyright (c) Jens Neuhalfen import re from itertools import chain from afew.filters.BaseFilter import Filter def _flatten(listOfLists): "Flatten one level of nesting" return chain.from_iterable(listOfLists) class PropagateTagsByRegexInThreadFilter(Filter): """ This filter enables a very easy workflow where entire threads can be tagged automatically. Assuming the following workflow: all messages for projects or releases should be tagged as "project/A", "project/B" respectively "release/1.0.1" or "release/1.2.0". In most cases replies to messages retain their context: the project, the release(s), .. The following config will propagate all project/... or release/... tags from a thread to all new messages. [PropagateTagsByRegexInThreadFilter.1] propagate_tags = project/.* # do not tag spam filter = not is:spam [PropagateTagsByRegexInThreadFilter.2] propagate_tags = release/.* Implementation spec: This filter will search through all (new) messages matched by ``filter``. For each message ``m`` found it goes through the messages thread an collects all assigned tags that match the regexp ``propagate_tags`` (``t``). All matching tags ``t`` are then assigned to the new message. """ def handle_message(self, message): thread_query = 'thread:"%s"' % (message.get_thread_id(),) if self._filter: query = self.database.get_messages("(%s) AND (%s)" % (thread_query, self._filter)) else: query = self.database.get_messages(thread_query) # the query can only be iterated once, then it is exhausted # https://git.notmuchmail.org/git?p=notmuch;a=blob;f=bindings/python/notmuch/messages.py;h=cae5da508f353f12cca585cb056c0b9ed92e29b3;hb=HEAD messages = list(query) # flatten tags tags_in_thread_t = {m.get_tags() for m in messages} # a set of Tags instances tags_in_thread = set(_flatten(tags_in_thread_t)) # filter tags propagatable_tags_in_thread = {tag for tag in tags_in_thread if self._propagate_tags.fullmatch(tag)} if len(propagatable_tags_in_thread): self.add_tags(message, *propagatable_tags_in_thread) def __init__(self, database, propagate_tags, filter=None, **kwargs): if filter: self.message = "Propagating tag(s) matching regexp /%s/ from threads to (new) messages matching '%s'" % ( propagate_tags, filter) else: self.message = "Propagating tag(s) matching regexp /%s/ from threads to (new) messages'" % ( propagate_tags,) self._filter = filter self._propagate_tags = re.compile(propagate_tags) super(PropagateTagsByRegexInThreadFilter, self).__init__(database, **kwargs) afew-3.0.1/afew/filters/PropagateTagsInThreadFilter.py000066400000000000000000000027661400000201700227220ustar00rootroot00000000000000# Copyright (c) Jens Neuhalfen from afew.filters.BaseFilter import Filter class PropagateTagsInThreadFilter(Filter): """ Propagate tags in threads. For each new message the mail thread is examined. If one of the configured tags is found, it is automatically attached to the new message. This enables a very easy workflow where entire threads can be tagged automatically. Config: [PropagateTagsInThreadFilter] propagate_tags = "project_A;billing;private" filter = not is:spam """ def handle_message(self, message): for tag in self._propagate_tags: tag_query = 'thread:"%s" AND is:"%s"' % (message.get_thread_id(), tag) if self._filter: query = self.database.get_messages("(%s) AND (%s)" % (tag_query, self._filter)) else: query = self.database.get_messages(tag_query) if len(list(query)): self.add_tags(message, tag) def __init__(self, database, propagate_tags="", filter=None, **kwargs): if filter: self.message = "Propagating tag(s) '%s' for messages matching '%s' to whole threads" % ( propagate_tags, filter) else: self.message = "Propagating tag(s) '%s' to whole threads" % (propagate_tags,) self._filter = filter self._propagate_tags = [t for t in propagate_tags.split(";") if len(t) > 0] super(PropagateTagsInThreadFilter, self).__init__(database, **kwargs) afew-3.0.1/afew/filters/SentMailsFilter.py000066400000000000000000000047301400000201700204320ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import re from afew.filters.BaseFilter import Filter from afew.NotmuchSettings import notmuch_settings class SentMailsFilter(Filter): message = 'Tagging all mails sent by myself to others' _bare_email_re = re.compile(r"[^<]*<(?P[^@<>]+@[^@<>]+)>") def __init__(self, database, sent_tag='', to_transforms=''): super().__init__(database) my_addresses = set() my_addresses.add(notmuch_settings.get('user', 'primary_email')) if notmuch_settings.has_option('user', 'other_email'): for other_email in notmuch_settings.get_list('user', 'other_email'): my_addresses.add(other_email) self.query = ( '(' + ' OR '.join('from:"%s"' % address for address in my_addresses) + ') AND NOT (' + ' OR '.join('to:"%s"' % address for address in my_addresses) + ')' ) self.sent_tag = sent_tag self.to_transforms = to_transforms if to_transforms: self.__email_to_tags = self.__build_email_to_tags(to_transforms) def handle_message(self, message): if self.sent_tag: self.add_tags(message, self.sent_tag) if self.to_transforms: for header in ('To', 'Cc', 'Bcc'): email = self.__get_bare_email(message.get_header(header)) for tag in self.__pick_tags(email): self.add_tags(message, tag) else: break def __build_email_to_tags(self, to_transforms): email_to_tags = dict() for rule in to_transforms.split(): if ':' in rule: email, tags = rule.split(':') email_to_tags[email] = tuple(tags.split(';')) else: email = rule email_to_tags[email] = tuple() return email_to_tags def __get_bare_email(self, email): if not '<' in email: return email else: match = self._bare_email_re.search(email) return match.group('email') def __pick_tags(self, email): if email in self.__email_to_tags: tags = self.__email_to_tags[email] if tags: return tags else: user_part, domain_part = email.split('@') return (user_part,) return tuple() afew-3.0.1/afew/filters/SpamFilter.py000066400000000000000000000010571400000201700174320ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> from afew.filters.HeaderMatchingFilter import HeaderMatchingFilter class SpamFilter(HeaderMatchingFilter): message = 'Tagging spam messages' header = 'X-Spam-Flag' pattern = 'YES' def __init__(self, database, tags='+spam', spam_tag=None, **kwargs): if spam_tag is not None: # this is for backward-compatibility tags = '+' + spam_tag kwargs['tags'] = [tags] super().__init__(database, **kwargs) afew-3.0.1/afew/filters/__init__.py000066400000000000000000000004201400000201700171140ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import os import glob __all__ = list(filename[:-3] for filename in glob.glob1(os.path.dirname(__file__), '*.py') if filename != '__init__.py') afew-3.0.1/afew/main.py000066400000000000000000000021131400000201700146320ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import sys from afew.MailMover import MailMover try: from .files import watch_for_new_files, quick_find_dirs_hack except ImportError: watch_available = False else: watch_available = True def main(options, database, query_string): if options.tag: for filter_ in options.enable_filters: filter_.run(query_string) filter_.commit(options.dry_run) elif options.watch: if not watch_available: sys.exit('Sorry, this feature requires Linux and pyinotify') watch_for_new_files(options, database, quick_find_dirs_hack(database.db_path)) elif options.move_mails: for maildir, rules in options.mail_move_rules.items(): mover = MailMover(options.mail_move_age, options.mail_move_rename, options.dry_run, options.notmuch_args) mover.move(maildir, rules) mover.close() else: sys.exit('Weird... please file a bug containing your command line.') afew-3.0.1/afew/tests/000077500000000000000000000000001400000201700145015ustar00rootroot00000000000000afew-3.0.1/afew/tests/__init__.py000066400000000000000000000001351400000201700166110ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) 2013 Patrick Gerken afew-3.0.1/afew/tests/test_dkimvalidityfilter.py000066400000000000000000000127151400000201700220200ustar00rootroot00000000000000"""Test suite for DKIMValidityFilter. """ import unittest from email.utils import make_msgid from unittest import mock import dkim import dns.exception from afew.Database import Database from afew.filters.DKIMValidityFilter import DKIMValidityFilter class _AddTags: # pylint: disable=too-few-public-methods """Mock for `add_tags` method of base filter. We need to easily collect tags added by filter for test assertion. """ def __init__(self, tags): self._tags = tags def __call__(self, message, *tags): self._tags.update(tags) def _make_dkim_validity_filter(): """Make `DKIMValidityFilter` with mocked `DKIMValidityFilter.add_tags` method, so in tests we can easily check what tags were added by filter without fiddling with db. """ tags = set() add_tags = _AddTags(tags) dkim_filter = DKIMValidityFilter(Database()) dkim_filter.add_tags = add_tags return dkim_filter, tags def _make_message(): """Make mock email Message. Mocked methods: - `get_header()` returns non-empty string. When testing with mocked function for verifying DKIM signature, DKIM signature doesn't matter as long as it's non-empty string. - `get_filenames()` returns list of non-empty string. When testing with mocked file open, it must just be non-empty string. - `get_message_id()` returns some generated message ID. """ message = mock.Mock() message.get_header.return_value = 'sig' message.get_filenames.return_value = ['a'] message.get_message_id.return_value = make_msgid() return message class TestDKIMValidityFilter(unittest.TestCase): """Test suite for `DKIMValidityFilter`. """ @mock.patch('afew.filters.DKIMValidityFilter.open', mock.mock_open(read_data=b'')) def test_no_dkim_header(self): """Test message without DKIM-Signature header doesn't get any tags. """ dkim_filter, tags = _make_dkim_validity_filter() message = _make_message() message.get_header.return_value = False with mock.patch('afew.filters.DKIMValidityFilter.dkim.verify') \ as dkim_verify: dkim_verify.return_value = True dkim_filter.handle_message(message) self.assertSetEqual(tags, set()) @mock.patch('afew.filters.DKIMValidityFilter.open', mock.mock_open(read_data=b'')) def test_dkim_all_ok(self): """Test message, with multiple files all having good signature, gets only 'dkim-ok' tag. """ dkim_filter, tags = _make_dkim_validity_filter() message = _make_message() message.get_filenames.return_value = ['a', 'b', 'c'] with mock.patch('afew.filters.DKIMValidityFilter.dkim.verify') \ as dkim_verify: dkim_verify.return_value = True dkim_filter.handle_message(message) self.assertSetEqual(tags, {'dkim-ok'}) @mock.patch('afew.filters.DKIMValidityFilter.open', mock.mock_open(read_data=b'')) def test_dkim_all_fail(self): """Test message, with multiple files all having bad signature, gets only 'dkim-fail' tag. """ dkim_filter, tags = _make_dkim_validity_filter() message = _make_message() message.get_filenames.return_value = ['a', 'b', 'c'] with mock.patch('afew.filters.DKIMValidityFilter.dkim.verify') \ as dkim_verify: dkim_verify.return_value = False dkim_filter.handle_message(message) self.assertSetEqual(tags, {'dkim-fail'}) @mock.patch('afew.filters.DKIMValidityFilter.open', mock.mock_open(read_data=b'')) def test_dkim_some_fail(self): """Test message, with multiple files but only some having bad signature, still gets only 'dkim-fail' tag. """ dkim_filter, tags = _make_dkim_validity_filter() message = _make_message() message.get_filenames.return_value = ['a', 'b', 'c'] with mock.patch('afew.filters.DKIMValidityFilter.dkim.verify') \ as dkim_verify: dkim_verify.side_effect = [True, False, True] dkim_filter.handle_message(message) self.assertSetEqual(tags, {'dkim-fail'}) @mock.patch('afew.filters.DKIMValidityFilter.open', mock.mock_open(read_data=b'')) def test_dkim_dns_resolve_failure(self): """Test message, on which DNS resolution failure happens when verifying DKIM signature, gets only 'dkim-fail' tag. """ dkim_filter, tags = _make_dkim_validity_filter() message = _make_message() with mock.patch('afew.filters.DKIMValidityFilter.dkim.verify') \ as dkim_verify: dkim_verify.side_effect = dns.resolver.NoNameservers() dkim_filter.handle_message(message) self.assertSetEqual(tags, {'dkim-fail'}) @mock.patch('afew.filters.DKIMValidityFilter.open', mock.mock_open(read_data=b'')) def test_dkim_verify_failed(self): """Test message, on which DKIM key parsing failure occurs, gets only 'dkim-fail' tag. """ dkim_filter, tags = _make_dkim_validity_filter() message = _make_message() with mock.patch('afew.filters.DKIMValidityFilter.dkim.verify') \ as dkim_verify: dkim_verify.side_effect = dkim.KeyFormatError() dkim_filter.handle_message(message) self.assertSetEqual(tags, {'dkim-fail'}) afew-3.0.1/afew/tests/test_mailmover.py000066400000000000000000000137271400000201700201170ustar00rootroot00000000000000# SPDX-License-Identifier: ISC import email.message from email.utils import make_msgid from freezegun import freeze_time import mailbox import notmuch import os import shutil import tempfile import unittest from afew.Database import Database from afew.NotmuchSettings import notmuch_settings, write_notmuch_settings def create_mail(msg, maildir, notmuch_db, tags, old=False): email_message = email.message.EmailMessage() # freezegun doesn't handle time zones properly when generating UNIX # timestamps. When the local timezone is UTC+2, the generated timestamp # is 2 hours ahead of what it should be. Due to this we need to make sure # that the dates are always sufficiently far behind 2019-01-30 12:00 to # handle up to UTC+12 . if old: email_message['Date'] = 'Wed, 10 Jan 2019 13:00:00 +0100' else: email_message['Date'] = 'Wed, 20 Jan 2019 13:00:00 +0100' email_message['From'] = 'You ' email_message['To'] = 'Me ' email_message['Message-ID'] = make_msgid() email_message.set_content(msg) maildir_message = mailbox.MaildirMessage(email_message) message_key = maildir.add(maildir_message) fname = os.path.join(maildir._path, maildir._lookup(message_key)) notmuch_msg = notmuch_db.add_message(fname) for tag in tags: notmuch_msg.add_tag(tag, False) # Remove the angle brackets automatically added around the message ID by make_msgid. stripped_msgid = email_message['Message-ID'].strip('<>') return (stripped_msgid, msg) @freeze_time("2019-01-30 12:00:00") class TestMailMover(unittest.TestCase): def setUp(self): self.test_dir = tempfile.mkdtemp() os.environ['MAILDIR'] = self.test_dir os.environ['NOTMUCH_CONFIG'] = os.path.join(self.test_dir, 'notmuch-config') notmuch_settings['database'] = {'path': self.test_dir} notmuch_settings['new'] = {'tags': 'new'} write_notmuch_settings() # Create notmuch database Database().open(create=True).close() self.root = mailbox.Maildir(self.test_dir) self.inbox = self.root.add_folder('inbox') self.archive = self.root.add_folder('archive') self.spam = self.root.add_folder('spam') # Dict of rules that are passed to MailMover. # # The top level key represents a particular mail directory to work on. # # The second level key is the notmuch query that MailMover will execute, # and its value is the directory to move the matching emails to. self.rules = { '.inbox': { 'tag:archive AND NOT tag:spam': '.archive', 'tag:spam': '.spam', }, '.archive': { 'NOT tag:archive AND NOT tag:spam': '.inbox', 'tag:spam': '.spam', }, '.spam': { 'NOT tag:spam AND tag:archive': '.archive', 'NOT tag:spam AND NOT tag:archive': '.inbox', }, } def tearDown(self): shutil.rmtree(self.test_dir) @staticmethod def get_folder_content(db, folder): return { (os.path.basename(msg.get_message_id()), msg.get_part(1).decode()) for msg in db.do_query('folder:{}'.format(folder)).search_messages() } def test_all_rule_cases(self): from afew import MailMover with Database() as db: expect_inbox = set([ create_mail('In inbox, untagged\n', self.inbox, db, []), create_mail('In archive, untagged\n', self.archive, db, []), create_mail('In spam, untagged\n', self.spam, db, []), ]) expect_archive = set([ create_mail('In inbox, tagged archive\n', self.inbox, db, ['archive']), create_mail('In archive, tagged archive\n', self.archive, db, ['archive']), create_mail('In spam, tagged archive\n', self.spam, db, ['archive']), ]) expect_spam = set([ create_mail('In inbox, tagged spam\n', self.inbox, db, ['spam']), create_mail('In inbox, tagged archive, spam\n', self.inbox, db, ['archive', 'spam']), create_mail('In archive, tagged spam\n', self.archive, db, ['spam']), create_mail('In archive, tagged archive, spam\n', self.archive, db, ['archive', 'spam']), create_mail('In spam, tagged spam\n', self.spam, db, ['spam']), create_mail('In spam, tagged archive, spam\n', self.spam, db, ['archive', 'spam']), ]) mover = MailMover.MailMover(quiet=True) mover.move('.inbox', self.rules['.inbox']) mover.move('.archive', self.rules['.archive']) mover.move('.spam', self.rules['.spam']) mover.close() with Database() as db: self.assertEqual(expect_inbox, self.get_folder_content(db, '.inbox')) self.assertEqual(expect_archive, self.get_folder_content(db, '.archive')) self.assertEqual(expect_spam, self.get_folder_content(db, '.spam')) def test_max_age(self): from afew import MailMover with Database() as db: expect_inbox = set([ create_mail('In inbox, tagged archive, old\n', self.inbox, db, ['archive'], old=True), ]) expect_archive = set([ create_mail('In inbox, tagged archive\n', self.inbox, db, ['archive']), ]) expect_spam = set([]) mover = MailMover.MailMover(max_age=15, quiet=True) mover.move('.inbox', self.rules['.inbox']) mover.move('.archive', self.rules['.archive']) mover.move('.spam', self.rules['.spam']) mover.close() with Database() as db: self.assertEqual(expect_inbox, self.get_folder_content(db, '.inbox')) self.assertEqual(expect_archive, self.get_folder_content(db, '.archive')) self.assertEqual(expect_spam, self.get_folder_content(db, '.spam')) afew-3.0.1/afew/tests/test_settings.py000066400000000000000000000016261400000201700177570ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) 2013 Patrick Gerken import unittest class TestFilterRegistry(unittest.TestCase): def test_all_filters_exist(self): from afew import FilterRegistry self.assertTrue(hasattr(FilterRegistry.all_filters, 'get')) def test_entry_point_registration(self): from afew import FilterRegistry class FakeRegistry: name = 'test' def load(self): return 'class' registry = FilterRegistry.FilterRegistry([FakeRegistry()]) self.assertEqual('class', registry['test']) def test_add_FilterRegistry(self): from afew import FilterRegistry try: FilterRegistry.all_filters['test'] = 'class' self.assertEqual('class', FilterRegistry.all_filters['test']) finally: del FilterRegistry.all_filters['test'] afew-3.0.1/afew/tests/test_utils.py000066400000000000000000000002121400000201700172450ustar00rootroot00000000000000# import unittest from afew import utils class TestUtils(unittest.TestCase): pass if __name__ == '__main__': unittest.main() afew-3.0.1/afew/utils.py000066400000000000000000000012141400000201700150470ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import re from datetime import datetime def get_message_summary(message): when = datetime.fromtimestamp(float(message.get_date())) sender = get_sender(message) subject = message.get_header('Subject') return '[{date}] {sender} | {subject}'.format(date=when, sender=sender, subject=subject) def get_sender(message): sender = message.get_header('From') name_match = re.search(r'(.+) <.+@.+\..+>', sender) if name_match: sender = name_match.group(1) return sender afew-3.0.1/afew/version.py000066400000000000000000000001641400000201700153770ustar00rootroot00000000000000# coding: utf-8 # file generated by setuptools_scm # don't change, don't track in version control version = '3.0.1' afew-3.0.1/docs/000077500000000000000000000000001400000201700133455ustar00rootroot00000000000000afew-3.0.1/docs/_static/000077500000000000000000000000001400000201700147735ustar00rootroot00000000000000afew-3.0.1/docs/_static/.keep000066400000000000000000000000001400000201700157060ustar00rootroot00000000000000afew-3.0.1/docs/commandline.rst000066400000000000000000000072151400000201700163720ustar00rootroot00000000000000Command Line Usage ================== Ultimately afew is a command line tool. You have to specify an action, and whether to act on all messages, or only on new messages. The actions you can choose from are: tag run the tag filters. See `Initial tagging`_. watch continuously monitor the mailbox for new files move-mails move mail files between maildir folders Initial tagging --------------- Basic tagging stuff requires no configuration, just run .. code-block:: sh $ afew --tag --new # or to tag *all* messages $ afew --tag --all To do this automatically you can add the following hook into your `~/.offlineimaprc`: .. code-block:: ini postsynchook = ionice -c 3 chrt --idle 0 /bin/sh -c "notmuch new && afew --tag --new" There is a lot more to say about general filter :doc:`configuration` and the different :doc:`filters` provided by afew. Simulation ^^^^^^^^^^ Adding `--dry-run` to any `--tag` or `--sync-tags` action prevents modification of the notmuch db. Add some `-vv` goodness to see some action. Move Mode --------- To invoke afew in move mode, provide the `--move-mails` option on the command line. Move mode will respect `--dry-run`, so throw in `--verbose` and watch what effects a real run would have. In move mode, afew will check all mails (or only recent ones) in the configured maildir folders, deciding whether they should be moved to another folder. The decision is based on rules defined in your config file. A rule is bound to a source folder and specifies a target folder into which a mail will be moved that is matched by an associated query. This way you will be able to transfer your sorting principles roughly to the classic folder based maildir structure understood by your traditional mail server. Tag your mails with notmuch, call afew `--move-mails` in an offlineimap presynchook and enjoy a clean inbox in your webinterface/GUI-client at work. Note that in move mode, afew calls `notmuch new` after moving mails around. You can use `afew -m --notmuch-args=--no-hooks` in a pre-new notmuch hook to avoid loops. For information on how to configure rules for move mode, what you can do with it and what you can't, please refer to :doc:`move_mode`. Commandline help ---------------- The full set of options is: .. code-block:: sh $ afew --help Usage: afew [options] [--] [query] Options: -h, --help show this help message and exit Actions: Please specify exactly one action. -t, --tag run the tag filters -w, --watch continuously monitor the mailbox for new files -m, --move-mails move mail files between maildir folders Query modifiers: Please specify either --all or --new or a query string. -a, --all operate on all messages -n, --new operate on all new messages General options: -C NOTMUCH_CONFIG, --notmuch-config=NOTMUCH_CONFIG path to the notmuch configuration file [default: $NOTMUCH_CONFIG or ~/.notmuch-config] -e ENABLE_FILTERS, --enable-filters=ENABLE_FILTERS filter classes to use, separated by ',' [default: filters specified in afew's config] -d, --dry-run don't change the db [default: False] -R REFERENCE_SET_SIZE, --reference-set-size=REFERENCE_SET_SIZE size of the reference set [default: 1000] -T DAYS, --reference-set-timeframe=DAYS do not use mails older than DAYS days [default: 30] -v, --verbose be more verbose, can be given multiple times afew-3.0.1/docs/conf.py000066400000000000000000000175201400000201700146510ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) 2011 Justus Winter <4winter@informatik.uni-hamburg.de> # afew documentation build configuration file, created by # sphinx-quickstart on Fri Dec 23 21:19:37 2011. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys, os from pkg_resources import get_distribution # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath('../..')) # Create mocks so we don't depend on non standard modules to build the # documentation class Mock: def __init__(self, *args, **kwargs): pass def __getattr__(self, name): return Mock if name != '__file__' else '/dev/null' MOCK_MODULES = [ 'notmuch', 'notmuch.globals', 'argparse', ] for mod_name in MOCK_MODULES: sys.modules[mod_name] = Mock() # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.coverage', 'sphinx.ext.viewcode'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'afew' copyright = u'afewmail project' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The full version, including alpha/beta/rc tags. pretended_version = os.environ.get('SETUPTOOLS_SCM_PRETEND_VERSION') if pretended_version: release = pretended_version else: release = get_distribution('afew').version # The X.Y.Z version. version = '.'.join(release.split('.')[:3]) # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'afewdoc' # -- Options for LaTeX output -------------------------------------------------- # The paper size ('letter' or 'a4'). #latex_paper_size = 'letter' # The font size ('10pt', '11pt' or '12pt'). #latex_font_size = '10pt' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'afew.tex', u'afew Documentation', u'Justus Winter', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Additional stuff for the LaTeX preamble. #latex_preamble = '' # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'afew', u'afew Documentation', [u'Justus Winter'], 1) ] # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = { 'python': ('https://docs.python.org/', None), 'notmuch': ('https://notmuch.readthedocs.io/en/latest/', None), 'alot': ('https://alot.readthedocs.io/en/latest/', None), } afew-3.0.1/docs/configuration.rst000066400000000000000000000124371400000201700167550ustar00rootroot00000000000000Configuration ============= Configuration File ------------------ Customization of tag filters takes place in afew's config file in `~/.config/afew/config`. NotMuch Config -------------- afew tries to adapt to the new tag that notmuch sets on new email, but has mostly been developed and used against the **new** tag. To use that, make sure that `~/.notmuch-config` contains: .. code-block:: ini [new] tags=new afew reads the notmuch database location from notmuch config. When no database path is set in notmuch config, afew uses the `MAILDIR` environment variable when set, or `$HOME/mail` as a fallback, like notmuch CLI does. If a relative path is provided, afew prepends `$HOME/` to the path in the same manner as notmuch, which was introduced in version 0.28 of notmuch. Filter Configuration -------------------- You can modify filters, and define your own versions of the base Filter that allow you to tag messages in a similar way to the `notmuch tag` command, using the config file. The default config file is: .. code-block:: ini [SpamFilter] [KillThreadsFilter] [ListMailsFilter] [ArchiveSentMailsFilter] sent_tag = '' [InboxFilter] See the :doc:`filters` page for the details of those filters and the custom arguments they accept. You can add filters based on the base filter as well. These can be customised by specifying settings beneath them. The standard settings, which apply to all filters, are: message text that will be displayed while running this filter if the verbosity is high enough. query the query to use against the messages, specified in standard notmuch format. Note that you don't need to specify the **new** tag - afew will add that when run with the `--new` flag. tags the tags to add or remove for messages that match the query. Tags to add are preceded by a **+** and tags to remove are preceded by a **-**. Multiple tags are separated by semicolons. tags_blacklist if the message has one of these tags, don't add `tags` to it. Tags are separated by semicolons. So to add the **deer** tag to any message to or from `antelope@deer.com` you could do: .. code-block:: ini [Filter.1] query = 'antelope@deer.com' tags = +deer message = Wild animals ahoy You can also (in combination with the InboxFilter) have email skip the Inbox by removing the new tag before you get to the InboxFilter: .. code-block:: ini [Filter.2] query = from'pointyheaded@boss.com' tags = -new;+boss message = Message from above Full Sample Config ------------------ Showing some sample configs is the easiest way to understand. The `notmuch initial tagging page`_ shows a sample config: .. _notmuch initial tagging page: http://notmuchmail.org/initial_tagging/ .. code-block:: sh # immediately archive all messages from "me" notmuch tag -new -- tag:new and from:me@example.com # delete all messages from a spammer: notmuch tag +deleted -- tag:new and from:spam@spam.com # tag all message from notmuch mailing list notmuch tag +notmuch -- tag:new and to:notmuch@notmuchmail.org # finally, retag all "new" messages "inbox" and "unread" notmuch tag +inbox +unread -new -- tag:new The (roughly) equivalent set up in afew would be: .. code-block:: ini [ArchiveSentMailsFilter] [Filter.1] message = Delete all messages from spammer query = from:spam@spam.com tags = +deleted;-new [Filter.2] message = Tag all messages from the notmuch mailing list query = to:notmuch@notmuchmail.org tags = +notmuch [InboxFilter] Not that the queries do not generally include `tag:new` because this is implied when afew is run with the `--new` flag. The differences between them is that * the ArchiveSentMailsFilter will add tags specified by `sent_tag` option (default `''` means add no tags. You may want to set it to `sent`), as well as archiving the email. And it will not archive email that has been sent to one of your own addresses. * the InboxFilter does not add the **unread** tag. But most mail clients will manage the unread status directly in maildir. More Filter Examples -------------------- Here are a few more example filters from github dotfiles: .. code-block:: ini [Filter.1] query = 'sicsa-students@sicsa.ac.uk' tags = +sicsa message = sicsa [Filter.2] query = 'from:foosoc.ed@gmail.com OR from:GT Silber OR from:lizzie.brough@eusa.ed.ac.uk' tags = +soc;+foo message = foosoc [Filter.3] query = 'folder:gmail/G+' tags = +G+ message = gmail spam # skip inbox [Filter.6] query = 'to:notmuch@notmuchmail.org AND (subject:emacs OR subject:elisp OR "(defun" OR "(setq" OR PATCH)' tags = -new message = notmuch emacs stuff # Assuming the following workflow: all messages for projects or releases should be tagged # as "project/A", "project/B" respectively "release/1.0.1" or "release/1.2.0". # # In most cases replies to messages retain their context: the project, the release(s), .. # # The following config will propagate all project/... or release/... tags from a thread # to all new messages. [PropagateTagsByRegexInThreadFilter.1] propagate_tags = project/.* # do not tag spam filter = not is:spam [PropagateTagsByRegexInThreadFilter.2] propagate_tags = release/.*afew-3.0.1/docs/extending.rst000066400000000000000000000035641400000201700160740ustar00rootroot00000000000000Extending afew ============== You can put python files in `~/.config/afew/` and they will be imported by afew. If you use that python file to define a `Filter` class and use the `register_filter` decorator then you can refer to it in your filter configuration. So an example small filter you could add might be: .. code-block:: python from afew.filters.BaseFilter import Filter from afew.FilterRegistry import register_filter PROJECT_MAPPING = { 'fabric': 'deployment', 'oldname': 'new-name', } @register_filter class RedmineFilter(Filter): message = 'Create tag based on redmine project' query = 'NOT tag:redmine' def handle_message(self, message): project = message.get_header('X-Redmine-Project') if project in PROJECT_MAPPING: project = PROJECT_MAPPING[project] self.add_tags(message, 'redmine', project) We have defined the `message` and `query` class variables that are used by the parent class `Filter`. The `message` is printed when running with verbose flags. The `query` is used to select messages to run against - here we ensure we don't bother looking at messages we've already looked at. The `handle_message()` method is the key one to implement. This will be called for each message that matches the query. The argument is a `notmuch message object`_ and the key methods used by the afew filters are `get_header()`, `get_filename()` and `get_thread()`. .. _notmuch message object: http://pythonhosted.org/notmuch/#message-a-single-message Of the methods inherited from the `Filter` class the key ones are `add_tags()` and `remove_tags()`, but read about the :doc:`implementation` or just read the source code to get your own ideas. Once you've defined your filter, you can add it to your config like any other filter: .. code-block:: ini [RedmineFilter] afew-3.0.1/docs/filters.rst000066400000000000000000000174231400000201700155560ustar00rootroot00000000000000Filters ======= The default filter set (if you don't specify anything in the config) is: .. code-block:: ini [SpamFilter] [KillThreadsFilter] [ListMailsFilter] [ArchiveSentMailsFilter] [InboxFilter] The standard filter :doc:`configuration` can be applied to these filters as well. Though note that most of the filters below set their own value for message, query and/or tags, and some ignore some of the standard settings. ArchiveSentMailsFilter ---------------------- It extends `SentMailsFilter` with the following feature: * Emails filtered by this filter have the **new** tag removed, so will not have the **inbox** tag added by the InboxFilter. DKIMValidityFilter ------------------ This filter verifies DKIM signatures of E-Mails with DKIM header, and adds `dkin-ok` or `dkin-fail` tags. DMARCReportInspectionFilter --------------------------- DMARC reports usually come in ZIP files. To check the report you have to unpack and search thru XML document which is very tedious. This filter tags the message as follows: if there's any SPF failure in any attachment, tag the message with "dmarc-spf-fail" tag, otherwise tag with "dmarc-spf-ok" if there's any DKIM failure in any attachment, tag the message with "dmarc-dkim-fail" tag, otherwise tag with "dmarc-dkim-ok" FolderNameFilter ---------------- For each email, it looks at all folders it is in, and uses the path and filename as a tag, for the email. So if you have a procmail or sieve set up that puts emails in folders for you, this might be useful. * folder_explicit_list = * Tag mails with tag in only. is a space separated list, not enclosed in quotes or any other way. * Empty list means all folders (of course blacklist still applies). * The default is empty list. * You may use it e.g. to set tags only for specific folders like 'Sent'. * folder_blacklist = * Never tag mails with tag in . is a space separated list, not enclosed in quotes or any other way. * The default is to blacklist no folders. * You may use it e.g. to avoid mails being tagged as 'INBOX' when there is the more standard 'inbox' tag. * folder_transforms = * Transform folder names according to the specified rules before tagging mails. is a space separated list consisting of 'folder:tag' style pairs. The colon separates the name of the folder to be transformed from the tag it is to be transformed into. * The default is to transform to folder names. * You may use the rules e.g. to transform the name of your 'Junk' folder into your 'spam' tag or fix capitalization of your draft and sent folder: .. code-block:: ini folder_transforms = Junk:spam Drafts:draft Sent:sent * folder_lowercases = true * Use lowercase tags for all folder names * maildir_separator = * Use to split your maildir hierarchy into individual tags. * The default is to split on '.' * If your maildir hierarchy is represented in the filesystem as collapsed dirs, is used to split it again before applying tags. If your maildir looks like this: .. code-block:: ini [...] /path/to/maildir/devel.afew/[cur|new|tmp]/... /path/to/maildir/devel.alot/[cur|new|tmp]/... /path/to/maildir/devel.notmuch/[cur|new|tmp]/... [...] the mails in your afew folder will be tagged with 'devel' and 'afew'. If instead your hierarchy is split by a more conventional '/' or any other divider .. code-block:: ini [...] /path/to/maildir/devel/afew/[cur|new|tmp]/... /path/to/maildir/devel/alot/[cur|new|tmp]/... /path/to/maildir/devel/notmuch/[cur|new|tmp]/... [...] you need to configure that divider to have your mails properly tagged: .. code-block:: ini maildir_separator = / HeaderMatchingFilter -------------------- This filter adds tags to a message if the named header matches the regular expression given. The tags can be set, or based on the match. The settings you can use are: * header = * pattern = * tags = If you surround a tag with `{}` then it will be replaced with the named match. Some examples are: .. code-block:: ini [HeaderMatchingFilter.1] header = X-Spam-Flag pattern = YES tags = +spam [HeaderMatchingFilter.2] header = List-Id pattern = <(?P.*)> tags = +lists;+{list_id} [HeaderMatchingFilter.3] header = X-Redmine-Project pattern = (?P.*) tags = +redmine;+{project} SpamFilter and ListMailsFilter are implemented using HeaderMatchingFilter, and are only slightly more complicated than the above examples. InboxFilter ----------- This removes the **new** tag, and adds the **inbox** tag, to any message that isn't killed or spam. (The new tags are set in your notmuch config, and default to just **new**.) KillThreadsFilter ----------------- If the new message has been added to a thread that has already been tagged **killed** then add the **killed** tag to this message. This allows for ignoring all replies to a particular thread. ListMailsFilter --------------- This filter looks for the `List-Id` header, and if it finds it, adds a tag **lists** and a tag named **lists/**. MeFilter -------- Add filter tagging mail sent directly to any of addresses defined in Notmuch config file: `primary_email` or `other_email`. Default tag is `to-me` and can be customized with `me_tag` option. SentMailsFilter --------------- The settings you can use are: * sent_tag = * Add to all mails sent from one of your configured mail addresses, *and not* to any of your addresses. * The default is to add no tag, so you need to specify something. * You may e.g. use it to tag all mails sent by you as 'sent'. This may make special sense in conjunction with a mail client that is able to not only search for threads but individual mails as well. * to_transforms = * Transform `To`/`Cc`/`Bcc` e-mail addresses to tags according to the specified rules. is a space separated list consisting of 'user_part@domain_part:tags' style pairs. The colon separates the e-mail address to be transformed from tags it is to be transformed into. ':tags' is optional and if empty, 'user_part' is used as tag. 'tags' can be a single tag or semi-colon separated list of tags. * It can be used for example to easily tag posts sent to mailing lists which at this stage don't have `List-Id` field. SpamFilter ---------- The settings you can use are: * spam_tag = * Add to all mails recognized as spam. * The default is 'spam'. * You may use it to tag your spam as 'junk', 'scum' or whatever suits your mood. Note that only a single tag is supported here. Email will be considered spam if the header `X-Spam-Flag` is present. Customizing filters ------------------- To customize these filters, there are basically two different possibilities: Let's say you like the SpamFilter, but it is way too polite 1. Create an filter object and customize it .. code-block:: ini [SpamFilter.0] # note the index message = meh The index is required if you want to create a new SpamFilter *in addition to* the default one. If you need just one customized SpamFilter, you can drop the index and customize the default instance. 2. Create a new type... .. code-block:: ini [ShitFilter(SpamFilter)] message = I hatez teh spam! and create an object or two .. code-block:: ini [ShitFilter.0] [ShitFilter.1] message = Me hatez it too. You can provide your own filter implementations too. You have to register your filters via entry points. See the afew setup.py for examples on how to register your filters. To add your filters, you just need to install your package in the context of the afew application. afew-3.0.1/docs/implementation.rst000066400000000000000000000011311400000201700171200ustar00rootroot00000000000000Implementation ============== Database Manager ---------------- The design of the database manager was inspired by alots database manager :class:`alot.db.DBManager`. .. module:: afew.Database .. autoclass:: Database :members: Filter ------ .. module:: afew.filters.BaseFilter .. autoclass:: Filter :members: Configuration management ------------------------ .. automodule:: afew.Settings :members: .. automodule:: afew.NotmuchSettings :members: Miscellanious utility functions ------------------------------- .. currentmodule:: afew.utils .. automodule:: afew.utils :members: afew-3.0.1/docs/index.rst000066400000000000000000000015561400000201700152150ustar00rootroot00000000000000Welcome to afew's documentation! ================================ `afew` is an initial tagging script for notmuch mail: * http://notmuchmail.org/ * http://notmuchmail.org/initial_tagging/ Its basic task is to provide automatic tagging each time new mail is registered with notmuch. In a classic setup, you might call it after `notmuch new` in an offlineimap post sync hook or in the notmuch `post-new` hook. It can do basic thing such as adding tags based on email headers or maildir folders, handling killed threads and spam. fyi: afew plays nicely with `alot`, a GUI for notmuch mail ;) * https://github.com/pazz/alot Contents: .. toctree:: :maxdepth: 2 quickstart installation commandline configuration filters move_mode extending implementation Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` afew-3.0.1/docs/installation.rst000066400000000000000000000024551400000201700166060ustar00rootroot00000000000000Installation ============ Requirements ------------ afew works with python 3.6+, and requires notmuch and its python bindings. On Debian/Ubuntu systems you can install these by doing: .. code-block:: sh $ sudo aptitude install notmuch python-notmuch python-dev python-setuptools Note: if you are installing `notmuch` using Homebrew on macOS, make sure to run ``$ brew install --with-python3 notmuch``, because the brew formula doesn't install python3 notmuch bindings by default. Unprivileged Install -------------------- It is recommended to install `afew` itself inside a virtualenv as an unprivileged user, either via checking out the source code and installing via setup.py, or via pip. .. code:: bash # create and activate virtualenv $ python -m venv --system-site-packages .venv $ source .venv/bin/activate # install via pip from PyPI: $ pip install afew # or install from source: $ python setup.py install --prefix=~/.local You might want to symlink `.venv/bin/afew` somewhere inside your path (~/bin/ in this case): .. code:: bash $ ln -snr .venv/bin/afew ~/.bin/afew Building documentation ---------------------- Documentation can be built in various formats using Sphinx: .. code:: bash # build docs into build/sphinx/{html,man} $ python setup.py build_sphinx -b html,man afew-3.0.1/docs/move_mode.rst000066400000000000000000000072661400000201700160640ustar00rootroot00000000000000Move Mode ========= Configuration Section --------------------- Here is a full sample configuration for move mode: .. code-block:: ini [MailMover] folders = INBOX Junk rename = False max_age = 15 # rules INBOX = 'tag:spam':Junk 'NOT tag:inbox':Archive Junk = 'NOT tag:spam AND tag:inbox':INBOX 'NOT tag:spam':Archive Below we explain what each bit of this means. Rules ----- First you need to specify which folders should be checked for mails that are to be moved (as a whitespace separated list). Folder names containing whitespace need to be quoted: .. code-block:: ini folders = INBOX Junk "Sent Mail" Then you have to specify rules that define move actions of the form .. code-block:: ini = ['':]+ Every mail in the `` folder that matches a `` will be moved into the `` folder associated with that query. A message that matches multiple queries will be copied to multiple destinations. You can bind as many rules to a maildir folder as you deem necessary. Just add them as elements of a (whitespace separated) list. Please note, though, that you need to specify at least one rule for every folder given by the `folders` option and at least one folder to check in order to use the move mode. .. code-block:: ini INBOX = 'tag:spam':Junk will bind one rule to the maildir folder `INBOX` that states that all mails in said folder that carry (potentially among others) the tag **spam** are to be moved into the folder `Junk`. With `` being an arbitrary notmuch query, you have the power to construct arbitrarily flexible rules. You can check for the absence of tags and look out for combinations of attributes: .. code-block:: ini Junk = 'NOT tag:spam AND tag:inbox':INBOX 'NOT tag:spam':Archive The above rules will move all mails in `Junk` that don't have the **spam** tag but do have an **inbox** tag into the directory `INBOX`. All other mails not tagged with **spam** will be moved into `Archive`. Max Age ------- You can limit the age of mails you want to move by setting the `max_age` option in the configuration section. By providing .. code-block:: ini max_age = 15 afew will only check mails at most 15 days old. Rename ------ Set this option if you are using the `mbsync` IMAP syncing tool. `mbsync` adds a unique identifier to files' names when it syncs them. If the `rename` option is not set, moving files can cause UID conflicts and prevent `mbsync` from syncing with error messages such as "Maildir error: duplicate UID 1234" or "UID 567 is beyond highest assigned UID 89". When the option is set, afew will rename files while moving them, removing the UID but preserving other `mbsync` information. This allows `mbsync` to assign a new UID to the file and avoid UID conflicts. If you are using `offlineimap`, you can safely ignore this option. .. code-block:: ini rename = True Limitations ----------- **(1)** Rules don't manipulate tags. .. code-block:: ini INBOX = 'NOT tag:inbox':Archive Junk = 'NOT tag:spam':INBOX The above combination of rules might prove tricky, since you might expect de-spammed mails to end up in `INBOX`. But since the `Junk` rule will *not* add an **inbox** tag, the next run in move mode might very well move the matching mails into `Archive`. Then again, if you remove the **spam** tag and do not set an **inbox** tag, how would you come to expect the mail would end up in your INBOX folder after moving it? ;) **(2)** There is no 1:1 mapping between folders and tags. And that's a feature. If you tag a mail with two tags and there is a rule for each of them, both rules will apply. Your mail will be copied into two destination folders, then removed from its original location. afew-3.0.1/docs/quickstart.rst000066400000000000000000000030531400000201700162720ustar00rootroot00000000000000Quick Start =========== The steps to get up and running are: * install the afew package * create the config files * add a notmuch post-new hook that calls afew Install ------- The following commands will get you going on Debian/Ubuntu systems: .. code-block:: sh $ sudo aptitude install notmuch python-notmuch dbacl $ git clone git://github.com/teythoon/afew.git $ cd afew $ python setup.py install --prefix=~/.local Ensure that `~/.local/bin` is in your path. One way is to add the following to your `~/.bashrc`: .. code-block:: sh if [ -d ~/.local/bin ]; then PATH=$PATH:~/.local/bin fi See :doc:`installation` for a more detailed guide. Initial Config -------------- Make sure that `~/.notmuch-config` reads: .. code-block:: ini [new] tags=new Put a list of filters into `~/.config/afew/config`: .. code-block:: ini # This is the default filter chain [SpamFilter] [KillThreadsFilter] [ListMailsFilter] [ArchiveSentMailsFilter] [InboxFilter] And create a `post-new` hook for notmuch to call afew: .. code-block:: sh $ notmuchdir=path/to/maildir/.notmuch $ mkdir -p "$notmuchdir/hooks" $ printf > "$notmuchdir/hooks/post-new" '#!/usr/bin/env sh\n$HOME/.local/bin/afew --tag --new\n' $ chmod u+x "$notmuchdir/hooks/post-new" Next Steps ---------- You can: * add extra :doc:`filters` for more custom filtering * make use of the :doc:`move_mode` to move your email between folders * run afew against all your old mail by running `afew --tag --all` * start :doc:`extending` afew afew-3.0.1/setup.cfg000066400000000000000000000000461400000201700142360ustar00rootroot00000000000000[egg_info] tag_build = tag_date = 0 afew-3.0.1/setup.py000077500000000000000000000047541400000201700141440ustar00rootroot00000000000000# SPDX-License-Identifier: ISC # Copyright (c) Justus Winter <4winter@informatik.uni-hamburg.de> import os from setuptools import setup, find_packages def get_requires(): if os.environ.get('TRAVIS') != 'true' and os.environ.get('READTHEDOCS') != 'True': yield 'notmuch' yield 'chardet' yield 'dkimpy' setup( name='afew', use_scm_version={'write_to': 'afew/version.py'}, description="An initial tagging script for notmuch mail", url="https://github.com/afewmail/afew", license="ISC", setup_requires=['setuptools_scm'], packages=find_packages(), test_suite='afew.tests', package_data={ 'afew': ['defaults/afew.config'] }, entry_points={ 'console_scripts': [ 'afew = afew.commands:main'], 'afew.filter': [ 'Filter = afew.filters.BaseFilter:Filter', 'ArchiveSentMailsFilter = afew.filters.ArchiveSentMailsFilter:ArchiveSentMailsFilter', 'DKIMValidityFilter = afew.filters.DKIMValidityFilter:DKIMValidityFilter', 'DMARCReportInspectionFilter = afew.filters.DMARCReportInspectionFilter:DMARCReportInspectionFilter', 'FolderNameFilter = afew.filters.FolderNameFilter:FolderNameFilter', 'HeaderMatchingFilter = afew.filters.HeaderMatchingFilter:HeaderMatchingFilter', 'InboxFilter = afew.filters.InboxFilter:InboxFilter', 'KillThreadsFilter = afew.filters.KillThreadsFilter:KillThreadsFilter', 'ListMailsFilter = afew.filters.ListMailsFilter:ListMailsFilter', 'MeFilter = afew.filters.MeFilter:MeFilter', 'SentMailsFilter = afew.filters.SentMailsFilter:SentMailsFilter', 'SpamFilter = afew.filters.SpamFilter:SpamFilter', 'PropagateTagsByRegexInThreadFilter = afew.filters.PropagateTagsByRegexInThreadFilter:PropagateTagsByRegexInThreadFilter', 'PropagateTagsInThreadFilter = afew.filters.PropagateTagsInThreadFilter:PropagateTagsInThreadFilter', ], }, install_requires=list(get_requires()), tests_require=['freezegun'], provides=['afew'], classifiers=[ 'License :: OSI Approved :: ISC License (ISCL)', 'Development Status :: 4 - Beta', 'Environment :: Console', 'Intended Audience :: End Users/Desktop', 'Programming Language :: Python', 'Topic :: Communications :: Email', 'Topic :: Communications :: Email :: Filters', 'Topic :: Utilities', 'Topic :: Database', ], )