././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1607203129.381565 scruffington-0.3.8.2/0000755000076700000240000000000000000000000013204 5ustar00zachstaff././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1607203129.3818126 scruffington-0.3.8.2/PKG-INFO0000644000076700000240000001446200000000000014310 0ustar00zachstaffMetadata-Version: 2.1 Name: scruffington Version: 0.3.8.2 Summary: The janitor Home-page: https://github.com/snare/scruffy Author: snare Author-email: snare@ho.ax License: MIT Description: Scruffy ======= .. image:: https://img.shields.io/travis/snare/scruffy.svg :target: https://travis-ci.org/snare/scruffy .. image:: https://img.shields.io/pypi/format/scruffington.svg :target: https://pypi.python.org/pypi/scruffington .. image:: https://readthedocs.org/projects/scruffy/badge/?version=latest :target: http://scruffy.readthedocs.org/en/latest/ *Scruffy. The Janitor.* Scruffy is a framework for taking care of a bunch of boilerplate in Python apps. It handles the loading of configuration files, the loading and management of plugins, and the management of other filesystem resources such as temporary files and directories, log files, etc. A typical use case for Scruffy is a command-line Python tool with some or all of the following requirements: * Read a set of configuration defaults * Read a local configuration file and apply it on top of the defaults * Allow overriding some configuration options with command line flags or at runtime * Load a core set of Python-based plugins * Load a set of user-defined Python-based plugins * Generate log files whose name, location and other logging settings are based on configuration * Store application state between runs in a file or database Scruffy is used by Voltron_ and Calculon_ .. _Voltron: https://github.com/snare/voltron .. _Calculon: https://github.com/snare/calculon Installation ------------ A standard python setup script is included. $ python setup.py install This will install the Scruffy package wherever that happens on your system. Alternately, Scruffy can be installed with `pip` from PyPi (where it's called `scruffington`, because I didn't check for a conflict before I named it). $ pip install scruffington Documentation ------------- Full documentation is hosted at readthedocs_ .. _readthedocs: http://scruffy.readthedocs.io/ Quick start ----------- Config ~~~~~~ Load a user config file, and apply it on top of a set of defaults loaded from inside the Python package we're currently running from. *thingy.yaml*: .. code:: yaml some_property: 1 other_property: a thing *thingy.py*: .. code:: python from scruffy import ConfigFile c = ConfigFile('thingy.yaml', load=True, defaults=File('defaults.yaml', parent=PackageDirectory()) ) print("c.some_property == {c.some_property}".format(c=c)) print("c.other_property == {c.other_property}".format(c=c)) Run it: :: $ python thingy.py c.some_property == 1 c.other_property == a thing Plugins ~~~~~~~ Load some plugins. *~/.thingy/plugins/example.py*: .. code:: python from scruffy import Plugin class ExamplePlugin(Plugin): def do_a_thing(self): print('{}.{} is doing a thing'.format(__name__, self.__class__.__name__)) *thingy.py*: .. code:: python from scruffy import PluginDirectory, PluginRegistry pd = PluginDirectory('~/.thingy/plugins') pd.load() for p in PluginRegistry.plugins: print("Initialising plugin {}".format(p)) p().do_a_thing() Run it: :: $ python thingy.py Initialising plugin example.ExamplePlugin is doing a thing Logging ~~~~~~~ Scruffy's `LogFile` class will do some configuration of Python's `logging` module. *log.py*: .. code:: python import logging from scruffy import LogFile log = logging.getLogger('main') log.setLevel(logging.INFO) LogFile('/tmp/thingy.log', logger='main').configure() log.info('Hello from log.py') */tmp/thingy.log*: :: Hello from log.py Environment ~~~~~~~~~~~ Scruffy's `Environment` class ties all the other stuff together. The other classes can be instantiated as named children of an `Environment`, which will load any `Config` objects, apply the configs to the other objects, and then prepare the other objects. *~/.thingy/config*: .. code:: yaml log_dir: /tmp/logs log_file: thingy.log *env.py*: .. code:: python from scruffy import * e = Environment( main_dir=Directory('~/.thingy', create=True, config=ConfigFile('config', defaults=File('defaults.yaml', parent=PackageDirectory())), lock=LockFile('lock') user_plugins=PluginDirectory('plugins') ), log_dir=Directory('{config:log_dir}', create=True LogFile('{config:log_file}', logger='main') ), pkg_plugins=PluginDirectory('plugins', parent=PackageDirectory()) ) License ------- See LICENSE file. If you use this and don't hate it, buy me a beer at a conference some time. Credits ------- Props to richo_. Flat duck pride. .. _richo: http://github.com/richo Keywords: scruffy Platform: UNKNOWN Description-Content-Type: text/x-rst ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1563251112.0 scruffington-0.3.8.2/README.rst0000644000076700000240000001111300000000000014670 0ustar00zachstaffScruffy ======= .. image:: https://img.shields.io/travis/snare/scruffy.svg :target: https://travis-ci.org/snare/scruffy .. image:: https://img.shields.io/pypi/format/scruffington.svg :target: https://pypi.python.org/pypi/scruffington .. image:: https://readthedocs.org/projects/scruffy/badge/?version=latest :target: http://scruffy.readthedocs.org/en/latest/ *Scruffy. The Janitor.* Scruffy is a framework for taking care of a bunch of boilerplate in Python apps. It handles the loading of configuration files, the loading and management of plugins, and the management of other filesystem resources such as temporary files and directories, log files, etc. A typical use case for Scruffy is a command-line Python tool with some or all of the following requirements: * Read a set of configuration defaults * Read a local configuration file and apply it on top of the defaults * Allow overriding some configuration options with command line flags or at runtime * Load a core set of Python-based plugins * Load a set of user-defined Python-based plugins * Generate log files whose name, location and other logging settings are based on configuration * Store application state between runs in a file or database Scruffy is used by Voltron_ and Calculon_ .. _Voltron: https://github.com/snare/voltron .. _Calculon: https://github.com/snare/calculon Installation ------------ A standard python setup script is included. $ python setup.py install This will install the Scruffy package wherever that happens on your system. Alternately, Scruffy can be installed with `pip` from PyPi (where it's called `scruffington`, because I didn't check for a conflict before I named it). $ pip install scruffington Documentation ------------- Full documentation is hosted at readthedocs_ .. _readthedocs: http://scruffy.readthedocs.io/ Quick start ----------- Config ~~~~~~ Load a user config file, and apply it on top of a set of defaults loaded from inside the Python package we're currently running from. *thingy.yaml*: .. code:: yaml some_property: 1 other_property: a thing *thingy.py*: .. code:: python from scruffy import ConfigFile c = ConfigFile('thingy.yaml', load=True, defaults=File('defaults.yaml', parent=PackageDirectory()) ) print("c.some_property == {c.some_property}".format(c=c)) print("c.other_property == {c.other_property}".format(c=c)) Run it: :: $ python thingy.py c.some_property == 1 c.other_property == a thing Plugins ~~~~~~~ Load some plugins. *~/.thingy/plugins/example.py*: .. code:: python from scruffy import Plugin class ExamplePlugin(Plugin): def do_a_thing(self): print('{}.{} is doing a thing'.format(__name__, self.__class__.__name__)) *thingy.py*: .. code:: python from scruffy import PluginDirectory, PluginRegistry pd = PluginDirectory('~/.thingy/plugins') pd.load() for p in PluginRegistry.plugins: print("Initialising plugin {}".format(p)) p().do_a_thing() Run it: :: $ python thingy.py Initialising plugin example.ExamplePlugin is doing a thing Logging ~~~~~~~ Scruffy's `LogFile` class will do some configuration of Python's `logging` module. *log.py*: .. code:: python import logging from scruffy import LogFile log = logging.getLogger('main') log.setLevel(logging.INFO) LogFile('/tmp/thingy.log', logger='main').configure() log.info('Hello from log.py') */tmp/thingy.log*: :: Hello from log.py Environment ~~~~~~~~~~~ Scruffy's `Environment` class ties all the other stuff together. The other classes can be instantiated as named children of an `Environment`, which will load any `Config` objects, apply the configs to the other objects, and then prepare the other objects. *~/.thingy/config*: .. code:: yaml log_dir: /tmp/logs log_file: thingy.log *env.py*: .. code:: python from scruffy import * e = Environment( main_dir=Directory('~/.thingy', create=True, config=ConfigFile('config', defaults=File('defaults.yaml', parent=PackageDirectory())), lock=LockFile('lock') user_plugins=PluginDirectory('plugins') ), log_dir=Directory('{config:log_dir}', create=True LogFile('{config:log_file}', logger='main') ), pkg_plugins=PluginDirectory('plugins', parent=PackageDirectory()) ) License ------- See LICENSE file. If you use this and don't hate it, buy me a beer at a conference some time. Credits ------- Props to richo_. Flat duck pride. .. _richo: http://github.com/richo././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1607203129.3763692 scruffington-0.3.8.2/scruffington.egg-info/0000755000076700000240000000000000000000000017405 5ustar00zachstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1607203129.0 scruffington-0.3.8.2/scruffington.egg-info/PKG-INFO0000644000076700000240000001446200000000000020511 0ustar00zachstaffMetadata-Version: 2.1 Name: scruffington Version: 0.3.8.2 Summary: The janitor Home-page: https://github.com/snare/scruffy Author: snare Author-email: snare@ho.ax License: MIT Description: Scruffy ======= .. image:: https://img.shields.io/travis/snare/scruffy.svg :target: https://travis-ci.org/snare/scruffy .. image:: https://img.shields.io/pypi/format/scruffington.svg :target: https://pypi.python.org/pypi/scruffington .. image:: https://readthedocs.org/projects/scruffy/badge/?version=latest :target: http://scruffy.readthedocs.org/en/latest/ *Scruffy. The Janitor.* Scruffy is a framework for taking care of a bunch of boilerplate in Python apps. It handles the loading of configuration files, the loading and management of plugins, and the management of other filesystem resources such as temporary files and directories, log files, etc. A typical use case for Scruffy is a command-line Python tool with some or all of the following requirements: * Read a set of configuration defaults * Read a local configuration file and apply it on top of the defaults * Allow overriding some configuration options with command line flags or at runtime * Load a core set of Python-based plugins * Load a set of user-defined Python-based plugins * Generate log files whose name, location and other logging settings are based on configuration * Store application state between runs in a file or database Scruffy is used by Voltron_ and Calculon_ .. _Voltron: https://github.com/snare/voltron .. _Calculon: https://github.com/snare/calculon Installation ------------ A standard python setup script is included. $ python setup.py install This will install the Scruffy package wherever that happens on your system. Alternately, Scruffy can be installed with `pip` from PyPi (where it's called `scruffington`, because I didn't check for a conflict before I named it). $ pip install scruffington Documentation ------------- Full documentation is hosted at readthedocs_ .. _readthedocs: http://scruffy.readthedocs.io/ Quick start ----------- Config ~~~~~~ Load a user config file, and apply it on top of a set of defaults loaded from inside the Python package we're currently running from. *thingy.yaml*: .. code:: yaml some_property: 1 other_property: a thing *thingy.py*: .. code:: python from scruffy import ConfigFile c = ConfigFile('thingy.yaml', load=True, defaults=File('defaults.yaml', parent=PackageDirectory()) ) print("c.some_property == {c.some_property}".format(c=c)) print("c.other_property == {c.other_property}".format(c=c)) Run it: :: $ python thingy.py c.some_property == 1 c.other_property == a thing Plugins ~~~~~~~ Load some plugins. *~/.thingy/plugins/example.py*: .. code:: python from scruffy import Plugin class ExamplePlugin(Plugin): def do_a_thing(self): print('{}.{} is doing a thing'.format(__name__, self.__class__.__name__)) *thingy.py*: .. code:: python from scruffy import PluginDirectory, PluginRegistry pd = PluginDirectory('~/.thingy/plugins') pd.load() for p in PluginRegistry.plugins: print("Initialising plugin {}".format(p)) p().do_a_thing() Run it: :: $ python thingy.py Initialising plugin example.ExamplePlugin is doing a thing Logging ~~~~~~~ Scruffy's `LogFile` class will do some configuration of Python's `logging` module. *log.py*: .. code:: python import logging from scruffy import LogFile log = logging.getLogger('main') log.setLevel(logging.INFO) LogFile('/tmp/thingy.log', logger='main').configure() log.info('Hello from log.py') */tmp/thingy.log*: :: Hello from log.py Environment ~~~~~~~~~~~ Scruffy's `Environment` class ties all the other stuff together. The other classes can be instantiated as named children of an `Environment`, which will load any `Config` objects, apply the configs to the other objects, and then prepare the other objects. *~/.thingy/config*: .. code:: yaml log_dir: /tmp/logs log_file: thingy.log *env.py*: .. code:: python from scruffy import * e = Environment( main_dir=Directory('~/.thingy', create=True, config=ConfigFile('config', defaults=File('defaults.yaml', parent=PackageDirectory())), lock=LockFile('lock') user_plugins=PluginDirectory('plugins') ), log_dir=Directory('{config:log_dir}', create=True LogFile('{config:log_file}', logger='main') ), pkg_plugins=PluginDirectory('plugins', parent=PackageDirectory()) ) License ------- See LICENSE file. If you use this and don't hate it, buy me a beer at a conference some time. Credits ------- Props to richo_. Flat duck pride. .. _richo: http://github.com/richo Keywords: scruffy Platform: UNKNOWN Description-Content-Type: text/x-rst ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1607203129.0 scruffington-0.3.8.2/scruffington.egg-info/SOURCES.txt0000644000076700000240000000047000000000000021272 0ustar00zachstaffREADME.rst setup.cfg setup.py scruffington.egg-info/PKG-INFO scruffington.egg-info/SOURCES.txt scruffington.egg-info/dependency_links.txt scruffington.egg-info/requires.txt scruffington.egg-info/top_level.txt scruffy/__init__.py scruffy/config.py scruffy/env.py scruffy/file.py scruffy/plugin.py scruffy/state.py././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1607203129.0 scruffington-0.3.8.2/scruffington.egg-info/dependency_links.txt0000644000076700000240000000000100000000000023453 0ustar00zachstaff ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1607203129.0 scruffington-0.3.8.2/scruffington.egg-info/requires.txt0000644000076700000240000000001300000000000021777 0ustar00zachstaffpyyaml six ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1607203129.0 scruffington-0.3.8.2/scruffington.egg-info/top_level.txt0000644000076700000240000000001000000000000022126 0ustar00zachstaffscruffy ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1607203129.380972 scruffington-0.3.8.2/scruffy/0000755000076700000240000000000000000000000014665 5ustar00zachstaff././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1563251112.0 scruffington-0.3.8.2/scruffy/__init__.py0000644000076700000240000000107100000000000016775 0ustar00zachstafffrom .env import Environment from .file import File, LogFile, LockFile, Directory, PluginDirectory, PackageDirectory, PackageFile from .plugin import PluginRegistry, Plugin, PluginManager from .config import ConfigNode, Config, ConfigEnv, ConfigFile, ConfigApplicator from .state import State __all__ = [ "Environment", "Directory", "PluginDirectory", "PackageDirectory", "PackageFile", "File", "LogFile", "LockFile", "PluginRegistry", "Plugin", "PluginManager", "ConfigNode", "Config", "ConfigEnv", "ConfigFile", "ConfigApplicator", "State" ] ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1565719770.0 scruffington-0.3.8.2/scruffy/config.py0000644000076700000240000002627400000000000016517 0ustar00zachstaff""" Config ------ Classes for loading and accessing configuration data. """ from six import string_types import copy import os import ast import yaml import re from six import string_types from .file import File class ConfigNode(object): """ Represents a Scruffy config object. Can be accessed as a dictionary, like this: >>> config['top-level-section']['second-level-property'] Or as a dictionary with a key path, like this: >>> config['top_level_section.second_level_property'] Or as an object, like this: >>> config.top_level_section.second_level_property """ def __init__(self, data={}, defaults={}, root=None, path=None): super(ConfigNode, self).__init__() self._root = root if not self._root: self._root = self self._path = path self._defaults = defaults self._data = copy.deepcopy(self._defaults) self.update(data) def __getitem__(self, key): c = self._child(key) v = c._get_value() if type(v) in [dict, list, type(None)]: return c else: return v def __setitem__(self, key, value): container, last = self._child(key)._resolve_path(create=True) container[last] = value def __getattr__(self, key): return self[key] def __setattr__(self, key, value): if key.startswith("_"): super(ConfigNode, self).__setattr__(key, value) else: self[key] = value def __str__(self): return str(self._get_value()) def __repr__(self): return str(self._get_value()) def __int__(self): return int(self._get_value()) def __float__(self): return float(self._get_value()) def __lt__(self, other): return self._get_value() < other def __le__(self, other): return self._get_value() <= other def __eq__(self, other): return self._get_value() == other def __ne__(self, other): return self._get_value() != other def __gt__(self, other): return self._get_value() > other def __ge__(self, other): return self._get_value() >= other def __contains__(self, key): return key in self._get_value() def __nonzero__(self): return self._get_value() != None def __bool__(self): return self._get_value() != None def items(self): return self._get_value().items() def keys(self): return self._get_value().keys() def __iter__(self): return self._get_value().__iter__() def _child(self, path): """ Return a ConfigNode object representing a child node with the specified relative path. """ if self._path: path = '{}.{}'.format(self._path, path) return ConfigNode(root=self._root, path=path) def _resolve_path(self, create=False): """ Returns a tuple of a reference to the last container in the path, and the last component in the key path. For example, with a self._value like this: { 'thing': { 'another': { 'some_leaf': 5, 'one_more': { 'other_leaf': 'x' } } } } And a self._path of: 'thing.another.some_leaf' This will return a tuple of a reference to the 'another' dict, and 'some_leaf', allowing the setter and casting methods to directly access the item referred to by the key path. """ # Split up the key path if isinstance(self._path, string_types): key_path = self._path.split('.') else: key_path = [self._path] # Start at the root node node = self._root._data nodes = [self._root._data] # Traverse along key path while len(key_path): # Get the next key in the key path key = key_path.pop(0) # See if the test could be an int for array access, if so assume it is try: key = int(key) except: pass # If the next level doesn't exist, create it if create: if type(node) == dict and key not in node: node[key] = {} elif type(node) == list and type(key) == int and len(node) < key: node.append([None for i in range(key-len(node))]) # Store the last node and traverse down the hierarchy nodes.append(node) try: node = node[key] except TypeError: if type(key) == int: raise IndexError(key) else: raise KeyError(key) return (nodes[-1], key) def _get_value(self): """ Get the value represented by this node. """ if self._path: try: container, last = self._resolve_path() return container[last] except KeyError: return None except IndexError: return None else: return self._data def update(self, data={}, options={}): """ Update the configuration with new data. This can be passed either or both `data` and `options`. `options` is a dict of keypath/value pairs like this (similar to CherryPy's config mechanism: >>> c.update(options={ ... 'server.port': 8080, ... 'server.host': 'localhost', ... 'admin.email': 'admin@lol' ... }) `data` is a dict of actual config data, like this: >>> c.update(data={ ... 'server': { ... 'port': 8080, ... 'host': 'localhost' ... }, ... 'admin': { ... 'email': 'admin@lol' ... } ... }) """ # Handle an update with a set of options like CherryPy does for key in options: self[key] = options[key] # Merge in any data in `data` if isinstance(data, ConfigNode): data = data._get_value() update_dict(self._get_value(), data) def reset(self): """ Reset the config to defaults. """ self._data = copy.deepcopy(self._defaults) def to_dict(self): """ Generate a plain dictionary. """ return self._get_value() class Config(ConfigNode): """ Config root node class. Just for convenience. """ class ConfigEnv(ConfigNode): """ Config based on based on environment variables. """ def __init__(self, prefix='SCRUFFY', *args, **kwargs): super(ConfigEnv, self).__init__(*args, **kwargs) # build options dictionary from environment variables starting with the prefix options = {} for key in [v for v in os.environ if v.startswith('__SC_') or v.startswith(prefix + '_')]: try: val = ast.literal_eval(os.environ[key]) except: val = os.environ[key] options[key.replace('__SC_', '').replace(prefix + '_', '').lower()] = val # update config with the values we've found self.update(options=options) class ConfigFile(Config, File): """ Config based on a loaded YAML or JSON file. """ def __init__(self, path=None, defaults=None, load=False, apply_env=False, env_prefix='SCRUFFY', *args, **kwargs): self._loaded = False self._defaults_file = defaults self._apply_env = apply_env self._env_prefix = env_prefix Config.__init__(self) File.__init__(self, path=path, *args, **kwargs) if load: self.load() def load(self, reload=False): """ Load the config and defaults from files. """ if reload or not self._loaded: # load defaults if self._defaults_file and isinstance(self._defaults_file, string_types): self._defaults_file = File(self._defaults_file, parent=self._parent) defaults = {} if self._defaults_file: defaults = yaml.safe_load(self._defaults_file.read().replace('\t', ' ')) # load data data = {} if self.exists: data = yaml.safe_load(self.read().replace('\t', ' ')) # initialise with the loaded data self._defaults = defaults self._data = copy.deepcopy(self._defaults) self.update(data=data) # if specified, apply environment variables if self._apply_env: self.update(ConfigEnv(self._env_prefix)) self._loaded = True return self def save(self): """ Save the config back to the config file. """ self.write(yaml.safe_dump(self._data, default_flow_style=False)) def prepare(self): """ Load the file when the Directory/Environment prepares us. """ self.load() class ConfigApplicator(object): """ Applies configs to other objects. """ def __init__(self, config): self.config = config def apply(self, obj): """ Apply the config to an object. """ if isinstance(obj, string_types): return self.apply_to_str(obj) def apply_to_str(self, obj): """ Apply the config to a string. """ toks = re.split('({config:|})', obj) newtoks = [] try: while len(toks): tok = toks.pop(0) if tok == '{config:': # pop the config variable, look it up var = toks.pop(0) val = self.config[var] # if we got an empty node, then it didn't exist if type(val) == ConfigNode and val == None: raise KeyError("No such config variable '{}'".format(var)) # add the value to the list newtoks.append(str(val)) # pop the '}' toks.pop(0) else: # not the start of a config block, just append it to the list newtoks.append(tok) return ''.join(newtoks) except IndexError: pass return obj def update_dict(target, source): """ Recursively merge values from a nested dictionary into another nested dictionary. For example: >>> target = { ... 'thing': 123, ... 'thang': { ... 'a': 1, ... 'b': 2 ... } ... } >>> source = { ... 'thang': { ... 'a': 666, ... 'c': 777 ... } ... } >>> update_dict(target, source) >>> target { 'thing': 123, 'thang': { 'a': 666, 'b': 2, 'c': 777 } } """ for k,v in source.items(): if isinstance(v, dict) and k in target and isinstance(source[k], dict): update_dict(target[k], v) else: target[k] = v ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1563251112.0 scruffington-0.3.8.2/scruffy/env.py0000644000076700000240000000766100000000000016041 0ustar00zachstaff""" Environment ----------- Classes for representing the encompassing environment in which your application runs. """ import os import yaml import itertools import errno import logging import logging.config from six import string_types from .file import Directory from .plugin import PluginManager from .config import ConfigNode, Config, ConfigEnv, ConfigApplicator, ConfigFile class Environment(object): """ An environment in which to run a program """ def __init__(self, setup_logging=True, *args, **kwargs): self._pm = PluginManager() self._children = {} self.config = None # find a config if we have one and load it self.config = self.find_config(kwargs) if self.config: self.config.load() # setup logging if setup_logging: if self.config != None and self.config.logging.dict_config != None: # configure logging from the configuration logging.config.dictConfig(self.config.logging.dict_config.to_dict()) else: # no dict config, set up a basic config so we at least get messages logged to stdout log = logging.getLogger() log.setLevel(logging.INFO) if len(list(filter(lambda h: isinstance(h, logging.StreamHandler), log.handlers))) == 0: log.addHandler(logging.StreamHandler()) # add children self.add(**kwargs) def __enter__(self): return self def __exit__(self, type, value, traceback): self.cleanup() def __getitem__(self, key): return self._children[key] def __getattr__(self, key): return self._children[key] def find_config(self, children): """ Find a config in our children so we can fill in variables in our other children with its data. """ named_config = None found_config = None # first see if we got a kwarg named 'config', as this guy is special if 'config' in children: if isinstance(children['config'], string_types): children['config'] = ConfigFile(children['config']) elif isinstance(children['config'], Config): children['config'] = children['config'] elif type(children['config']) == dict: children['config'] = Config(data=children['config']) else: raise TypeError("Don't know how to turn {} into a Config".format(type(children['config']))) named_config = children['config'] # next check the other kwargs for k in children: if isinstance(children[k], Config): found_config = children[k] # if we still don't have a config, see if there's a directory with one for k in children: if isinstance(children[k], Directory): for j in children[k]._children: if j == 'config' and not named_config: named_config = children[k]._children[j] if isinstance(children[k]._children[j], Config): found_config = children[k]._children[j] if named_config: return named_config else: return found_config def add(self, **kwargs): """ Add objects to the environment. """ for key in kwargs: if isinstance(kwargs[key], string_types): self._children[key] = Directory(kwargs[key]) else: self._children[key] = kwargs[key] self._children[key]._env = self self._children[key].apply_config(ConfigApplicator(self.config)) self._children[key].prepare() def cleanup(self): """ Clean up the environment """ for key in self._children: self._children[key].cleanup() @property def plugins(self): return self._pm.plugins ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1563251112.0 scruffington-0.3.8.2/scruffy/file.py0000644000076700000240000003121300000000000016156 0ustar00zachstaff""" File ---- Classes for representing and performing operations on files and directories. """ from __future__ import unicode_literals import os from six import string_types import yaml import copy import logging import logging.config import inspect import pkg_resources import shutil from .plugin import PluginManager class File(object): """ Represents a file that may or may not exist on the filesystem. Usually encapsulated by a Directory or an Environment. """ def __init__(self, path=None, create=False, cleanup=False, parent=None): super(File, self).__init__() self._parent = parent self._fpath = path self._create = create self._cleanup = cleanup if self._fpath: self._fpath = os.path.expanduser(self._fpath) def __enter__(self): self.prepare() return self def __exit__(self, type, value, traceback): self.cleanup() def __str__(self): return self.path def apply_config(self, applicator): """ Replace any config tokens in the file's path with values from the config. """ if isinstance(self._fpath, string_types): self._fpath = applicator.apply(self._fpath) def create(self): """ Create the file if it doesn't already exist. """ open(self.path, 'a').close() def remove(self): """ Remove the file if it exists. """ if self.exists: os.unlink(self.path) def prepare(self): """ Prepare the file for use in an Environment or Directory. This will create the file if the create flag is set. """ if self._create: self.create() def cleanup(self): """ Clean up the file after use in an Environment or Directory. This will remove the file if the cleanup flag is set. """ if self._cleanup: self.remove() @property def path(self): """ Get the path to the file relative to its parent. """ if self._parent: return os.path.join(self._parent.path, self._fpath) else: return self._fpath @property def name(self): """ Get the file name. """ return os.path.basename(self.path) @property def ext(self): """ Get the file's extension. """ return os.path.splitext(self.path)[1] @property def content(self): """ Property for the content of the file. """ return self.read() @property def exists(self): """ Whether or not the file exists. """ return self.path and os.path.exists(self.path) def read(self): """ Read and return the contents of the file. """ with open(self.path) as f: d = f.read() return d def write(self, data, mode='w'): """ Write data to the file. `data` is the data to write `mode` is the mode argument to pass to `open()` """ with open(self.path, mode) as f: f.write(data) class LogFile(File): """ A log file to configure with Python's logging module. """ def __init__(self, path=None, logger=None, loggers=[], formatter={}, format=None, *args, **kwargs): super(LogFile, self).__init__(path=path, *args, **kwargs) self._create = True self._cleanup = True self._formatter = formatter self._format = format if logger: self._loggers = [logger] else: self._loggers = loggers def prepare(self): """ Configure the log file. """ self.configure() def configure(self): """ Configure the Python logging module for this file. """ # build a file handler for this file handler = logging.FileHandler(self.path, delay=True) # if we got a format string, create a formatter with it if self._format: handler.setFormatter(logging.Formatter(self._format)) # if we got a string for the formatter, assume it's the name of a # formatter in the environment's config if isinstance(self._format, string_types): if self._env and self._env.config.logging.dict_config.formatters[self._formatter]: d = self._env.config.logging.dict_config.formatters[self._formatter].to_dict() handler.setFormatter(logging.Formatter(**d)) elif type(self._formatter) == dict: # if it's a dict it must be the actual formatter params handler.setFormatter(logging.Formatter(**self._formatter)) # add the file handler to whatever loggers were specified if len(self._loggers): for name in self._loggers: logging.getLogger(name).addHandler(handler) else: # none specified, just add it to the root logger logging.getLogger().addHandler(handler) class LockFile(File): """ A file that is automatically created and cleaned up. """ def __init__(self, *args, **kwargs): super(LockFile, self).__init__(*args, **kwargs) self._create = True self._cleanup = True def create(self): """ Create the file. If the file already exists an exception will be raised """ if not os.path.exists(self.path): open(self.path, 'a').close() else: raise Exception("File exists: {}".format(self.path)) class YamlFile(File): """ A yaml file that is parsed into a dictionary. """ @property def content(self): """ Parse the file contents into a dictionary. """ return yaml.safe_load(self.read()) class JsonFile(YamlFile): """ A json file that is parsed into a dictionary. """ class PackageFile(File): """ A file whose path is relative to a Python package. """ def __init__(self, path=None, create=False, cleanup=False, parent=None, package=None): super(PackageFile, self).__init__(path=path, create=create, cleanup=cleanup, parent=PackageDirectory(package=package)) class Directory(object): """ A filesystem directory. A Scruffy Environment usually encompasses a number of these. For example, the main Directory object may represent `~/.myproject`. >>> d = Directory({ ... path='~/.myproject', ... create=True, ... cleanup=False, ... children=[ ... ... ... ] ... }) `path` can be either a string representing the path to the directory, or a nested Directory object. If a Directory object is passed as the `path` its path will be requested instead. This is so Directory objects can be wrapped in others to inherit their properties. """ def __init__(self, path=None, base=None, create=True, cleanup=False, parent=None, **kwargs): self._path = path self._base = base self._create = create self._cleanup = cleanup self._pm = PluginManager() self._children = {} self._env = None self._parent = parent if self._path and isinstance(self._path, string_types): self._path = os.path.expanduser(self._path) self.add(**kwargs) def __enter__(self): self.create() return self def __exit__(self, type, value, traceback): self.cleanup() def __getitem__(self, key): return self._children[key] def __getattr__(self, key): return self._children[key] def apply_config(self, applicator): """ Replace any config tokens with values from the config. """ if isinstance(self._path, string_types): self._path = applicator.apply(self._path) for key in self._children: self._children[key].apply_config(applicator) @property def path(self): """ Return the path to this directory. """ p = '' if self._parent and self._parent.path: p = os.path.join(p, self._parent.path) if self._base: p = os.path.join(p, self._base) if self._path: p = os.path.join(p, self._path) return p def create(self): """ Create the directory. Directory will only be created if the create flag is set. """ if not self.exists: os.mkdir(self.path) def remove(self, recursive=True, ignore_error=True): """ Remove the directory. """ try: if recursive or self._cleanup == 'recursive': shutil.rmtree(self.path) else: os.rmdir(self.path) except Exception as e: if not ignore_error: raise e def prepare(self): """ Prepare the Directory for use in an Environment. This will create the directory if the create flag is set. """ if self._create: self.create() for k in self._children: self._children[k]._env = self._env self._children[k].prepare() def cleanup(self): """ Clean up children and remove the directory. Directory will only be removed if the cleanup flag is set. """ for k in self._children: self._children[k].cleanup() if self._cleanup: self.remove(True) def path_to(self, path): """ Find the path to something inside this directory. """ return os.path.join(self.path, str(path)) @property def exists(self): """ Check if the directory exists. """ return os.path.exists(self.path) def list(self): """ List the contents of the directory. """ return [File(f, parent=self) for f in os.listdir(self.path)] def write(self, filename, data, mode='w'): """ Write to a file in the directory. """ with open(self.path_to(str(filename)), mode) as f: f.write(data) def read(self, filename): """ Read a file from the directory. """ with open(self.path_to(str(filename))) as f: d = f.read() return d def add(self, *args, **kwargs): """ Add objects to the directory. """ for key in kwargs: if isinstance(kwargs[key], string_types): self._children[key] = File(kwargs[key]) else: self._children[key] = kwargs[key] self._children[key]._parent = self self._children[key]._env = self._env added = [] for arg in args: if isinstance(arg, File): self._children[arg.name] = arg self._children[arg.name]._parent = self self._children[arg.name]._env = self._env elif isinstance(arg, string_types): f = File(arg) added.append(f) self._children[arg] = f self._children[arg]._parent = self self._children[arg]._env = self._env else: raise TypeError(type(arg)) # if we were passed a single file/filename, return the File object for convenience if len(added) == 1: return added[0] if len(args) == 1: return args[0] class PluginDirectory(Directory): """ A filesystem directory containing plugins. """ def prepare(self): """ Preparing a plugin directory just loads the plugins. """ super(PluginDirectory, self).prepare() self.load() def load(self): """ Load the plugins in this directory. """ self._pm.load_plugins(self.path) class PackageDirectory(Directory): """ A filesystem directory relative to a Python package. """ def __init__(self, path=None, package=None, *args, **kwargs): super(PackageDirectory, self).__init__(path=path, *args, **kwargs) # if we weren't passed a package name, walk up the stack and find the first non-scruffy package if not package: frame = inspect.currentframe() while frame: if frame.f_globals['__package__'] != 'scruffy': package = frame.f_globals['__package__'] break frame = frame.f_back # if we found a package, set the path directory to the base dir for the package if package: self._base = pkg_resources.resource_filename(package, '') else: raise Exception('No package found') ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1563251112.0 scruffington-0.3.8.2/scruffy/plugin.py0000644000076700000240000000422600000000000016541 0ustar00zachstaff""" Plugin ------ Classes for representing and loading plugins. """ import os import imp import six class PluginRegistry(type): """ Metaclass that registers any classes using it in the `plugins` array """ plugins = [] def __init__(cls, name, bases, attrs): if name != 'Plugin' and cls.__name__ not in map(lambda x: x.__name__, PluginRegistry.plugins): PluginRegistry.plugins.append(cls) @six.add_metaclass(PluginRegistry) class Plugin(object): """ Top-level plugin class, using the PluginRegistry metaclass. All plugin modules must implement a single subclass of this class. This subclass will be the class collected in the PluginRegistry, and should contain references to any other resources required within the module. """ class PluginManager(object): """ Loads plugins which are automatically registered with the PluginRegistry class, and provides an interface to the plugin collection. """ def load_plugins(self, directory): """ Loads plugins from the specified directory. `directory` is the full path to a directory containing python modules which each contain a subclass of the Plugin class. There is no criteria for a valid plugin at this level - any python module found in the directory will be loaded. Only modules that implement a subclass of the Plugin class above will be collected. The directory will be traversed recursively. """ # walk directory for filename in os.listdir(directory): # path to file filepath = os.path.join(directory, filename) # if it's a file, load it modname, ext = os.path.splitext(filename) if os.path.isfile(filepath) and ext == '.py': file, path, descr = imp.find_module(modname, [directory]) if file: mod = imp.load_module(modname, file, path, descr) # if it's a directory, recurse into it if os.path.isdir(filepath): self.load_plugins(filepath) @property def plugins(self): return PluginRegistry.plugins././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1568422359.0 scruffington-0.3.8.2/scruffy/state.py0000644000076700000240000000621200000000000016360 0ustar00zachstaff""" State ----- Classes for storing a program's state. """ import os import atexit import yaml try: from sqlalchemy import create_engine, Column, Integer, String from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker, reconstructor Base = declarative_base() HAVE_SQL_ALCHEMY = True except: HAVE_SQL_ALCHEMY = False class State(object): """ A program's state. Contains a dictionary that can be periodically saved and restored at startup. Maybe later this will be subclassed with database connectors and whatnot, but for now it'll just save to a yaml file. """ @classmethod def state(cls, *args, **kwargs): return cls(*args, **kwargs) def __init__(self, path=None): self.path = path self.d = {} self.load() atexit.register(self._exit_handler) def __enter__(self): self.load() return self def __exit__(self, type, value, traceback): self.save() def __getitem__(self, key): try: return self.d[key] except KeyError: return None def __setitem__(self, key, value): self.d[key] = value def _exit_handler(self): self.save() def save(self): """ Save the state to a file. """ with open(self.path, 'w') as f: f.write(yaml.dump(dict(self.d))) def load(self): """ Load a saved state file. """ if os.path.exists(self.path): with open(self.path, 'r') as f: d = yaml.safe_load(f.read().replace('\t', ' '*4)) # don't clobber self.d if we successfully opened the state file # but it was empty if d: self.d = d def cleanup(self): """ Clean up the saved state. """ if os.path.exists(self.path): os.remove(self.path) if HAVE_SQL_ALCHEMY: class DBState(State, Base): """ State stored in a database, using SQLAlchemy. """ __tablename__ = 'state' id = Column(Integer, primary_key=True) data = Column(String) session = None @classmethod def state(cls, url=None, *args, **kwargs): if not cls.session: engine = create_engine(url, echo=False) Base.metadata.create_all(engine) Session = sessionmaker(bind=engine) cls.session = Session() inst = cls.session.query(DBState).first() if inst: return inst else: return cls(*args, **kwargs) def __init__(self, *args, **kwargs): super(DBState, self).__init__(*args, **kwargs) self.d = {} self.data = '{}' def save(self): self.data = yaml.dump(self.d) self.session.add(self) self.session.commit() @reconstructor def load(self): if self.data: self.d = yaml.safe_load(self.data) def cleanup(self): self.d = {} self.save() ././@PaxHeader0000000000000000000000000000003300000000000010211 xustar0027 mtime=1607203129.382672 scruffington-0.3.8.2/setup.cfg0000644000076700000240000000010300000000000015017 0ustar00zachstaff[bdist_wheel] universal = 1 [egg_info] tag_build = tag_date = 0 ././@PaxHeader0000000000000000000000000000002600000000000010213 xustar0022 mtime=1607202269.0 scruffington-0.3.8.2/setup.py0000755000076700000240000000074200000000000014724 0ustar00zachstafffrom setuptools import setup with open("README.rst", "r") as fp: long_description = fp.read() setup( name="scruffington", version="0.3.8.2", author="snare", author_email="snare@ho.ax", long_description=long_description, long_description_content_type="text/x-rst", description=("The janitor"), license="MIT", keywords="scruffy", url="https://github.com/snare/scruffy", packages=['scruffy'], install_requires=['pyyaml', 'six'], )