plumbum-1.6.0/0000755000232200023220000000000012610225742013546 5ustar debalancedebalanceplumbum-1.6.0/LICENSE0000644000232200023220000000206712153413102014547 0ustar debalancedebalanceCopyright (c) 2013 Tomer Filiba (tomerfiliba@gmail.com) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.plumbum-1.6.0/PKG-INFO0000644000232200023220000001770712610225742014657 0ustar debalancedebalanceMetadata-Version: 1.1 Name: plumbum Version: 1.6.0 Summary: Plumbum: shell combinators library Home-page: http://plumbum.readthedocs.org Author: Tomer Filiba Author-email: tomerfiliba@gmail.com License: MIT Description: .. image:: https://travis-ci.org/tomerfiliba/plumbum.svg?branch=master :target: https://travis-ci.org/tomerfiliba/plumbum Plumbum: Shell Combinators ========================== Ever wished the compactness of shell scripts be put into a **real** programming language? Say hello to *Plumbum Shell Combinators*. Plumbum (Latin for *lead*, which was used to create pipes back in the day) is a small yet feature-rich library for shell script-like programs in Python. The motto of the library is **"Never write shell scripts again"**, and thus it attempts to mimic the **shell syntax** ("shell combinators") where it makes sense, while keeping it all **Pythonic and cross-platform**. Apart from shell-like syntax and handy shortcuts, the library provides local and remote command execution (over SSH), local and remote file-system paths, easy working-directory and environment manipulation, and a programmatic Command-Line Interface (CLI) application toolkit. Now let's see some code! *This is only a teaser; the full documentation can be found at* `Read the Docs `_ Cheat Sheet ----------- Basics ****** .. code-block:: python >>> from plumbum import local >>> ls = local["ls"] >>> ls LocalCommand() >>> ls() u'build.py\ndist\ndocs\nLICENSE\nplumbum\nREADME.rst\nsetup.py\ntests\ntodo.txt\n' >>> notepad = local["c:\\windows\\notepad.exe"] >>> notepad() # Notepad window pops up u'' # Notepad window is closed by user, command returns Instead of writing ``xxx = local["xxx"]`` for every program you wish to use, you can also ``import`` commands .. code-block:: python >>> from plumbum.cmd import grep, wc, cat, head >>> grep LocalCommand() Piping ****** .. code-block:: python >>> chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"] >>> print chain /bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l >>> chain() u'13\n' Redirection *********** .. code-block:: python >>> ((cat < "setup.py") | head["-n", 4])() u'#!/usr/bin/env python\nimport os\n\ntry:\n' >>> (ls["-a"] > "file.list")() u'' >>> (cat["file.list"] | wc["-l"])() u'17\n' Working-directory manipulation ****************************** .. code-block:: python >>> local.cwd >>> with local.cwd(local.cwd / "docs"): ... chain() ... u'15\n' Foreground and background execution *********************************** .. code-block:: python >>> from plumbum import FG, BG >>> (ls["-a"] | grep["\\.py"]) & FG # The output is printed to stdout directly build.py .pydevproject setup.py >>> (ls["-a"] | grep["\\.py"]) & BG # The process runs "in the background" Command nesting *************** .. code-block:: python >>> from plumbum.cmd import sudo >>> print sudo[ifconfig["-a"]] /usr/bin/sudo /sbin/ifconfig -a >>> (sudo[ifconfig["-a"]] | grep["-i", "loop"]) & FG lo Link encap:Local Loopback UP LOOPBACK RUNNING MTU:16436 Metric:1 Remote commands (over SSH) ************************** Supports `openSSH `_-compatible clients, `PuTTY `_ (on Windows) and `Paramiko `_ (a pure-Python implementation of SSH2) .. code-block:: python >>> from plumbum import SshMachine >>> remote = SshMachine("somehost", user = "john", keyfile = "/path/to/idrsa") >>> r_ls = remote["ls"] >>> with remote.cwd("/lib"): ... (r_ls | grep["0.so.0"])() ... u'libusb-1.0.so.0\nlibusb-1.0.so.0.0.0\n' CLI applications **************** .. code-block:: python import logging from plumbum import cli class MyCompiler(cli.Application): verbose = cli.Flag(["-v", "--verbose"], help = "Enable verbose mode") include_dirs = cli.SwitchAttr("-I", list = True, help = "Specify include directories") @cli.switch("--loglevel", int) def set_log_level(self, level): """Sets the log-level of the logger""" logging.root.setLevel(level) def main(self, *srcfiles): print "Verbose:", self.verbose print "Include dirs:", self.include_dirs print "Compiling:", srcfiles if __name__ == "__main__": MyCompiler.run() Sample output +++++++++++++ :: $ python simple_cli.py -v -I foo/bar -Ispam/eggs x.cpp y.cpp z.cpp Verbose: True Include dirs: ['foo/bar', 'spam/eggs'] Compiling: ('x.cpp', 'y.cpp', 'z.cpp') Colors and Styles ----------------- .. code-block:: python from plumbum import colors with colors.red: print("This library provides safe, flexible color access.") print(colors.bold | "(and styles in general)", "are easy!") print("The simple 16 colors or", colors.orchid & colors.underline | '256 named colors,', colors.rgb(18, 146, 64) | "or full rgb colors", 'can be used.') print("Unsafe " + colors.bg.dark_khaki + "color access" + colors.bg.reset + " is available too.") .. image:: https://d2weczhvl823v0.cloudfront.net/tomerfiliba/plumbum/trend.png :alt: Bitdeli badge :target: https://bitdeli.com/free Keywords: path,local,remote,ssh,shell,pipe,popen,process,execution,color,cli Platform: POSIX Platform: Windows Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: Microsoft :: Windows Classifier: Operating System :: POSIX Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.2 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Topic :: Software Development :: Build Tools Classifier: Topic :: System :: Systems Administration Provides: plumbum plumbum-1.6.0/README.rst0000644000232200023220000001254112610225666015245 0ustar debalancedebalance.. image:: https://travis-ci.org/tomerfiliba/plumbum.svg?branch=master :target: https://travis-ci.org/tomerfiliba/plumbum Plumbum: Shell Combinators ========================== Ever wished the compactness of shell scripts be put into a **real** programming language? Say hello to *Plumbum Shell Combinators*. Plumbum (Latin for *lead*, which was used to create pipes back in the day) is a small yet feature-rich library for shell script-like programs in Python. The motto of the library is **"Never write shell scripts again"**, and thus it attempts to mimic the **shell syntax** ("shell combinators") where it makes sense, while keeping it all **Pythonic and cross-platform**. Apart from shell-like syntax and handy shortcuts, the library provides local and remote command execution (over SSH), local and remote file-system paths, easy working-directory and environment manipulation, and a programmatic Command-Line Interface (CLI) application toolkit. Now let's see some code! *This is only a teaser; the full documentation can be found at* `Read the Docs `_ Cheat Sheet ----------- Basics ****** .. code-block:: python >>> from plumbum import local >>> ls = local["ls"] >>> ls LocalCommand() >>> ls() u'build.py\ndist\ndocs\nLICENSE\nplumbum\nREADME.rst\nsetup.py\ntests\ntodo.txt\n' >>> notepad = local["c:\\windows\\notepad.exe"] >>> notepad() # Notepad window pops up u'' # Notepad window is closed by user, command returns Instead of writing ``xxx = local["xxx"]`` for every program you wish to use, you can also ``import`` commands .. code-block:: python >>> from plumbum.cmd import grep, wc, cat, head >>> grep LocalCommand() Piping ****** .. code-block:: python >>> chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"] >>> print chain /bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l >>> chain() u'13\n' Redirection *********** .. code-block:: python >>> ((cat < "setup.py") | head["-n", 4])() u'#!/usr/bin/env python\nimport os\n\ntry:\n' >>> (ls["-a"] > "file.list")() u'' >>> (cat["file.list"] | wc["-l"])() u'17\n' Working-directory manipulation ****************************** .. code-block:: python >>> local.cwd >>> with local.cwd(local.cwd / "docs"): ... chain() ... u'15\n' Foreground and background execution *********************************** .. code-block:: python >>> from plumbum import FG, BG >>> (ls["-a"] | grep["\\.py"]) & FG # The output is printed to stdout directly build.py .pydevproject setup.py >>> (ls["-a"] | grep["\\.py"]) & BG # The process runs "in the background" Command nesting *************** .. code-block:: python >>> from plumbum.cmd import sudo >>> print sudo[ifconfig["-a"]] /usr/bin/sudo /sbin/ifconfig -a >>> (sudo[ifconfig["-a"]] | grep["-i", "loop"]) & FG lo Link encap:Local Loopback UP LOOPBACK RUNNING MTU:16436 Metric:1 Remote commands (over SSH) ************************** Supports `openSSH `_-compatible clients, `PuTTY `_ (on Windows) and `Paramiko `_ (a pure-Python implementation of SSH2) .. code-block:: python >>> from plumbum import SshMachine >>> remote = SshMachine("somehost", user = "john", keyfile = "/path/to/idrsa") >>> r_ls = remote["ls"] >>> with remote.cwd("/lib"): ... (r_ls | grep["0.so.0"])() ... u'libusb-1.0.so.0\nlibusb-1.0.so.0.0.0\n' CLI applications **************** .. code-block:: python import logging from plumbum import cli class MyCompiler(cli.Application): verbose = cli.Flag(["-v", "--verbose"], help = "Enable verbose mode") include_dirs = cli.SwitchAttr("-I", list = True, help = "Specify include directories") @cli.switch("--loglevel", int) def set_log_level(self, level): """Sets the log-level of the logger""" logging.root.setLevel(level) def main(self, *srcfiles): print "Verbose:", self.verbose print "Include dirs:", self.include_dirs print "Compiling:", srcfiles if __name__ == "__main__": MyCompiler.run() Sample output +++++++++++++ :: $ python simple_cli.py -v -I foo/bar -Ispam/eggs x.cpp y.cpp z.cpp Verbose: True Include dirs: ['foo/bar', 'spam/eggs'] Compiling: ('x.cpp', 'y.cpp', 'z.cpp') Colors and Styles ----------------- .. code-block:: python from plumbum import colors with colors.red: print("This library provides safe, flexible color access.") print(colors.bold | "(and styles in general)", "are easy!") print("The simple 16 colors or", colors.orchid & colors.underline | '256 named colors,', colors.rgb(18, 146, 64) | "or full rgb colors", 'can be used.') print("Unsafe " + colors.bg.dark_khaki + "color access" + colors.bg.reset + " is available too.") .. image:: https://d2weczhvl823v0.cloudfront.net/tomerfiliba/plumbum/trend.png :alt: Bitdeli badge :target: https://bitdeli.com/free plumbum-1.6.0/setup.cfg0000644000232200023220000000016312610225742015367 0ustar debalancedebalance[nosetests] verbosity = 2 detailed-errors = 1 [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 plumbum-1.6.0/setup.py0000644000232200023220000000364212610225666015272 0ustar debalancedebalance#!/usr/bin/env python import os try: from setuptools import setup, Command except ImportError: from distutils.core import setup, Command HERE = os.path.dirname(__file__) exec(open(os.path.join(HERE, "plumbum", "version.py")).read()) class PyDocs(Command): user_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): import subprocess import sys os.chdir('docs') errno = subprocess.call(['make', 'html']) sys.exit(errno) setup(name = "plumbum", version = version_string, # @UndefinedVariable description = "Plumbum: shell combinators library", author = "Tomer Filiba", author_email = "tomerfiliba@gmail.com", license = "MIT", url = "http://plumbum.readthedocs.org", packages = ["plumbum", "plumbum.cli", "plumbum.commands", "plumbum.machines", "plumbum.path", "plumbum.fs", "plumbum.colorlib"], platforms = ["POSIX", "Windows"], provides = ["plumbum"], keywords = "path, local, remote, ssh, shell, pipe, popen, process, execution, color, cli", cmdclass = { 'docs':PyDocs}, # use_2to3 = False, # zip_safe = True, long_description = open(os.path.join(HERE, "README.rst"), "r").read(), classifiers = [ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.2", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Software Development :: Build Tools", "Topic :: System :: Systems Administration", ], ) plumbum-1.6.0/plumbum.egg-info/0000755000232200023220000000000012610225742016721 5ustar debalancedebalanceplumbum-1.6.0/plumbum.egg-info/PKG-INFO0000644000232200023220000001770712610225742020032 0ustar debalancedebalanceMetadata-Version: 1.1 Name: plumbum Version: 1.6.0 Summary: Plumbum: shell combinators library Home-page: http://plumbum.readthedocs.org Author: Tomer Filiba Author-email: tomerfiliba@gmail.com License: MIT Description: .. image:: https://travis-ci.org/tomerfiliba/plumbum.svg?branch=master :target: https://travis-ci.org/tomerfiliba/plumbum Plumbum: Shell Combinators ========================== Ever wished the compactness of shell scripts be put into a **real** programming language? Say hello to *Plumbum Shell Combinators*. Plumbum (Latin for *lead*, which was used to create pipes back in the day) is a small yet feature-rich library for shell script-like programs in Python. The motto of the library is **"Never write shell scripts again"**, and thus it attempts to mimic the **shell syntax** ("shell combinators") where it makes sense, while keeping it all **Pythonic and cross-platform**. Apart from shell-like syntax and handy shortcuts, the library provides local and remote command execution (over SSH), local and remote file-system paths, easy working-directory and environment manipulation, and a programmatic Command-Line Interface (CLI) application toolkit. Now let's see some code! *This is only a teaser; the full documentation can be found at* `Read the Docs `_ Cheat Sheet ----------- Basics ****** .. code-block:: python >>> from plumbum import local >>> ls = local["ls"] >>> ls LocalCommand() >>> ls() u'build.py\ndist\ndocs\nLICENSE\nplumbum\nREADME.rst\nsetup.py\ntests\ntodo.txt\n' >>> notepad = local["c:\\windows\\notepad.exe"] >>> notepad() # Notepad window pops up u'' # Notepad window is closed by user, command returns Instead of writing ``xxx = local["xxx"]`` for every program you wish to use, you can also ``import`` commands .. code-block:: python >>> from plumbum.cmd import grep, wc, cat, head >>> grep LocalCommand() Piping ****** .. code-block:: python >>> chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"] >>> print chain /bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l >>> chain() u'13\n' Redirection *********** .. code-block:: python >>> ((cat < "setup.py") | head["-n", 4])() u'#!/usr/bin/env python\nimport os\n\ntry:\n' >>> (ls["-a"] > "file.list")() u'' >>> (cat["file.list"] | wc["-l"])() u'17\n' Working-directory manipulation ****************************** .. code-block:: python >>> local.cwd >>> with local.cwd(local.cwd / "docs"): ... chain() ... u'15\n' Foreground and background execution *********************************** .. code-block:: python >>> from plumbum import FG, BG >>> (ls["-a"] | grep["\\.py"]) & FG # The output is printed to stdout directly build.py .pydevproject setup.py >>> (ls["-a"] | grep["\\.py"]) & BG # The process runs "in the background" Command nesting *************** .. code-block:: python >>> from plumbum.cmd import sudo >>> print sudo[ifconfig["-a"]] /usr/bin/sudo /sbin/ifconfig -a >>> (sudo[ifconfig["-a"]] | grep["-i", "loop"]) & FG lo Link encap:Local Loopback UP LOOPBACK RUNNING MTU:16436 Metric:1 Remote commands (over SSH) ************************** Supports `openSSH `_-compatible clients, `PuTTY `_ (on Windows) and `Paramiko `_ (a pure-Python implementation of SSH2) .. code-block:: python >>> from plumbum import SshMachine >>> remote = SshMachine("somehost", user = "john", keyfile = "/path/to/idrsa") >>> r_ls = remote["ls"] >>> with remote.cwd("/lib"): ... (r_ls | grep["0.so.0"])() ... u'libusb-1.0.so.0\nlibusb-1.0.so.0.0.0\n' CLI applications **************** .. code-block:: python import logging from plumbum import cli class MyCompiler(cli.Application): verbose = cli.Flag(["-v", "--verbose"], help = "Enable verbose mode") include_dirs = cli.SwitchAttr("-I", list = True, help = "Specify include directories") @cli.switch("--loglevel", int) def set_log_level(self, level): """Sets the log-level of the logger""" logging.root.setLevel(level) def main(self, *srcfiles): print "Verbose:", self.verbose print "Include dirs:", self.include_dirs print "Compiling:", srcfiles if __name__ == "__main__": MyCompiler.run() Sample output +++++++++++++ :: $ python simple_cli.py -v -I foo/bar -Ispam/eggs x.cpp y.cpp z.cpp Verbose: True Include dirs: ['foo/bar', 'spam/eggs'] Compiling: ('x.cpp', 'y.cpp', 'z.cpp') Colors and Styles ----------------- .. code-block:: python from plumbum import colors with colors.red: print("This library provides safe, flexible color access.") print(colors.bold | "(and styles in general)", "are easy!") print("The simple 16 colors or", colors.orchid & colors.underline | '256 named colors,', colors.rgb(18, 146, 64) | "or full rgb colors", 'can be used.') print("Unsafe " + colors.bg.dark_khaki + "color access" + colors.bg.reset + " is available too.") .. image:: https://d2weczhvl823v0.cloudfront.net/tomerfiliba/plumbum/trend.png :alt: Bitdeli badge :target: https://bitdeli.com/free Keywords: path,local,remote,ssh,shell,pipe,popen,process,execution,color,cli Platform: POSIX Platform: Windows Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: Microsoft :: Windows Classifier: Operating System :: POSIX Classifier: Programming Language :: Python :: 2.6 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.2 Classifier: Programming Language :: Python :: 3.3 Classifier: Programming Language :: Python :: 3.4 Classifier: Programming Language :: Python :: 3.5 Classifier: Topic :: Software Development :: Build Tools Classifier: Topic :: System :: Systems Administration Provides: plumbum plumbum-1.6.0/plumbum.egg-info/dependency_links.txt0000644000232200023220000000000112610225742022767 0ustar debalancedebalance plumbum-1.6.0/plumbum.egg-info/SOURCES.txt0000644000232200023220000000221512610225742020605 0ustar debalancedebalanceLICENSE MANIFEST.in README.rst setup.cfg setup.py plumbum/__init__.py plumbum/_testtools.py plumbum/colors.py plumbum/lib.py plumbum/version.py plumbum.egg-info/PKG-INFO plumbum.egg-info/SOURCES.txt plumbum.egg-info/dependency_links.txt plumbum.egg-info/top_level.txt plumbum/cli/__init__.py plumbum/cli/application.py plumbum/cli/progress.py plumbum/cli/switches.py plumbum/cli/terminal.py plumbum/cli/termsize.py plumbum/colorlib/__init__.py plumbum/colorlib/__main__.py plumbum/colorlib/_ipython_ext.py plumbum/colorlib/factories.py plumbum/colorlib/names.py plumbum/colorlib/styles.py plumbum/commands/__init__.py plumbum/commands/base.py plumbum/commands/daemons.py plumbum/commands/modifiers.py plumbum/commands/processes.py plumbum/fs/__init__.py plumbum/fs/atomic.py plumbum/fs/mounts.py plumbum/machines/__init__.py plumbum/machines/_windows.py plumbum/machines/base.py plumbum/machines/env.py plumbum/machines/local.py plumbum/machines/paramiko_machine.py plumbum/machines/remote.py plumbum/machines/session.py plumbum/machines/ssh_machine.py plumbum/path/__init__.py plumbum/path/base.py plumbum/path/local.py plumbum/path/remote.py plumbum/path/utils.pyplumbum-1.6.0/plumbum.egg-info/top_level.txt0000644000232200023220000000001012610225742021442 0ustar debalancedebalanceplumbum plumbum-1.6.0/MANIFEST.in0000644000232200023220000000004312064300512015271 0ustar debalancedebalanceinclude LICENSE include README.rst plumbum-1.6.0/plumbum/0000755000232200023220000000000012610225742015227 5ustar debalancedebalanceplumbum-1.6.0/plumbum/__init__.py0000644000232200023220000000466512610225666017360 0ustar debalancedebalancer""" Plumbum Shell Combinators ------------------------- A wrist-handy library for writing shell-like scripts in Python, that can serve as a ``Popen`` replacement, and much more:: >>> from plumbum.cmd import ls, grep, wc, cat >>> ls() u'build.py\ndist\ndocs\nLICENSE\nplumbum\nREADME.rst\nsetup.py\ntests\ntodo.txt\n' >>> chain = ls["-a"] | grep["-v", "py"] | wc["-l"] >>> print chain /bin/ls -a | /bin/grep -v py | /usr/bin/wc -l >>> chain() u'12\n' >>> ((ls["-a"] | grep["-v", "py"]) > "/tmp/foo.txt")() u'' >>> ((cat < "/tmp/foo.txt") | wc["-l"])() u'12\n' >>> from plumbum import local, FG, BG >>> with local.cwd("/tmp"): ... (ls | wc["-l"]) & FG ... 13 # printed directly to the interpreter's stdout >>> (ls | wc["-l"]) & BG >>> f=_ >>> f.stdout # will wait for the process to terminate u'9\n' Plumbum includes local/remote path abstraction, working directory and environment manipulation, process execution, remote process execution over SSH, tunneling, SCP-based upload/download, and a {arg|opt}parse replacement for the easy creation of command-line interface (CLI) programs. See http://plumbum.readthedocs.org for full details """ from plumbum.commands import ProcessExecutionError, CommandNotFound, ProcessTimedOut from plumbum.commands import FG, BG, TF, RETCODE, ERROUT, NOHUP from plumbum.path import Path, LocalPath, RemotePath from plumbum.machines import local, BaseRemoteMachine, SshMachine, PuttyMachine from plumbum.version import version __author__ = "Tomer Filiba (tomerfiliba@gmail.com)" __version__ = version #=================================================================================================== # Module hack: ``from plumbum.cmd import ls`` #=================================================================================================== import sys from types import ModuleType class LocalModule(ModuleType): """The module-hack that allows us to use ``from plumbum.cmd import some_program``""" __all__ = () # to make help() happy __package__ = __name__ def __getattr__(self, name): try: return local[name] except CommandNotFound: raise AttributeError(name) __path__ = [] __file__ = __file__ cmd = LocalModule(__name__ + ".cmd", LocalModule.__doc__) sys.modules[cmd.__name__] = cmd del sys del ModuleType del LocalModule plumbum-1.6.0/plumbum/path/0000755000232200023220000000000012610225742016163 5ustar debalancedebalanceplumbum-1.6.0/plumbum/path/__init__.py0000644000232200023220000000033412230041362020264 0ustar debalancedebalancefrom plumbum.path.local import LocalPath, LocalWorkdir from plumbum.path.remote import RemotePath, RemoteWorkdir from plumbum.path.base import Path, FSUser, RelativePath from plumbum.path.utils import copy, move, delete plumbum-1.6.0/plumbum/path/local.py0000644000232200023220000002436112610225666017642 0ustar debalancedebalanceimport os import sys import glob import shutil import errno import logging from contextlib import contextmanager from plumbum.lib import _setdoc, IS_WIN32 from plumbum.path.base import Path, FSUser from plumbum.path.remote import RemotePath try: from pwd import getpwuid, getpwnam from grp import getgrgid, getgrnam except ImportError: def getpwuid(x): return (None,) def getgrgid(x): return (None,) def getpwnam(x): raise OSError("`getpwnam` not supported") def getgrnam(x): raise OSError("`getgrnam` not supported") try: # Py3 import urllib.parse as urlparse import urllib.request as urllib except ImportError: import urlparse import urllib logger = logging.getLogger("plumbum.local") #=================================================================================================== # Local Paths #=================================================================================================== class LocalPath(Path): """The class implementing local-machine paths""" CASE_SENSITIVE = not IS_WIN32 def __new__(cls, *parts): if len(parts) == 1 and \ isinstance(parts[0], cls) and \ not isinstance(parts[0], LocalWorkdir): return parts[0] if not parts: raise TypeError("At least one path part is require (none given)") if any(isinstance(path, RemotePath) for path in parts): raise TypeError("LocalPath cannot be constructed from %r" % (parts,)) self = super(LocalPath, cls).__new__(cls, os.path.normpath(os.path.join(*(str(p) for p in parts)))) return self @property def _path(self): return str(self) def _get_info(self): return self._path def __getstate__(self): return {"_path" : self._path} def _form(self, *parts): return LocalPath(*parts) @property @_setdoc(Path) def name(self): return os.path.basename(str(self)) @property @_setdoc(Path) def dirname(self): return LocalPath(os.path.dirname(str(self))) @property @_setdoc(Path) def suffix(self): return os.path.splitext(str(self))[1] @property def suffixes(self): exts = [] base = str(self) while True: base, ext = os.path.splitext(base) if ext: exts.append(ext) else: return list(reversed(exts)) @property @_setdoc(Path) def uid(self): uid = self.stat().st_uid name = getpwuid(uid)[0] return FSUser(uid, name) @property @_setdoc(Path) def gid(self): gid = self.stat().st_gid name = getgrgid(gid)[0] return FSUser(gid, name) @_setdoc(Path) def join(self, *others): return LocalPath(self, *others) @_setdoc(Path) def list(self): return [self / fn for fn in os.listdir(str(self))] @_setdoc(Path) def iterdir(self): try: return (self.__class__(fn.name) for fn in os.scandir(str(self))) except NameError: return (self / fn for fn in os.listdir(str(self))) @_setdoc(Path) def is_dir(self): return os.path.isdir(str(self)) @_setdoc(Path) def is_file(self): return os.path.isfile(str(self)) @_setdoc(Path) def is_symlink(self): return os.path.islink(str(self)) @_setdoc(Path) def exists(self): return os.path.exists(str(self)) @_setdoc(Path) def stat(self): return os.stat(str(self)) @_setdoc(Path) def with_name(self, name): return LocalPath(self.dirname) / name @property @_setdoc(Path) def stem(self): return self.name.rsplit(os.path.extsep)[0] @_setdoc(Path) def with_suffix(self, suffix, depth=1): if (suffix and not suffix.startswith(os.path.extsep) or suffix == os.path.extsep): raise ValueError("Invalid suffix %r" % (suffix)) name = self.name depth = len(self.suffixes) if depth is None else min(depth, len(self.suffixes)) for i in range(depth): name, ext = os.path.splitext(name) return LocalPath(self.dirname) / (name + suffix) @_setdoc(Path) def glob(self, pattern): fn = lambda pat: [LocalPath(m) for m in glob.glob(str(self / pat))] return self._glob(pattern, fn) @_setdoc(Path) def delete(self): if not self.exists(): return if self.is_dir(): shutil.rmtree(str(self)) else: try: os.remove(str(self)) except OSError: # file might already been removed (a race with other threads/processes) _, ex, _ = sys.exc_info() if ex.errno != errno.ENOENT: raise @_setdoc(Path) def move(self, dst): if isinstance(dst, RemotePath): raise TypeError("Cannot move local path %s to %r" % (self, dst)) shutil.move(str(self), str(dst)) return LocalPath(dst) @_setdoc(Path) def copy(self, dst, override = False): if isinstance(dst, RemotePath): raise TypeError("Cannot copy local path %s to %r" % (self, dst)) dst = LocalPath(dst) if override: dst.delete() if self.is_dir(): shutil.copytree(str(self), str(dst)) else: dst_dir = LocalPath(dst).dirname if not dst_dir.exists(): dst_dir.mkdir() shutil.copy2(str(self), str(dst)) return dst @_setdoc(Path) def mkdir(self): if not self.exists(): try: os.makedirs(str(self)) except OSError: # directory might already exist (a race with other threads/processes) _, ex, _ = sys.exc_info() if ex.errno != errno.EEXIST: raise @_setdoc(Path) def open(self, mode = "rb"): return open(str(self), mode) @_setdoc(Path) def read(self, encoding=None): with self.open("rb") as f: data = f.read() if encoding: data = data.decode(encoding) return data @_setdoc(Path) def write(self, data, encoding=None): if encoding: data = data.encode(encoding) with self.open("wb") as f: f.write(data) @_setdoc(Path) def chown(self, owner = None, group = None, recursive = None): if not hasattr(os, "chown"): raise OSError("os.chown() not supported") uid = self.uid if owner is None else (owner if isinstance(owner, int) else getpwnam(owner)[2]) gid = self.gid if group is None else (group if isinstance(group, int) else getgrnam(group)[2]) os.chown(str(self), uid, gid) if recursive or (recursive is None and self.is_dir()): for subpath in self.walk(): os.chown(str(subpath), uid, gid) @_setdoc(Path) def chmod(self, mode): if not hasattr(os, "chmod"): raise OSError("os.chmod() not supported") os.chmod(str(self), mode) @_setdoc(Path) def access(self, mode = 0): return os.access(str(self), self._access_mode_to_flags(mode)) @_setdoc(Path) def link(self, dst): if isinstance(dst, RemotePath): raise TypeError("Cannot create a hardlink from local path %s to %r" % (self, dst)) if hasattr(os, "link"): os.link(str(self), str(dst)) else: from plumbum.machines.local import local # windows: use mklink if self.is_dir(): local["cmd"]("/C", "mklink", "/D", "/H", str(dst), str(self)) else: local["cmd"]("/C", "mklink", "/H", str(dst), str(self)) @_setdoc(Path) def symlink(self, dst): if isinstance(dst, RemotePath): raise TypeError("Cannot create a symlink from local path %s to %r" % (self, dst)) if hasattr(os, "symlink"): os.symlink(str(self), str(dst)) else: from plumbum.machines.local import local # windows: use mklink if self.is_dir(): local["cmd"]("/C", "mklink", "/D", str(dst), str(self)) else: local["cmd"]("/C", "mklink", str(dst), str(self)) @_setdoc(Path) def unlink(self): try: if hasattr(os, "symlink") or not self.is_dir(): os.unlink(str(self)) else: # windows: use rmdir for directories and directory symlinks os.rmdir(str(self)) except OSError: # file might already been removed (a race with other threads/processes) _, ex, _ = sys.exc_info() if ex.errno != errno.ENOENT: raise @_setdoc(Path) def as_uri(self, scheme='file'): return urlparse.urljoin(str(scheme)+':', urllib.pathname2url(str(self))) @property @_setdoc(Path) def drive(self): return os.path.splitdrive(str(self))[0] @property @_setdoc(Path) def root(self): return os.path.sep class LocalWorkdir(LocalPath): """Working directory manipulator""" def __hash__(self): raise TypeError("unhashable type") def __new__(cls): return super(LocalWorkdir, cls).__new__(cls, os.getcwd()) def chdir(self, newdir): """Changes the current working directory to the given one :param newdir: The destination director (a string or a ``LocalPath``) """ if isinstance(newdir, RemotePath): raise TypeError("newdir cannot be %r" % (newdir,)) logger.debug("Chdir to %s", newdir) os.chdir(str(newdir)) return self.__class__() def getpath(self): """Returns the current working directory as a ``LocalPath`` object""" return LocalPath(self._path) @contextmanager def __call__(self, newdir): """A context manager used to ``chdir`` into a directory and then ``chdir`` back to the previous location; much like ``pushd``/``popd``. :param newdir: The destination director (a string or a ``LocalPath``) """ prev = self._path newdir = self.chdir(newdir) try: yield newdir finally: self.chdir(prev) plumbum-1.6.0/plumbum/path/utils.py0000644000232200023220000000636312610225666017712 0ustar debalancedebalancefrom plumbum.path.base import Path from plumbum.lib import six from plumbum.machines.local import local, LocalPath def delete(*paths): """Deletes the given paths. The arguments can be either strings, :class:`local paths `, :class:`remote paths `, or iterables of such. No error is raised if any of the paths does not exist (it is silently ignored) """ for p in paths: if isinstance(p, Path): p.delete() elif isinstance(p, six.string_types): local.path(p).delete() elif hasattr(p, "__iter__"): delete(*p) else: raise TypeError("Cannot delete %r" % (p,)) def _move(src, dst): ret = copy(src, dst) delete(src) return ret def move(src, dst): """Moves the source path onto the destination path; ``src`` and ``dst`` can be either strings, :class:`LocalPaths ` or :class:`RemotePath `; any combination of the three will work. .. versionadded:: 1.3 ``src`` can also be a list of strings/paths, in which case ``dst`` must not exist or be a directory. """ if not isinstance(dst, Path): dst = local.path(dst) if isinstance(src, (tuple, list)): if not dst.exists(): dst.mkdir() elif not dst.is_dir(): raise ValueError("When using multiple sources, dst %r must be a directory" % (dst,)) for src2 in src: move(src2, dst) return dst elif not isinstance(src, Path): src = local.path(src) if isinstance(src, LocalPath): if isinstance(dst, LocalPath): return src.move(dst) else: return _move(src, dst) elif isinstance(dst, LocalPath): return _move(src, dst) elif src.remote == dst.remote: return src.move(dst) else: return _move(src, dst) def copy(src, dst): """ Copy (recursively) the source path onto the destination path; ``src`` and ``dst`` can be either strings, :class:`LocalPaths ` or :class:`RemotePath `; any combination of the three will work. .. versionadded:: 1.3 ``src`` can also be a list of strings/paths, in which case ``dst`` must not exist or be a directory. """ if not isinstance(dst, Path): dst = local.path(dst) if isinstance(src, (tuple, list)): if not dst.exists(): dst.mkdir() elif not dst.is_dir(): raise ValueError("When using multiple sources, dst %r must be a directory" % (dst,)) for src2 in src: copy(src2, dst) return dst elif not isinstance(src, Path): src = local.path(src) if isinstance(src, LocalPath): if isinstance(dst, LocalPath): return src.copy(dst) else: dst.remote.upload(src, dst) return dst elif isinstance(dst, LocalPath): src.remote.download(src, dst) return dst elif src.remote == dst.remote: return src.copy(dst) else: with local.tempdir() as tmp: copy(src, tmp) copy(tmp / src.name, dst) return dst plumbum-1.6.0/plumbum/path/base.py0000644000232200023220000003330112610225666017454 0ustar debalancedebalancefrom __future__ import absolute_import import itertools import operator import os from plumbum.lib import six from abc import abstractmethod, abstractproperty import warnings from functools import reduce class FSUser(int): """A special object that represents a file-system user. It derives from ``int``, so it behaves just like a number (``uid``/``gid``), but also have a ``.name`` attribute that holds the string-name of the user, if given (otherwise ``None``) """ def __new__(cls, val, name = None): self = int.__new__(cls, val) self.name = name return self class Path(str, six.ABC): """An abstraction over file system paths. This class is abstract, and the two implementations are :class:`LocalPath ` and :class:`RemotePath `. """ CASE_SENSITIVE = True def __repr__(self): return "<%s %s>" % (self.__class__.__name__, str(self)) def __div__(self, other): """Joins two paths""" return self.join(other) __truediv__ = __div__ def __floordiv__(self, expr): """Returns a (possibly empty) list of paths that matched the glob-pattern under this path""" return self.glob(expr) def __iter__(self): """Iterate over the files in this directory""" return iter(self.list()) def __eq__(self, other): if isinstance(other, Path): return self._get_info() == other._get_info() elif isinstance(other, str): if self.CASE_SENSITIVE: return str(self) == other else: return str(self).lower() == other.lower() else: return NotImplemented def __ne__(self, other): return not (self == other) def __gt__(self, other): return str(self) > str(other) def __ge__(self, other): return str(self) >= str(other) def __lt__(self, other): return str(self) < str(other) def __le__(self, other): return str(self) <= str(other) def __hash__(self): if self.CASE_SENSITIVE: return hash(str(self)) else: return hash(str(self).lower()) def __nonzero__(self): return bool(str(self)) __bool__ = __nonzero__ @abstractmethod def _form(self, *parts): pass def up(self, count = 1): """Go up in ``count`` directories (the default is 1)""" return self.join("../" * count) def walk(self, filter = lambda p: True, dir_filter = lambda p: True): # @ReservedAssignment """traverse all (recursive) sub-elements under this directory, that match the given filter. By default, the filter accepts everything; you can provide a custom filter function that takes a path as an argument and returns a boolean :param filter: the filter (predicate function) for matching results. Only paths matching this predicate are returned. Defaults to everything. :param dir_filter: the filter (predicate function) for matching directories. Only directories matching this predicate are recursed into. Defaults to everything. """ for p in self.list(): if filter(p): yield p if p.is_dir() and dir_filter(p): for p2 in p.walk(filter, dir_filter): yield p2 @abstractproperty def name(self): """The basename component of this path""" @property def basename(self): """Included for compatibility with older Plumbum code""" warnings.warn("Use .name instead", DeprecationWarning) return self.name @abstractproperty def stem(self): """The name without an extension, or the last component of the path""" @abstractproperty def dirname(self): """The dirname component of this path""" @abstractproperty def root(self): """The root of the file tree (`/` on Unix)""" @abstractproperty def drive(self): """The drive letter (on Windows)""" @abstractproperty def suffix(self): """The suffix of this file""" @abstractproperty def suffixes(self): """This is a list of all suffixes""" @abstractproperty def uid(self): """The user that owns this path. The returned value is a :class:`FSUser ` object which behaves like an ``int`` (as expected from ``uid``), but it also has a ``.name`` attribute that holds the string-name of the user""" @abstractproperty def gid(self): """The group that owns this path. The returned value is a :class:`FSUser ` object which behaves like an ``int`` (as expected from ``gid``), but it also has a ``.name`` attribute that holds the string-name of the group""" @abstractmethod def as_uri(self, scheme=None): """Returns a universal resource identifier. Use ``scheme`` to force a scheme.""" @abstractmethod def _get_info(self): pass @abstractmethod def join(self, *parts): """Joins this path with any number of paths""" @abstractmethod def list(self): """Returns the files in this directory""" @abstractmethod def iterdir(self): """Returns an iterator over the directory. Might be slightly faster on Python 3.5 than .list()""" @abstractmethod def is_dir(self): """Returns ``True`` if this path is a directory, ``False`` otherwise""" def isdir(self): """Included for compatibility with older Plumbum code""" warnings.warn("Use .is_dir() instead", DeprecationWarning) return self.is_dir() @abstractmethod def is_file(self): """Returns ``True`` if this path is a regular file, ``False`` otherwise""" def isfile(self): """Included for compatibility with older Plumbum code""" warnings.warn("Use .is_file() instead", DeprecationWarning) return self.is_file() def islink(self): """Included for compatibility with older Plumbum code""" warnings.warn("Use is_symlink instead", DeprecationWarning) return self.is_symlink() @abstractmethod def is_symlink(self): """Returns ``True`` if this path is a symbolic link, ``False`` otherwise""" @abstractmethod def exists(self): """Returns ``True`` if this path exists, ``False`` otherwise""" @abstractmethod def stat(self): """Returns the os.stats for a file""" pass @abstractmethod def with_name(self, name): """Returns a path with the name replaced""" @abstractmethod def with_suffix(self, suffix, depth=1): """Returns a path with the suffix replaced. Up to last ``depth`` suffixes will be replaces. None will replace all suffixes. If there are less than ``depth`` suffixes, this will replace all suffixes. ``.tar.gz`` is an example where ``depth=2`` or ``depth=None`` is useful""" def preferred_suffix(self, suffix): """Adds a suffix if one does not currently exist (otherwise, no change). Useful for loading files with a default suffix""" if len(self.suffixes) > 0: return self else: return self.with_suffix(suffix) @abstractmethod def glob(self, pattern): """Returns a (possibly empty) list of paths that matched the glob-pattern under this path""" @abstractmethod def delete(self): """Deletes this path (recursively, if a directory)""" @abstractmethod def move(self, dst): """Moves this path to a different location""" def rename(self, newname): """Renames this path to the ``new name`` (only the basename is changed)""" return self.move(self.up() / newname) @abstractmethod def copy(self, dst, override = False): """Copies this path (recursively, if a directory) to the destination path""" @abstractmethod def mkdir(self): """Creates a directory at this path; if the directory already exists, silently ignore""" @abstractmethod def open(self, mode = "r"): """opens this path as a file""" @abstractmethod def read(self, encoding=None): """returns the contents of this file. By default the data is binary (``bytes``), but you can specify the encoding, e.g., ``'latin1'`` or ``'utf8'``""" @abstractmethod def write(self, data, encoding=None): """writes the given data to this file. By default the data is expected to be binary (``bytes``), but you can specify the encoding, e.g., ``'latin1'`` or ``'utf8'``""" @abstractmethod def chown(self, owner = None, group = None, recursive = None): """Change ownership of this path. :param owner: The owner to set (either ``uid`` or ``username``), optional :param group: The group to set (either ``gid`` or ``groupname``), optional :param recursive: whether to change ownership of all contained files and subdirectories. Only meaningful when ``self`` is a directory. If ``None``, the value will default to ``True`` if ``self`` is a directory, ``False`` otherwise. """ @abstractmethod def chmod(self, mode): """Change the mode of path to the numeric mode. :param mode: file mode as for os.chmod """ @staticmethod def _access_mode_to_flags(mode, flags = {"f" : os.F_OK, "w" : os.W_OK, "r" : os.R_OK, "x" : os.X_OK}): if isinstance(mode, str): mode = reduce(operator.or_, [flags[m] for m in mode.lower()], 0) return mode @abstractmethod def access(self, mode = 0): """Test file existence or permission bits :param mode: a bitwise-or of access bits, or a string-representation thereof: ``'f'``, ``'x'``, ``'r'``, ``'w'`` for ``os.F_OK``, ``os.X_OK``, ``os.R_OK``, ``os.W_OK`` """ @abstractmethod def link(self, dst): """Creates a hard link from ``self`` to ``dst`` :param dst: the destination path """ @abstractmethod def symlink(self, dst): """Creates a symbolic link from ``self`` to ``dst`` :param dst: the destination path """ @abstractmethod def unlink(self): """Deletes a symbolic link""" def split(self): """Splits the path on directory separators, yielding a list of directories, e.g, ``"/var/log/messages"`` will yield ``['var', 'log', 'messages']``. """ parts = [] path = self while path != path.dirname: parts.append(path.name) path = path.dirname return parts[::-1] @property def parts(self): """Splits the directory into parts, including the base directroy, returns a tuple""" return tuple([self.root] + self.split()) def relative_to(self, source): """Computes the "relative path" require to get from ``source`` to ``self``. They satisfy the invariant ``source_path + (target_path - source_path) == target_path``. For example:: /var/log/messages - /var/log/messages = [] /var/log/messages - /var = [log, messages] /var/log/messages - / = [var, log, messages] /var/log/messages - /var/tmp = [.., log, messages] /var/log/messages - /opt = [.., var, log, messages] /var/log/messages - /opt/lib = [.., .., var, log, messages] """ if isinstance(source, str): source = self._form(source) parts = self.split() baseparts = source.split() ancestors = len(list(itertools.takewhile(lambda p: p[0] == p[1], zip(parts, baseparts)))) return RelativePath([".."] * (len(baseparts) - ancestors) + parts[ancestors:]) def __sub__(self, other): """Same as ``self.relative_to(other)``""" return self.relative_to(other) def _glob(self, pattern, fn): """Applies a glob string or list/tuple/iterable to the current path, using ``fn``""" if isinstance(pattern, str): return fn(pattern) else: results = [] for single_pattern in pattern: results.extend(fn(single_pattern)) return sorted(list(set(results))) class RelativePath(object): """ Relative paths are the "delta" required to get from one path to another. Note that relative path do not point at anything, and thus are not paths. Therefore they are system agnostic (but closed under addition) Paths are always absolute and point at "something", whether existent or not. Relative paths are created by subtracting paths (``Path.relative_to``) """ def __init__(self, parts): self.parts = parts def __str__(self): return "/".join(self.parts) def __iter__(self): return iter(self.parts) def __len__(self): return len(self.parts) def __getitem__(self, index): return self.parts[index] def __repr__(self): return "RelativePath(%r)" % (self.parts,) def __eq__(self, other): return str(self) == str(other) def __ne__(self, other): return not (self == other) def __gt__(self, other): return str(self) > str(other) def __ge__(self, other): return str(self) >= str(other) def __lt__(self, other): return str(self) < str(other) def __le__(self, other): return str(self) <= str(other) def __hash__(self): return hash(str(self)) def __nonzero__(self): return bool(str(self)) __bool__ = __nonzero__ def up(self, count = 1): return RelativePath(self.parts[:-count]) def __radd__(self, path): return path.join(*self.parts) plumbum-1.6.0/plumbum/path/remote.py0000644000232200023220000002357212610225666020046 0ustar debalancedebalanceimport errno from contextlib import contextmanager from plumbum.path.base import Path, FSUser from plumbum.lib import _setdoc, six from plumbum.commands import shquote try: # Py3 import urllib.request as urllib except ImportError: import urllib class StatRes(object): """POSIX-like stat result""" def __init__(self, tup): self._tup = tuple(tup) def __getitem__(self, index): return self._tup[index] st_mode = mode = property(lambda self: self[0]) st_ino = ino = property(lambda self: self[1]) st_dev = dev = property(lambda self: self[2]) st_nlink = nlink = property(lambda self: self[3]) st_uid = uid = property(lambda self: self[4]) st_gid = gid = property(lambda self: self[5]) st_size = size = property(lambda self: self[6]) st_atime = atime = property(lambda self: self[7]) st_mtime = mtime = property(lambda self: self[8]) st_ctime = ctime = property(lambda self: self[9]) class RemotePath(Path): """The class implementing remote-machine paths""" def __new__(cls, remote, *parts): if not parts: raise TypeError("At least one path part is required (none given)") windows = (remote.uname.lower() == "windows") normed = [] parts = (remote._session.run("pwd")[1].strip(),) + parts for p in parts: if windows: plist = str(p).replace("\\", "/").split("/") else: plist = str(p).split("/") if not plist[0]: plist.pop(0) del normed[:] for item in plist: if item == "" or item == ".": continue if item == "..": if normed: normed.pop(-1) else: normed.append(item) if windows: self = super(RemotePath, cls).__new__(cls, "\\".join(normed)) self.CASE_SENSITIVE = False # On this object only else: self = super(RemotePath, cls).__new__(cls, "/" + "/".join(normed)) self.CASE_SENSITIVE = True self.remote = remote return self def _form(self, *parts): return RemotePath(self.remote, *parts) @property def _path(self): return str(self) @property @_setdoc(Path) def name(self): if not "/" in str(self): return str(self) return str(self).rsplit("/", 1)[1] @property @_setdoc(Path) def dirname(self): if not "/" in str(self): return str(self) return self.__class__(self.remote, str(self).rsplit("/", 1)[0]) @property @_setdoc(Path) def suffix(self): return '.' + self.name.rsplit('.',1)[1] @property @_setdoc(Path) def suffixes(self): name = self.name exts = [] while '.' in name: name, ext = name.rsplit('.',1) exts.append('.' + ext) return list(reversed(exts)) @property @_setdoc(Path) def uid(self): uid, name = self.remote._path_getuid(self) return FSUser(int(uid), name) @property @_setdoc(Path) def gid(self): gid, name = self.remote._path_getgid(self) return FSUser(int(gid), name) def _get_info(self): return (self.remote, self._path) @_setdoc(Path) def join(self, *parts): return RemotePath(self.remote, self, *parts) @_setdoc(Path) def list(self): if not self.is_dir(): return [] return [self.join(fn) for fn in self.remote._path_listdir(self)] @_setdoc(Path) def iterdir(self): if not self.is_dir(): return () return (self.join(fn) for fn in self.remote._path_listdir(self)) @_setdoc(Path) def is_dir(self): res = self.remote._path_stat(self) if not res: return False return res.text_mode == "directory" @_setdoc(Path) def is_file(self): res = self.remote._path_stat(self) if not res: return False return res.text_mode in ("regular file", "regular empty file") @_setdoc(Path) def is_symlink(self): res = self.remote._path_stat(self) if not res: return False return res.text_mode == "symbolic link" @_setdoc(Path) def exists(self): return self.remote._path_stat(self) is not None @_setdoc(Path) def stat(self): res = self.remote._path_stat(self) if res is None: raise OSError(errno.ENOENT) return res @_setdoc(Path) def with_name(self, name): return self.__class__(self.remote, self.dirname) / name @_setdoc(Path) def with_suffix(self, suffix, depth=1): if (suffix and not suffix.startswith('.') or suffix == '.'): raise ValueError("Invalid suffix %r" % (suffix)) name = self.name depth = len(self.suffixes) if depth is None else min(depth, len(self.suffixes)) for i in range(depth): name, ext = name.rsplit('.',1) return self.__class__(self.remote, self.dirname) / (name + suffix) @_setdoc(Path) def glob(self, pattern): fn = lambda pat: [RemotePath(self.remote, m) for m in self.remote._path_glob(self, pat)] return self._glob(pattern, fn) @_setdoc(Path) def delete(self): if not self.exists(): return self.remote._path_delete(self) unlink = delete @_setdoc(Path) def move(self, dst): if isinstance(dst, RemotePath): if dst.remote is not self.remote: raise TypeError("dst points to a different remote machine") elif not isinstance(dst, six.string_types): raise TypeError("dst must be a string or a RemotePath (to the same remote machine), " "got %r" % (dst,)) self.remote._path_move(self, dst) @_setdoc(Path) def copy(self, dst, override = False): if isinstance(dst, RemotePath): if dst.remote is not self.remote: raise TypeError("dst points to a different remote machine") elif not isinstance(dst, six.string_types): raise TypeError("dst must be a string or a RemotePath (to the same remote machine), " "got %r" % (dst,)) if override: if isinstance(dst, six.string_types): dst = RemotePath(self.remote, dst) dst.remove() self.remote._path_copy(self, dst) @_setdoc(Path) def mkdir(self): self.remote._path_mkdir(self) @_setdoc(Path) def read(self, encoding=None): data = self.remote._path_read(self) if encoding: data = data.decode(encoding) return data @_setdoc(Path) def write(self, data, encoding=None): if encoding: data = data.encode(encoding) self.remote._path_write(self, data) @_setdoc(Path) def chown(self, owner = None, group = None, recursive = None): self.remote._path_chown(self, owner, group, self.is_dir() if recursive is None else recursive) @_setdoc(Path) def chmod(self, mode): self.remote._path_chmod(mode, self) @_setdoc(Path) def access(self, mode = 0): mode = self._access_mode_to_flags(mode) res = self.remote._path_stat(self) if res is None: return False mask = res.st_mode & 0x1ff return ((mask >> 6) & mode) or ((mask >> 3) & mode) @_setdoc(Path) def link(self, dst): if isinstance(dst, RemotePath): if dst.remote is not self.remote: raise TypeError("dst points to a different remote machine") elif not isinstance(dst, six.string_types): raise TypeError("dst must be a string or a RemotePath (to the same remote machine), " "got %r" % (dst,)) self.remote._path_link(self, dst, False) @_setdoc(Path) def symlink(self, dst): if isinstance(dst, RemotePath): if dst.remote is not self.remote: raise TypeError("dst points to a different remote machine") elif not isinstance(dst, six.string_types): raise TypeError("dst must be a string or a RemotePath (to the same remote machine), " "got %r" % (dst,)) self.remote._path_link(self, dst, True) def open(self): pass @_setdoc(Path) def as_uri(self, scheme = 'ssh'): return '{0}://{1}{2}'.format(scheme, self.remote._fqhost, urllib.pathname2url(str(self))) @property @_setdoc(Path) def stem(self): return self.name.rsplit('.')[0] @property @_setdoc(Path) def root(self): return '/' @property @_setdoc(Path) def drive(self): return '' class RemoteWorkdir(RemotePath): """Remote working directory manipulator""" def __new__(cls, remote): self = super(RemoteWorkdir, cls).__new__(cls, remote, remote._session.run("pwd")[1].strip()) return self def __hash__(self): raise TypeError("unhashable type") def chdir(self, newdir): """Changes the current working directory to the given one""" self.remote._session.run("cd %s" % (shquote(newdir),)) return self.__class__(self.remote) def getpath(self): """Returns the current working directory as a `remote path ` object""" return RemotePath(self.remote, self) @contextmanager def __call__(self, newdir): """A context manager used to ``chdir`` into a directory and then ``chdir`` back to the previous location; much like ``pushd``/``popd``. :param newdir: The destination director (a string or a :class:`RemotePath `) """ prev = self._path changed_dir = self.chdir(newdir) try: yield changed_dir finally: self.chdir(prev) plumbum-1.6.0/plumbum/fs/0000755000232200023220000000000012610225742015637 5ustar debalancedebalanceplumbum-1.6.0/plumbum/fs/__init__.py0000644000232200023220000000004612212311055017737 0ustar debalancedebalance""" file-system related operations """plumbum-1.6.0/plumbum/fs/mounts.py0000644000232200023220000000211012212310230017511 0ustar debalancedebalanceimport re class MountEntry(object): """ Represents a mount entry (device file, mount point and file system type) """ def __init__(self, dev, point, fstype, options): self.dev = dev self.point = point self.fstype = fstype self.options = options.split(",") def __str__(self): return "%s on %s type %s (%s)" % (self.dev, self.point, self.fstype, ",".join(self.options)) MOUNT_PATTERN = re.compile(r"(.+?)\s+on\s+(.+?)\s+type\s+(\S+)(?:\s+\((.+?)\))?") def mount_table(): """returns the system's current mount table (a list of :class:`MountEntry ` objects)""" from plumbum.cmd import mount table = [] for line in mount().splitlines(): m = MOUNT_PATTERN.match(line) if not m: continue table.append(MountEntry(*m.groups())) return table def mounted(fs): """ Indicates if a the given filesystem (device file or mount point) is currently mounted """ return any(fs == entry.dev or fs == entry.point for entry in mount_table()) plumbum-1.6.0/plumbum/fs/atomic.py0000644000232200023220000002260612610225666017500 0ustar debalancedebalance""" Atomic file operations """ import os import threading import sys import atexit from contextlib import contextmanager from plumbum.machines.local import local from plumbum.lib import six if not hasattr(threading, "get_ident"): try: import thread except ImportError: import _thread as thread threading.get_ident = thread.get_ident del thread try: import fcntl except ImportError: import msvcrt try: from pywintypes import error as WinError from win32file import LockFileEx, UnlockFile, OVERLAPPED from win32con import LOCKFILE_EXCLUSIVE_LOCK, LOCKFILE_FAIL_IMMEDIATELY except ImportError: raise ImportError("On Windows, we require Python for Windows Extensions (pywin32)") @contextmanager def locked_file(fileno, blocking = True): hndl = msvcrt.get_osfhandle(fileno) try: LockFileEx(hndl, LOCKFILE_EXCLUSIVE_LOCK | (0 if blocking else LOCKFILE_FAIL_IMMEDIATELY), 0xffffffff, 0xffffffff, OVERLAPPED()) except WinError: _, ex, _ = sys.exc_info() raise WindowsError(*ex.args) try: yield finally: UnlockFile(hndl, 0, 0, 0xffffffff, 0xffffffff) else: if hasattr(fcntl, "lockf"): @contextmanager def locked_file(fileno, blocking = True): fcntl.lockf(fileno, fcntl.LOCK_EX | (0 if blocking else fcntl.LOCK_NB)) try: yield finally: fcntl.lockf(fileno, fcntl.LOCK_UN) else: @contextmanager def locked_file(fileno, blocking = True): fcntl.flock(fileno, fcntl.LOCK_EX | (0 if blocking else fcntl.LOCK_NB)) try: yield finally: fcntl.flock(fileno, fcntl.LOCK_UN) class AtomicFile(object): """ Atomic file operations implemented using file-system advisory locks (``flock`` on POSIX, ``LockFile`` on Windows). .. note:: On Linux, the manpage says ``flock`` might have issues with NFS mounts. You should take this into account. .. versionadded:: 1.3 """ CHUNK_SIZE = 32 * 1024 def __init__(self, filename, ignore_deletion = False): self.path = local.path(filename) self._ignore_deletion = ignore_deletion self._thdlock = threading.Lock() self._owned_by = None self._fileobj = None self.reopen() def __repr__(self): return "" % (self.path,) if self._fileobj else "" def __del__(self): self.close() def __enter__(self): return self def __exit__(self, t, v, tb): self.close() def close(self): if self._fileobj is not None: self._fileobj.close() self._fileobj = None def reopen(self): """ Close and reopen the file; useful when the file was deleted from the file system by a different process """ self.close() self._fileobj = os.fdopen(os.open(str(self.path), os.O_CREAT | os.O_RDWR, 384), "r+b", 0) @contextmanager def locked(self, blocking = True): """ A context manager that locks the file; this function is reentrant by the thread currently holding the lock. :param blocking: if ``True``, the call will block until we can grab the file system lock. if ``False``, the call may fail immediately with the underlying exception (``IOError`` or ``WindowsError``) """ if self._owned_by == threading.get_ident(): yield return with self._thdlock: with locked_file(self._fileobj.fileno(), blocking): if not self.path.exists() and not self._ignore_deletion: raise ValueError("Atomic file removed from filesystem") self._owned_by = threading.get_ident() try: yield finally: self._owned_by = None def delete(self): """ Atomically delete the file (holds the lock while doing it) """ with self.locked(): self.path.delete() def _read_all(self): self._fileobj.seek(0) data = [] while True: buf = self._fileobj.read(self.CHUNK_SIZE) data.append(buf) if len(buf) < self.CHUNK_SIZE: break return six.b("").join(data) def read_atomic(self): """Atomically read the entire file""" with self.locked(): return self._read_all() def read_shared(self): """Read the file **without** holding the lock""" return self._read_all() def write_atomic(self, data): """Writes the given data atomically to the file. Note that it overwrites the entire file; ``write_atomic("foo")`` followed by ``write_atomic("bar")`` will result in only ``"bar"``. """ with self.locked(): self._fileobj.seek(0) while data: chunk = data[:self.CHUNK_SIZE] self._fileobj.write(chunk) data = data[len(chunk):] self._fileobj.flush() self._fileobj.truncate() class AtomicCounterFile(object): """ An atomic counter based on AtomicFile. Each time you call ``next()``, it will atomically read and increment the counter's value, returning its previous value Example:: acf = AtomicCounterFile.open("/some/file") print acf.next() # e.g., 7 print acf.next() # 8 print acf.next() # 9 .. versionadded:: 1.3 """ def __init__(self, atomicfile, initial = 0): """ :param atomicfile: an :class:`AtomicFile ` instance :param initial: the initial value (used when the first time the file is created) """ self.atomicfile = atomicfile self.initial = initial def __enter__(self): return self def __exit__(self, t, v, tb): self.close() def close(self): self.atomicfile.close() @classmethod def open(cls, filename): """ Shortcut for ``AtomicCounterFile(AtomicFile(filename))`` """ return cls(AtomicFile(filename)) def reset(self, value = None): """ Reset the counter's value to the one given. If ``None``, it will default to the initial value provided to the constructor """ if value is None: value = self.initial if not isinstance(value, six.integer_types): raise TypeError("value must be an integer, not %r" % (type(value),)) self.atomicfile.write_atomic(str(value).encode("utf8")) def next(self): """ Read and increment the counter, returning its previous value """ with self.atomicfile.locked(): curr = self.atomicfile.read_atomic().decode("utf8") if not curr: curr = self.initial else: curr = int(curr) self.atomicfile.write_atomic(str(curr + 1).encode("utf8")) return curr class PidFileTaken(SystemExit): """ This exception is raised when PidFile.acquire fails to lock the pid file. Note that it derives from ``SystemExit``, so unless explicitly handled, it will terminate the process cleanly """ def __init__(self, msg, pid): SystemExit.__init__(self, msg) self.pid = pid class PidFile(object): """ A PID file is a file that's locked by some process from the moment it starts until it dies (the OS will clear the lock when the process exits). It is used to prevent two instances of the same process (normally a daemon) from running concurrently. The PID file holds its process' PID, so you know who's holding it. .. versionadded:: 1.3 """ def __init__(self, filename): self.atomicfile = AtomicFile(filename) self._ctx = None def __enter__(self): self.acquire() def __exit__(self, t, v, tb): self.release() def __del__(self): try: self.release() except Exception: pass def close(self): self.atomicfile.close() def acquire(self): """ Attempt to acquire the PID file. If it's already locked, raises :class:`PidFileTaken `. You should normally acquire the file as early as possible when the program starts """ if self._ctx is not None: return self._ctx = self.atomicfile.locked(blocking = False) try: self._ctx.__enter__() except (IOError, OSError): self._ctx = None try: pid = self.atomicfile.read_shared().strip().decode("utf8") except (IOError, OSError): pid = "Unknown" raise PidFileTaken("PID file %r taken by process %s" % (self.atomicfile.path, pid), pid) else: self.atomicfile.write_atomic(str(os.getpid()).encode("utf8")) atexit.register(self.release) def release(self): """ Release the PID file (should only happen when the program terminates) """ if self._ctx is None: return self.atomicfile.delete() try: self._ctx.__exit__(None, None, None) finally: self._ctx = None plumbum-1.6.0/plumbum/_testtools.py0000644000232200023220000000202012610225666017777 0ustar debalancedebalanceimport os import sys import unittest from plumbum import local from plumbum.lib import IS_WIN32 def ensure_skipIf(unittest): """ This will ensure that unittest has skipIf. Call like:: import unittest ensure_skipIf(unittest) """ if not hasattr(unittest, "skipIf"): import logging import functools def skipIf(condition, reason): def deco(func): if not condition: return func else: @functools.wraps(func) def wrapper(*args, **kwargs): logging.warn("skipping test: "+reason) return wrapper return deco unittest.skipIf = skipIf ensure_skipIf(unittest) skipIf = unittest.skipIf skip_on_windows = unittest.skipIf(IS_WIN32, "Does not work on Windows (yet)") skip_without_chown = unittest.skipIf(not hasattr(os, "chown"), "os.chown not supported") skip_without_tty = unittest.skipIf(not sys.stdin.isatty(), "Not a TTY") plumbum-1.6.0/plumbum/colors.py0000644000232200023220000000120612610225666017106 0ustar debalancedebalance""" This module imitates a real module, providing standard syntax like from `plumbum.colors` and from `plumbum.colors.bg` to work alongside all the standard syntax for colors. """ from __future__ import print_function import sys import os import atexit from plumbum.colorlib import ansicolors, main _reset = ansicolors.reset.now if __name__ == '__main__': main() else: # Don't register an exit if this is called using -m! atexit.register(_reset) # Oddly, the order here matters for Python2, but not Python3 sys.modules[__name__ + '.fg'] = ansicolors.fg sys.modules[__name__ + '.bg'] = ansicolors.bg sys.modules[__name__] = ansicolors plumbum-1.6.0/plumbum/lib.py0000644000232200023220000000655712610225666016371 0ustar debalancedebalanceimport sys from contextlib import contextmanager from abc import ABCMeta import inspect IS_WIN32 = (sys.platform == "win32") def _setdoc(super): # @ReservedAssignment """This inherits the docs on the current class. Not really needed for Python 3.5, due to new behavoir of inspect.getdoc, but still doesn't hurt.""" def deco(func): func.__doc__ = getattr(getattr(super, func.__name__, None), "__doc__", None) return func return deco class ProcInfo(object): def __init__(self, pid, uid, stat, args): self.pid = pid self.uid = uid self.stat = stat self.args = args def __repr__(self): return "ProcInfo(%r, %r, %r, %r)" % (self.pid, self.uid, self.stat, self.args) class six(object): """ A light-weight version of six (which works on IronPython) """ PY3 = sys.version_info[0] >= 3 ABC = ABCMeta('ABC', (object,), {'__module__':__name__, '__slots__':()}) # Be sure to use named-tuple access, so that usage is not affected try: getfullargspec = staticmethod(inspect.getfullargspec) except AttributeError: getfullargspec = staticmethod(inspect.getargspec) # extra fields will not be available if PY3: integer_types = (int,) string_types = (str,) MAXSIZE = sys.maxsize ascii = ascii # @UndefinedVariable bytes = bytes # @ReservedAssignment unicode_type = str @staticmethod def b(s): return s.encode("latin-1", "replace") @staticmethod def u(s): return s @staticmethod def get_method_function(m): return m.__func__ else: integer_types = (int, long) string_types = (str, unicode) MAXSIZE = getattr(sys, "maxsize", sys.maxint) ascii = repr # @ReservedAssignment bytes = str # @ReservedAssignment unicode_type = unicode @staticmethod def b(st): return st @staticmethod def u(s): return s.decode("unicode-escape") @staticmethod def get_method_function(m): return m.im_func # Try/except fails because io has the wrong StringIO in Python2 # You'll get str/unicode errors if six.PY3: from io import StringIO else: from StringIO import StringIO @contextmanager def captured_stdout(stdin = ""): """ Captures stdout (similar to the redirect_stdout in Python 3.4+, but with slightly different arguments) """ prevstdin = sys.stdin prevstdout = sys.stdout sys.stdin = StringIO(six.u(stdin)) sys.stdout = StringIO() try: yield sys.stdout finally: sys.stdin = prevstdin sys.stdout = prevstdout class StaticProperty(object): """This acts like a static property, allowing access via class or object. This is a non-data descriptor.""" def __init__(self, function): self._function = function self.__doc__ = function.__doc__ def __get__(self, obj, klass=None): return self._function() def getdoc(object): """ This gets a docstring if avaiable, and cleans it, but does not look up docs in inheritance tree (Pre 3.5 behavior of ``inspect.getdoc``). """ try: doc = object.__doc__ except AttributeError: return None if not isinstance(doc, str): return None return inspect.cleandoc(doc) plumbum-1.6.0/plumbum/commands/0000755000232200023220000000000012610225742017030 5ustar debalancedebalanceplumbum-1.6.0/plumbum/commands/__init__.py0000644000232200023220000000051212610225666021144 0ustar debalancedebalancefrom plumbum.commands.base import shquote, shquote_list, BaseCommand, ERROUT, ConcreteCommand from plumbum.commands.modifiers import ExecutionModifier, Future, FG, BG, TF, RETCODE, NOHUP from plumbum.commands.processes import run_proc from plumbum.commands.processes import ProcessExecutionError, ProcessTimedOut, CommandNotFound plumbum-1.6.0/plumbum/commands/processes.py0000644000232200023220000002362512610225666021425 0ustar debalancedebalanceimport time import atexit import heapq from subprocess import Popen from threading import Thread from plumbum.lib import IS_WIN32, six try: from queue import Queue, Empty as QueueEmpty except ImportError: from Queue import Queue, Empty as QueueEmpty try: from io import StringIO except ImportError: from cStringIO import StringIO #=================================================================================================== # utility functions #=================================================================================================== def _check_process(proc, retcode, timeout, stdout, stderr): if getattr(proc, "_timed_out", False): raise ProcessTimedOut("Process did not terminate within %s seconds" % (timeout,), getattr(proc, "argv", None)) if retcode is not None: if hasattr(retcode, "__contains__"): if proc.returncode not in retcode: raise ProcessExecutionError(getattr(proc, "argv", None), proc.returncode, stdout, stderr) elif proc.returncode != retcode: raise ProcessExecutionError(getattr(proc, "argv", None), proc.returncode, stdout, stderr) return proc.returncode, stdout, stderr def _iter_lines(proc, decode, linesize): try: from selectors import DefaultSelector, EVENT_READ except ImportError: # Pre Python 3.4 implementation from select import select def selector(): while True: rlist, _, _ = select([proc.stdout, proc.stderr], [], []) for stream in rlist: yield (stream is proc.stderr), decode(stream.readline(linesize)) else: # Python 3.4 implementation def selector(): sel = DefaultSelector() sel.register(proc.stdout, EVENT_READ, 0) sel.register(proc.stderr, EVENT_READ, 1) while True: for key, mask in sel.select(): yield key.data, decode(key.fileobj.readline(linesize)) for ret in selector(): yield ret if proc.poll() is not None: break for line in proc.stdout: yield 0, decode(line) for line in proc.stderr: yield 1, decode(line) #=================================================================================================== # Exceptions #=================================================================================================== class ProcessExecutionError(EnvironmentError): """Represents the failure of a process. When the exit code of a terminated process does not match the expected result, this exception is raised by :func:`run_proc `. It contains the process' return code, stdout, and stderr, as well as the command line used to create the process (``argv``) """ def __init__(self, argv, retcode, stdout, stderr): Exception.__init__(self, argv, retcode, stdout, stderr) self.argv = argv self.retcode = retcode if six.PY3 and isinstance(stdout, six.bytes): stdout = six.ascii(stdout) if six.PY3 and isinstance(stderr, six.bytes): stderr = six.ascii(stderr) self.stdout = stdout self.stderr = stderr def __str__(self): stdout = "\n | ".join(str(self.stdout).splitlines()) stderr = "\n | ".join(str(self.stderr).splitlines()) lines = ["Command line: %r" % (self.argv,), "Exit code: %s" % (self.retcode)] if stdout: lines.append("Stdout: | %s" % (stdout,)) if stderr: lines.append("Stderr: | %s" % (stderr,)) return "\n".join(lines) class ProcessTimedOut(Exception): """Raises by :func:`run_proc ` when a ``timeout`` has been specified and it has elapsed before the process terminated""" def __init__(self, msg, argv): Exception.__init__(self, msg, argv) self.argv = argv class CommandNotFound(AttributeError): """Raised by :func:`local.which ` and :func:`RemoteMachine.which ` when a command was not found in the system's ``PATH``""" def __init__(self, program, path): Exception.__init__(self, program, path) self.program = program self.path = path #=================================================================================================== # Timeout thread #=================================================================================================== class MinHeap(object): def __init__(self, items = ()): self._items = list(items) heapq.heapify(self._items) def __len__(self): return len(self._items) def push(self, item): heapq.heappush(self._items, item) def pop(self): heapq.heappop(self._items) def peek(self): return self._items[0] _timeout_queue = Queue() _shutting_down = False def _timeout_thread_func(): waiting = MinHeap() try: while not _shutting_down: if waiting: ttk, _ = waiting.peek() timeout = max(0, ttk - time.time()) else: timeout = None try: proc, time_to_kill = _timeout_queue.get(timeout = timeout) if proc is SystemExit: # terminate return waiting.push((time_to_kill, proc)) except QueueEmpty: pass now = time.time() while waiting: ttk, proc = waiting.peek() if ttk > now: break waiting.pop() try: if proc.poll() is None: proc.kill() proc._timed_out = True except EnvironmentError: pass except Exception: if _shutting_down: # to prevent all sorts of exceptions during interpreter shutdown pass else: raise bgthd = Thread(target = _timeout_thread_func, name = "PlumbumTimeoutThread") bgthd.setDaemon(True) bgthd.start() def _register_proc_timeout(proc, timeout): if timeout is not None: _timeout_queue.put((proc, time.time() + timeout)) def _shutdown_bg_threads(): global _shutting_down _shutting_down = True _timeout_queue.put((SystemExit, 0)) # grace period bgthd.join(0.1) atexit.register(_shutdown_bg_threads) #=================================================================================================== # run_proc #=================================================================================================== def run_proc(proc, retcode, timeout = None): """Waits for the given process to terminate, with the expected exit code :param proc: a running Popen-like object :param retcode: the expected return (exit) code of the process. It defaults to 0 (the convention for success). If ``None``, the return code is ignored. It may also be a tuple (or any object that supports ``__contains__``) of expected return codes. :param timeout: the number of seconds (a ``float``) to allow the process to run, before forcefully terminating it. If ``None``, not timeout is imposed; otherwise the process is expected to terminate within that timeout value, or it will be killed and :class:`ProcessTimedOut ` will be raised :returns: A tuple of (return code, stdout, stderr) """ _register_proc_timeout(proc, timeout) stdout, stderr = proc.communicate() proc._end_time = time.time() if not stdout: stdout = six.b("") if not stderr: stderr = six.b("") if getattr(proc, "encoding", None): stdout = stdout.decode(proc.encoding, "ignore") stderr = stderr.decode(proc.encoding, "ignore") return _check_process(proc, retcode, timeout, stdout, stderr) #=================================================================================================== # iter_lines #=================================================================================================== def iter_lines(proc, retcode = 0, timeout = None, linesize = -1, _iter_lines = _iter_lines): """Runs the given process (equivalent to run_proc()) and yields a tuples of (out, err) line pairs. If the exit code of the process does not match the expected one, :class:`ProcessExecutionError ` is raised. :param retcode: The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass ``None``. It may also be a tuple (or any iterable) of expected exit codes. :param timeout: The maximal amount of time (in seconds) to allow the process to run. ``None`` means no timeout is imposed; otherwise, if the process hasn't terminated after that many seconds, the process will be forcefully terminated an exception will be raised :param linesize: Maximum number of characters to read from stdout/stderr at each iteration. ``-1`` (default) reads until a b'\\n' is encountered. :returns: An iterator of (out, err) line tuples. """ encoding = getattr(proc, "encoding", None) if encoding: decode = lambda s: s.decode(encoding).rstrip() else: decode = lambda s: s _register_proc_timeout(proc, timeout) buffers = [StringIO(), StringIO()] for t, line in _iter_lines(proc, decode, linesize): ret = [None, None] ret[t] = line buffers[t].write(line + "\n") yield ret # this will take care of checking return code and timeouts _check_process(proc, retcode, timeout, *(s.getvalue() for s in buffers)) plumbum-1.6.0/plumbum/commands/modifiers.py0000644000232200023220000002735312610225666021402 0ustar debalancedebalanceimport os from select import select from subprocess import PIPE import sys from itertools import chain from plumbum.commands.processes import run_proc, ProcessExecutionError from plumbum.commands.base import AppendingStdoutRedirection, StdoutRedirection class Future(object): """Represents a "future result" of a running process. It basically wraps a ``Popen`` object and the expected exit code, and provides poll(), wait(), returncode, stdout, and stderr. """ def __init__(self, proc, expected_retcode, timeout = None): self.proc = proc self._expected_retcode = expected_retcode self._timeout = timeout self._returncode = None self._stdout = None self._stderr = None def __repr__(self): return "" % (self.proc.argv, self._returncode if self.ready() else "running",) def poll(self): """Polls the underlying process for termination; returns ``False`` if still running, or ``True`` if terminated""" if self.proc.poll() is not None: self.wait() return self._returncode is not None ready = poll def wait(self): """Waits for the process to terminate; will raise a :class:`plumbum.commands.ProcessExecutionError` in case of failure""" if self._returncode is not None: return self._returncode, self._stdout, self._stderr = run_proc(self.proc, self._expected_retcode, self._timeout) @property def stdout(self): """The process' stdout; accessing this property will wait for the process to finish""" self.wait() return self._stdout @property def stderr(self): """The process' stderr; accessing this property will wait for the process to finish""" self.wait() return self._stderr @property def returncode(self): """The process' returncode; accessing this property will wait for the process to finish""" self.wait() return self._returncode #=================================================================================================== # execution modifiers #=================================================================================================== class ExecutionModifier(object): __slots__ = () def __repr__(self): """Automatically creates a representation for given subclass with slots. Ignore hidden properties.""" slots = {} for cls in self.__mro__: for prop in getattr(cls, "__slots__", ()): if prop[0] != '_': slots[prop] = getattr(self, prop) mystrs = ("{0} = {1}".format(name, slots[name]) for name in slots) return "{0}({1})".format(self.__class__.__name__, ", ".join(mystrs)) @classmethod def __call__(cls, *args, **kwargs): return cls(*args, **kwargs) class BG(ExecutionModifier): """ An execution modifier that runs the given command in the background, returning a :class:`Future ` object. In order to mimic shell syntax, it applies when you right-and it with a command. If you wish to expect a different return code (other than the normal success indicate by 0), use ``BG(retcode)``. Example:: future = sleep[5] & BG # a future expecting an exit code of 0 future = sleep[5] & BG(7) # a future expecting an exit code of 7 .. note:: When processes run in the **background** (either via ``popen`` or :class:`& BG `), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see `#48 `_ """ __slots__ = ("retcode",) def __init__(self, retcode=0): self.retcode = retcode def __rand__(self, cmd): return Future(cmd.popen(), self.retcode) BG = BG() """ An execution modifier that runs the given command in the background, returning a :class:`Future ` object. In order to mimic shell syntax, it applies when you right-and it with a command. If you wish to expect a different return code (other than the normal success indicate by 0), use ``BG(retcode)``. Example:: future = sleep[5] & BG # a future expecting an exit code of 0 future = sleep[5] & BG(7) # a future expecting an exit code of 7 .. note:: When processes run in the **background** (either via ``popen`` or :class:`& BG `), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see `#48 `_ """ class FG(ExecutionModifier): """ An execution modifier that runs the given command in the foreground, passing it the current process' stdin, stdout and stderr. Useful for interactive programs that require a TTY. There is no return value. In order to mimic shell syntax, it applies when you right-and it with a command. If you wish to expect a different return code (other than the normal success indicate by 0), use ``FG(retcode)``. Example:: vim & FG # run vim in the foreground, expecting an exit code of 0 vim & FG(7) # run vim in the foreground, expecting an exit code of 7 """ __slots__ = ("retcode",) def __init__(self, retcode=0): self.retcode = retcode def __rand__(self, cmd): cmd(retcode = self.retcode, stdin = None, stdout = None, stderr = None) FG = FG() class TEE(ExecutionModifier): """Run a command, dumping its stdout/stderr to the current process's stdout and stderr, but ALSO return them. Useful for interactive programs that expect a TTY but also have valuable output. Use as: ls["-l"] & TEE Returns a tuple of (return code, stdout, stderr), just like ``run()``. """ __slots__ = ("retcode", "buffered") def __init__(self, retcode=0, buffered=True): """`retcode` is the return code to expect to mean "success". Set `buffered` to False to disable line-buffering the output, which may cause stdout and stderr to become more entangled than usual. """ self.retcode = retcode self.buffered = buffered def __rand__(self, cmd): with cmd.bgrun(retcode=self.retcode, stdin=None, stdout=PIPE, stderr=PIPE) as p: outbuf = [] errbuf = [] out = p.stdout err = p.stderr buffers = {out: outbuf, err: errbuf} tee_to = {out: sys.stdout, err: sys.stderr} while p.poll() is None: ready, _, _ = select((out, err), (), ()) for fd in ready: buf = buffers[fd] data = os.read(fd.fileno(), 4096) if not data: # eof continue # Python conveniently line-buffers stdout and stderr for # us, so all we need to do is write to them tee_to[fd].write(data.decode('utf-8')) # And then "unbuffered" is just flushing after each write if not self.buffered: tee_to[fd].flush() buf.append(data) stdout = ''.join([x.decode('utf-8') for x in outbuf]) stderr = ''.join([x.decode('utf-8') for x in errbuf]) return p.returncode, stdout, stderr TEE = TEE() class TF(ExecutionModifier): """ An execution modifier that runs the given command, but returns True/False depending on the retcode. This returns True if the expected exit code is returned, and false if it is not. This is useful for checking true/false bash commands. If you wish to expect a different return code (other than the normal success indicate by 0), use ``TF(retcode)``. If you want to run the process in the forground, then use ``TF(FG=True)``. Example:: local['touch']['/root/test'] & TF * Returns False, since this cannot be touched local['touch']['/root/test'] & TF(1) # Returns True local['touch']['/root/test'] & TF(FG=True) * Returns False, will show error message """ __slots__ = ("retcode", "FG") def __init__(self, retcode=0, FG=False): """`retcode` is the return code to expect to mean "success". Set `FG` to True to run in the foreground. """ self.retcode = retcode self.FG = FG @classmethod def __call__(cls, *args, **kwargs): return cls(*args, **kwargs) def __rand__(self, cmd): try: if self.FG: cmd(retcode = self.retcode, stdin = None, stdout = None, stderr = None) else: cmd(retcode = self.retcode) return True except ProcessExecutionError: return False TF = TF() class RETCODE(ExecutionModifier): """ An execution modifier that runs the given command, causing it to run and return the retcode. This is useful for working with bash commands that have important retcodes but not very useful output. If you want to run the process in the forground, then use ``RETCODE(FG=True)``. Example:: local['touch']['/root/test'] & RETCODE # Returns 1, since this cannot be touched local['touch']['/root/test'] & RETCODE(FG=True) * Returns 1, will show error message """ __slots__ = ("foreground",) def __init__(self, FG=False): """`FG` to True to run in the foreground. """ self.foreground = FG @classmethod def __call__(cls, *args, **kwargs): return cls(*args, **kwargs) def __rand__(self, cmd): if self.foreground: return cmd.run(retcode = None, stdin = None, stdout = None, stderr = None)[0] else: return cmd.run(retcode = None)[0] RETCODE = RETCODE() class NOHUP(ExecutionModifier): """ An execution modifier that runs the given command in the background, disconnected from the current process, returning a standard popen object. It will keep running even if you close the current process. In order to slightly mimic shell syntax, it applies when you right-and it with a command. If you wish to use a diffent working directory or different stdout, stderr, you can use named arguments. The default is ``NOHUP( cwd=local.cwd, stdout='nohup.out', stderr=None)``. If stderr is None, stderr will be sent to stdout. Use ``os.devnull`` for null output. Will respect redirected output. Example:: sleep[5] & NOHUP # Outputs to nohup.out sleep[5] & NOHUP(stdout=os.devnull) # No output The equivelent bash command would be .. code-block:: bash nohup sleep 5 & """ __slots__ = ('cwd', 'stdout', 'stderr', 'append') def __init__(self, cwd='.', stdout='nohup.out', stderr=None, append=True): """ Set ``cwd``, ``stdout``, or ``stderr``. Runs as a forked process. You can set ``append=False``, too. """ self.cwd = cwd self.stdout = stdout self.stderr = stderr self.append = append def __rand__(self, cmd): if isinstance(cmd, StdoutRedirection): stdout = cmd.file append = False cmd = cmd.cmd elif isinstance(cmd, AppendingStdoutRedirection): stdout = cmd.file append = True cmd = cmd.cmd else: stdout = self.stdout append = self.append return cmd.nohup(cmd, self.cwd, stdout, self.stderr, append) NOHUP = NOHUP() plumbum-1.6.0/plumbum/commands/daemons.py0000644000232200023220000000674612610225666021052 0ustar debalancedebalanceimport subprocess import os import time import sys import errno import signal import traceback from plumbum.commands.processes import ProcessExecutionError class _fake_lock(object): """Needed to allow normal os.exit() to work without error""" def acquire(self, val): return True def release(self): pass def posix_daemonize(command, cwd, stdout=None, stderr=None, append=True): if stdout is None: stdout = os.devnull if stderr is None: stderr = stdout MAX_SIZE = 16384 rfd, wfd = os.pipe() argv = command.formulate() firstpid = os.fork() if firstpid == 0: # first child: become session leader, os.close(rfd) rc = 0 try: os.setsid() os.umask(0) stdin = open(os.devnull, "r") stdout = open(stdout, "a" if append else "w") stderr = open(stderr, "a" if append else "w") signal.signal(signal.SIGHUP, signal.SIG_IGN) proc = command.popen(cwd = cwd, close_fds = True, stdin = stdin.fileno(), stdout = stdout.fileno(), stderr = stderr.fileno()) os.write(wfd, str(proc.pid).encode("utf8")) except: rc = 1 tbtext = "".join(traceback.format_exception(*sys.exc_info()))[-MAX_SIZE:] os.write(wfd, tbtext.encode("utf8")) finally: os.close(wfd) os._exit(rc) else: # wait for first child to die os.close(wfd) _, rc = os.waitpid(firstpid, 0) output = os.read(rfd, MAX_SIZE) try: output = output.decode("utf8") except UnicodeError: pass if rc == 0 and output.isdigit(): secondpid = int(output) else: raise ProcessExecutionError(argv, rc, "", output) proc = subprocess.Popen.__new__(subprocess.Popen) proc._child_created = True proc.returncode = None proc.stdout = None proc.stdin = None proc.stderr = None proc.pid = secondpid proc.universal_newlines = False proc._input = None proc._waitpid_lock = _fake_lock() proc._communication_started = False proc.args = argv proc.argv = argv def poll(self = proc): if self.returncode is None: try: os.kill(self.pid, 0) except OSError: ex = sys.exc_info()[1] if ex.errno == errno.ESRCH: # process does not exist self.returncode = 0 else: raise return self.returncode def wait(self = proc): while self.returncode is None: if self.poll() is None: time.sleep(0.5) return proc.returncode proc.poll = poll proc.wait = wait return proc def win32_daemonize(command, cwd, stdout=None, stderr=None, append=True): if stdout is None: stdout = os.devnull if stderr is None: stderr = stdout DETACHED_PROCESS = 0x00000008 stdin = open(os.devnull, "r") stdout = open(stdout, "a" if append else "w") stderr = open(stderr, "a" if append else "w") return command.popen(cwd = cwd, stdin = stdin.fileno(), stdout = stdout.fileno(), stderr = stderr.fileno(), creationflags = subprocess.CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS) plumbum-1.6.0/plumbum/commands/base.py0000644000232200023220000003650412610225666020331 0ustar debalancedebalanceimport subprocess import functools from contextlib import contextmanager from plumbum.commands.processes import run_proc, iter_lines from plumbum.lib import six from tempfile import TemporaryFile from subprocess import PIPE, Popen class RedirectionError(Exception): """Raised when an attempt is made to redirect an process' standard handle, which was already redirected to/from a file""" #=================================================================================================== # Utilities #=================================================================================================== # modified from the stdlib pipes module for windows _safechars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@%_-+=:,./' _funnychars = '"`$\\' def shquote(text): """Quotes the given text with shell escaping (assumes as syntax similar to ``sh``)""" if not text: return "''" text = str(text) if not text: return "''" for c in text: if c not in _safechars: break else: return text if "'" not in text: return "'" + text + "'" res = "".join(('\\' + c if c in _funnychars else c) for c in text) return '"' + res + '"' def shquote_list(seq): return [shquote(item) for item in seq] #=================================================================================================== # Commands #=================================================================================================== class BaseCommand(object): """Base of all command objects""" __slots__ = ["cwd", "env", "encoding"] def __str__(self): return " ".join(self.formulate()) def __or__(self, other): """Creates a pipe with the other command""" return Pipeline(self, other) def __gt__(self, file): """Redirects the process' stdout to the given file""" return StdoutRedirection(self, file) def __rshift__(self, file): """Redirects the process' stdout to the given file (appending)""" return AppendingStdoutRedirection(self, file) def __ge__(self, file): """Redirects the process' stderr to the given file""" return StderrRedirection(self, file) def __lt__(self, file): """Redirects the given file into the process' stdin""" return StdinRedirection(self, file) def __lshift__(self, data): """Redirects the given data into the process' stdin""" return StdinDataRedirection(self, data) def __getitem__(self, args): """Creates a bound-command with the given arguments""" if not isinstance(args, (tuple, list)): args = [args, ] if not args: return self if isinstance(self, BoundCommand): return BoundCommand(self.cmd, self.args + list(args)) else: return BoundCommand(self, args) def __call__(self, *args, **kwargs): """A shortcut for `run(args)`, returning only the process' stdout""" return self.run(args, **kwargs)[1] def _get_encoding(self): raise NotImplementedError() def with_env(self, **envvars): """Returns a BoundEnvCommand with the given environment variables""" if not envvars: return self return BoundEnvCommand(self, envvars) setenv = with_env @property def machine(self): raise NotImplementedError() def formulate(self, level = 0, args = ()): """Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by ``Popen`` or shells. :param level: The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed :param args: The arguments passed to this command (a tuple) :returns: A list of strings """ raise NotImplementedError() def popen(self, args = (), **kwargs): """Spawns the given command, returning a ``Popen``-like object. .. note:: When processes run in the **background** (either via ``popen`` or :class:`& BG `), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see `#48 `_ :param args: Any arguments to be passed to the process (a tuple) :param kwargs: Any keyword-arguments to be passed to the ``Popen`` constructor :returns: A ``Popen``-like object """ raise NotImplementedError() def nohup(self, command, cwd='.', stdout='nohup.out', stderr=None, append=True): """Runs a command detached.""" return self.machine.daemonic_popen(self, cwd, stdout, stderr, append) @contextmanager def bgrun(self, args = (), **kwargs): """Runs the given command as a context manager, allowing you to create a `pipeline `_ (not in the UNIX sense) of programs, parallelizing their work. In other words, instead of running programs one after the other, you can start all of them at the same time and wait for them to finish. For a more thorough review, see `Lightweight Asynchronism `_. Example:: from plumbum.cmd import mkfs with mkfs["-t", "ext3", "/dev/sda1"] as p1: with mkfs["-t", "ext3", "/dev/sdb1"] as p2: pass .. note:: When processes run in the **background** (either via ``popen`` or :class:`& BG `), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see `#48 `_ For the arguments, see :func:`run `. :returns: A Popen object, augmented with a ``.run()`` method, which returns a tuple of (return code, stdout, stderr) """ retcode = kwargs.pop("retcode", 0) timeout = kwargs.pop("timeout", None) p = self.popen(args, **kwargs) was_run = [False] def runner(): if was_run[0]: return # already done was_run[0] = True try: return run_proc(p, retcode, timeout) finally: del p.run # to break cyclic reference p -> cell -> p for f in [p.stdin, p.stdout, p.stderr]: try: f.close() except Exception: pass p.run = runner yield p runner() def run(self, args = (), **kwargs): """Runs the given command (equivalent to popen() followed by :func:`run_proc `). If the exit code of the process does not match the expected one, :class:`ProcessExecutionError ` is raised. :param args: Any arguments to be passed to the process (a tuple) :param retcode: The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass ``None``. It may also be a tuple (or any iterable) of expected exit codes. .. note:: this argument must be passed as a keyword argument. :param timeout: The maximal amount of time (in seconds) to allow the process to run. ``None`` means no timeout is imposed; otherwise, if the process hasn't terminated after that many seconds, the process will be forcefully terminated an exception will be raised .. note:: this argument must be passed as a keyword argument. :param kwargs: Any keyword-arguments to be passed to the ``Popen`` constructor :returns: A tuple of (return code, stdout, stderr) """ with self.bgrun(args, **kwargs) as p: return p.run() class BoundCommand(BaseCommand): __slots__ = ["cmd", "args"] def __init__(self, cmd, args): self.cmd = cmd self.args = list(args) def __repr__(self): return "BoundCommand(%r, %r)" % (self.cmd, self.args) def _get_encoding(self): return self.cmd._get_encoding() def formulate(self, level = 0, args = ()): return self.cmd.formulate(level + 1, self.args + list(args)) @property def machine(self): return self.cmd.machine def popen(self, args = (), **kwargs): if isinstance(args, six.string_types): args = [args, ] return self.cmd.popen(self.args + list(args), **kwargs) class BoundEnvCommand(BaseCommand): __slots__ = ["cmd", "envvars"] def __init__(self, cmd, envvars): self.cmd = cmd self.envvars = envvars def __repr__(self): return "BoundEnvCommand(%r, %r)" % (self.cmd, self.envvars) def _get_encoding(self): return self.cmd._get_encoding() def formulate(self, level = 0, args = ()): return self.cmd.formulate(level, args) @property def machine(self): return self.cmd.machine def popen(self, args = (), **kwargs): with self.machine.env(**self.envvars): return self.cmd.popen(args, **kwargs) class Pipeline(BaseCommand): __slots__ = ["srccmd", "dstcmd"] def __init__(self, srccmd, dstcmd): self.srccmd = srccmd self.dstcmd = dstcmd def __repr__(self): return "Pipeline(%r, %r)" % (self.srccmd, self.dstcmd) def _get_encoding(self): return self.srccmd._get_encoding() or self.dstcmd._get_encoding() def formulate(self, level = 0, args = ()): return self.srccmd.formulate(level + 1) + ["|"] + self.dstcmd.formulate(level + 1, args) @property def machine(self): return self.srccmd.machine def popen(self, args = (), **kwargs): src_kwargs = kwargs.copy() src_kwargs["stdout"] = PIPE src_kwargs["stderr"] = PIPE srcproc = self.srccmd.popen(args, **src_kwargs) kwargs["stdin"] = srcproc.stdout dstproc = self.dstcmd.popen(**kwargs) # allow p1 to receive a SIGPIPE if p2 exits srcproc.stdout.close() srcproc.stderr.close() if srcproc.stdin: srcproc.stdin.close() dstproc.srcproc = srcproc # monkey-patch .wait() to wait on srcproc as well (it's expected to die when dstproc dies) dstproc_wait = dstproc.wait @functools.wraps(Popen.wait) def wait2(*args, **kwargs): rc_dst = dstproc_wait(*args, **kwargs) rc_src = srcproc.wait(*args, **kwargs) dstproc.returncode = rc_src or rc_dst return dstproc.returncode dstproc.wait = wait2 return dstproc class BaseRedirection(BaseCommand): __slots__ = ["cmd", "file"] SYM = None KWARG = None MODE = None def __init__(self, cmd, file): self.cmd = cmd self.file = file def _get_encoding(self): return self.cmd._get_encoding() def __repr__(self): return "%s(%r, %r)" % (self.__class__.__name__, self.cmd, self.file) def formulate(self, level = 0, args = ()): return self.cmd.formulate(level + 1, args) + [self.SYM, shquote(getattr(self.file, "name", self.file))] @property def machine(self): return self.cmd.machine def popen(self, args = (), **kwargs): from plumbum.machines.local import LocalPath from plumbum.machines.remote import RemotePath if self.KWARG in kwargs and kwargs[self.KWARG] not in (PIPE, None): raise RedirectionError("%s is already redirected" % (self.KWARG,)) if isinstance(self.file, six.string_types + (LocalPath,)): f = kwargs[self.KWARG] = open(str(self.file), self.MODE) elif isinstance(self.file, RemotePath): raise TypeError("Cannot redirect to/from remote paths") else: kwargs[self.KWARG] = self.file f = None try: return self.cmd.popen(args, **kwargs) finally: if f: f.close() class StdinRedirection(BaseRedirection): __slots__ = [] SYM = "<" KWARG = "stdin" MODE = "r" class StdoutRedirection(BaseRedirection): __slots__ = [] SYM = ">" KWARG = "stdout" MODE = "w" class AppendingStdoutRedirection(BaseRedirection): __slots__ = [] SYM = ">>" KWARG = "stdout" MODE = "a" class StderrRedirection(BaseRedirection): __slots__ = [] SYM = "2>" KWARG = "stderr" MODE = "w" class ERROUT(int): def __repr__(self): return "ERROUT" def __str__(self): return "&1" ERROUT = ERROUT(subprocess.STDOUT) class StdinDataRedirection(BaseCommand): __slots__ = ["cmd", "data"] CHUNK_SIZE = 16000 def __init__(self, cmd, data): self.cmd = cmd self.data = data def _get_encoding(self): return self.cmd._get_encoding() def formulate(self, level = 0, args = ()): return ["echo %s" % (shquote(self.data),), "|", self.cmd.formulate(level + 1, args)] @property def machine(self): return self.cmd.machine def popen(self, args = (), **kwargs): if "stdin" in kwargs and kwargs["stdin"] != PIPE: raise RedirectionError("stdin is already redirected") data = self.data if isinstance(data, six.unicode_type) and self._get_encoding() is not None: data = data.encode(self._get_encoding()) f = TemporaryFile() while data: chunk = data[:self.CHUNK_SIZE] f.write(chunk) data = data[self.CHUNK_SIZE:] f.seek(0) # try: return self.cmd.popen(args, stdin = f, **kwargs) # finally: # f.close() class ConcreteCommand(BaseCommand): QUOTE_LEVEL = None __slots__ = ["executable", "encoding"] def __init__(self, executable, encoding): self.executable = executable self.encoding = encoding self.cwd = None self.env = None def __str__(self): return str(self.executable) def __repr__(self): return "{0}({1})".format(type(self).__name__, self.executable) def _get_encoding(self): return self.encoding def formulate(self, level = 0, args = ()): argv = [str(self.executable)] for a in args: if not a and a != "": continue if isinstance(a, BaseCommand): if level >= self.QUOTE_LEVEL: argv.extend(shquote_list(a.formulate(level + 1))) else: argv.extend(a.formulate(level + 1)) elif isinstance(a, (list, tuple)): argv.extend(shquote(b) if level >= self.QUOTE_LEVEL else str(b) for b in a) else: argv.append(shquote(a) if level >= self.QUOTE_LEVEL else str(a)) # if self.encoding: # argv = [a.encode(self.encoding) for a in argv if isinstance(a, six.string_types)] return argv plumbum-1.6.0/plumbum/machines/0000755000232200023220000000000012610225742017016 5ustar debalancedebalanceplumbum-1.6.0/plumbum/machines/ssh_machine.py0000644000232200023220000003017612610225666021665 0ustar debalancedebalancefrom plumbum.lib import _setdoc, IS_WIN32 from plumbum.machines.remote import BaseRemoteMachine from plumbum.machines.session import ShellSession from plumbum.machines.local import local from plumbum.path.local import LocalPath from plumbum.path.remote import RemotePath from plumbum.commands import ProcessExecutionError, shquote import warnings class SshTunnel(object): """An object representing an SSH tunnel (created by :func:`SshMachine.tunnel `)""" __slots__ = ["_session"] def __init__(self, session): self._session = session def __repr__(self): if self._session.alive(): return "" % (self._session.proc,) else: return "" def __enter__(self): return self def __exit__(self, t, v, tb): self.close() def close(self): """Closes(terminates) the tunnel""" self._session.close() class SshMachine(BaseRemoteMachine): """ An implementation of :class:`remote machine ` over SSH. Invoking a remote command translates to invoking it over SSH :: with SshMachine("yourhostname") as rem: r_ls = rem["ls"] # r_ls is the remote `ls` # executing r_ls() translates to `ssh yourhostname ls` :param host: the host name to connect to (SSH server) :param user: the user to connect as (if ``None``, the default will be used) :param port: the server's port (if ``None``, the default will be used) :param keyfile: the path to the identity file (if ``None``, the default will be used) :param ssh_command: the ``ssh`` command to use; this has to be a ``Command`` object; if ``None``, the default ssh client will be used. :param scp_command: the ``scp`` command to use; this has to be a ``Command`` object; if ``None``, the default scp program will be used. :param ssh_opts: any additional options for ``ssh`` (a list of strings) :param scp_opts: any additional options for ``scp`` (a list of strings) :param password: the password to use; requires ``sshpass`` be installed. Cannot be used in conjunction with ``ssh_command`` or ``scp_command`` (will be ignored). NOTE: THIS IS A SECURITY RISK! :param encoding: the remote machine's encoding (defaults to UTF8) :param connect_timeout: specify a connection timeout (the time until shell prompt is seen). The default is 10 seconds. Set to ``None`` to disable :param new_session: whether or not to start the background session as a new session leader (setsid). This will prevent it from being killed on Ctrl+C (SIGINT) """ def __init__(self, host, user = None, port = None, keyfile = None, ssh_command = None, scp_command = None, ssh_opts = (), scp_opts = (), password = None, encoding = "utf8", connect_timeout = 10, new_session = False): if ssh_command is None: if password is not None: ssh_command = local["sshpass"]["-p", password, "ssh"] else: ssh_command = local["ssh"] if scp_command is None: if password is not None: scp_command = local["sshpass"]["-p", password, "scp"] else: scp_command = local["scp"] scp_args = [] ssh_args = [] if user: self._fqhost = "%s@%s" % (user, host) else: self._fqhost = host if port: ssh_args.extend(["-p", str(port)]) scp_args.extend(["-P", str(port)]) if keyfile: ssh_args.extend(["-i", str(keyfile)]) scp_args.extend(["-i", str(keyfile)]) scp_args.append("-r") ssh_args.extend(ssh_opts) scp_args.extend(scp_opts) self._ssh_command = ssh_command[tuple(ssh_args)] self._scp_command = scp_command[tuple(scp_args)] BaseRemoteMachine.__init__(self, encoding = encoding, connect_timeout = connect_timeout, new_session = new_session) def __str__(self): return "ssh://%s" % (self._fqhost,) @_setdoc(BaseRemoteMachine) def popen(self, args, ssh_opts = (), **kwargs): cmdline = [] cmdline.extend(ssh_opts) cmdline.append(self._fqhost) if args and hasattr(self, "env"): envdelta = self.env.getdelta() cmdline.extend(["cd", str(self.cwd), "&&"]) if envdelta: cmdline.append("env") cmdline.extend("%s=%s" % (k, v) for k, v in envdelta.items()) if isinstance(args, (tuple, list)): cmdline.extend(args) else: cmdline.append(args) return self._ssh_command[tuple(cmdline)].popen(**kwargs) def nohup(self, command): """ Runs the given command using ``nohup`` and redirects std handles, allowing the command to run "detached" from its controlling TTY or parent. Does not return anything. Depreciated (use command.nohup or daemonic_popen). """ warnings.warn("Use .nohup on the command or use daemonic_popen)", DeprecationWarning) self.daemonic_popen(command, cwd='.', stdout=None, stderr=None, append=False) def daemonic_popen(self, command, cwd='.', stdout=None, stderr=None, append=True): """ Runs the given command using ``nohup`` and redirects std handles, allowing the command to run "detached" from its controlling TTY or parent. Does not return anything. .. versionadded:: 1.6.0 """ if stdout is None: stdout = "/dev/null" if stderr is None: stderr = "&1" if str(cwd) == '.': args = [] else: args = ["cd", str(cwd), "&&"] args.append("nohup") args.extend(command.formulate()) args.extend([(">>" if append else ">")+str(stdout), "2"+(">>" if (append and stderr!="&1") else ">")+str(stderr), "` object can be used as a *context-manager*. The more conventional use case is the following:: +---------+ +---------+ | Your | | Remote | | Machine | | Machine | +----o----+ +---- ----+ | ^ | | lport dport | | \______SSH TUNNEL____/ (secure) Here, you wish to communicate safely between port ``lport`` of your machine and port ``dport`` of the remote machine. Communication is tunneled over SSH, so the connection is authenticated and encrypted. The more general case is shown below (where ``dport != "localhost"``):: +---------+ +-------------+ +-------------+ | Your | | Remote | | Destination | | Machine | | Machine | | Machine | +----o----+ +---- ----o---+ +---- --------+ | ^ | ^ | | | | lhost:lport | | dhost:dport | | | | \_____SSH TUNNEL_____/ \_____SOCKET____/ (secure) (not secure) Usage:: rem = SshMachine("megazord") with rem.tunnel(1234, 5678): sock = socket.socket() sock.connect(("localhost", 1234)) # sock is now tunneled to megazord:5678 """ ssh_opts = ["-L", "[%s]:%s:[%s]:%s" % (lhost, lport, dhost, dport)] proc = self.popen((), ssh_opts = ssh_opts, new_session = True) return SshTunnel(ShellSession(proc, self.encoding, connect_timeout = self.connect_timeout)) def _translate_drive_letter(self, path): # replace c:\some\path with /c/some/path path = str(path) if ":" in path: path = "/" + path.replace(":", "").replace("\\", "/") return path @_setdoc(BaseRemoteMachine) def download(self, src, dst): if isinstance(src, LocalPath): raise TypeError("src of download cannot be %r" % (src,)) if isinstance(src, RemotePath) and src.remote != self: raise TypeError("src %r points to a different remote machine" % (src,)) if isinstance(dst, RemotePath): raise TypeError("dst of download cannot be %r" % (dst,)) if IS_WIN32: src = self._translate_drive_letter(src) dst = self._translate_drive_letter(dst) self._scp_command("%s:%s" % (self._fqhost, shquote(src)), dst) @_setdoc(BaseRemoteMachine) def upload(self, src, dst): if isinstance(src, RemotePath): raise TypeError("src of upload cannot be %r" % (src,)) if isinstance(dst, LocalPath): raise TypeError("dst of upload cannot be %r" % (dst,)) if isinstance(dst, RemotePath) and dst.remote != self: raise TypeError("dst %r points to a different remote machine" % (dst,)) if IS_WIN32: src = self._translate_drive_letter(src) dst = self._translate_drive_letter(dst) self._scp_command(src, "%s:%s" % (self._fqhost, shquote(dst))) class PuttyMachine(SshMachine): """ PuTTY-flavored SSH connection. The programs ``plink`` and ``pscp`` are expected to be in the path (or you may provide your own ``ssh_command`` and ``scp_command``) Arguments are the same as for :class:`plumbum.machines.remote.SshMachine` """ def __init__(self, host, user = None, port = None, keyfile = None, ssh_command = None, scp_command = None, ssh_opts = (), scp_opts = (), encoding = "utf8", connect_timeout = 10, new_session = False): if ssh_command is None: ssh_command = local["plink"] if scp_command is None: scp_command = local["pscp"] if not ssh_opts: ssh_opts = ["-ssh"] if user is None: user = local.env.user if port is not None: ssh_opts.extend(["-P", str(port)]) port = None SshMachine.__init__(self, host, user, port, keyfile = keyfile, ssh_command = ssh_command, scp_command = scp_command, ssh_opts = ssh_opts, scp_opts = scp_opts, encoding = encoding, connect_timeout = connect_timeout, new_session = new_session) def __str__(self): return "putty-ssh://%s" % (self._fqhost,) def _translate_drive_letter(self, path): # pscp takes care of windows paths automatically return path @_setdoc(BaseRemoteMachine) def session(self, isatty = False, new_session = False): return ShellSession(self.popen((), (["-t"] if isatty else ["-T"]), new_session = new_session), self.encoding, isatty, self.connect_timeout) plumbum-1.6.0/plumbum/machines/__init__.py0000644000232200023220000000031412462734643021137 0ustar debalancedebalancefrom plumbum.machines.local import LocalCommand, LocalMachine, local from plumbum.machines.remote import BaseRemoteMachine, RemoteCommand from plumbum.machines.ssh_machine import SshMachine, PuttyMachine plumbum-1.6.0/plumbum/machines/session.py0000644000232200023220000002224512610225666021065 0ustar debalancedebalanceimport time import random import logging import threading from plumbum.commands import BaseCommand, run_proc from plumbum.lib import six class ShellSessionError(Exception): """Raises when something goes wrong when calling :func:`ShellSession.popen `""" pass class SSHCommsError(EOFError): """Raises when the communication channel can't be created on the remote host or it times out.""" class SSHCommsChannel2Error(SSHCommsError): """Raises when channel 2 (stderr) is not available""" shell_logger = logging.getLogger("plumbum.shell") #=================================================================================================== # Shell Session Popen #=================================================================================================== class MarkedPipe(object): """A pipe-like object from which you can read lines; the pipe will return report EOF (the empty string) when a special marker is detected""" __slots__ = ["pipe", "marker"] def __init__(self, pipe, marker): self.pipe = pipe self.marker = marker if six.PY3: self.marker = six.bytes(self.marker, "ascii") def close(self): """'Closes' the marked pipe; following calls to ``readline`` will return """"" # consume everything while self.readline(): pass self.pipe = None def readline(self): """Reads the next line from the pipe; returns "" when the special marker is reached. Raises ``EOFError`` if the underlying pipe has closed""" if self.pipe is None: return six.b("") line = self.pipe.readline() if not line: raise EOFError() if line.strip() == self.marker: self.pipe = None line = six.b("") return line class SessionPopen(object): """A shell-session-based ``Popen``-like object (has the following attributes: ``stdin``, ``stdout``, ``stderr``, ``returncode``)""" def __init__(self, argv, isatty, stdin, stdout, stderr, encoding): self.argv = argv self.isatty = isatty self.stdin = stdin self.stdout = stdout self.stderr = stderr self.encoding = encoding self.returncode = None self._done = False def poll(self): """Returns the process' exit code or ``None`` if it's still running""" if self._done: return self.returncode else: return None def wait(self): """Waits for the process to terminate and returns its exit code""" self.communicate() return self.returncode def communicate(self, input = None): """Consumes the process' stdout and stderr until the it terminates. :param input: An optional bytes/buffer object to send to the process over stdin :returns: A tuple of (stdout, stderr) """ stdout = [] stderr = [] sources = [("1", stdout, self.stdout)] if not self.isatty: # in tty mode, stdout and stderr are unified sources.append(("2", stderr, self.stderr)) i = 0 while sources: if input: chunk = input[:1000] self.stdin.write(chunk) self.stdin.flush() input = input[1000:] i = (i + 1) % len(sources) name, coll, pipe = sources[i] try: line = pipe.readline() shell_logger.debug("%s> %r", name, line) except EOFError: shell_logger.debug("%s> Nothing returned.", name) msg = "No communication channel detected. Does the remote exist?" msgerr = "No stderr result detected. Does the remote have Bash as the default shell?" raise SSHCommsChannel2Error(msgerr) if name=="2" else SSHCommsError(msg) if not line: del sources[i] else: coll.append(line) if self.isatty: stdout.pop(0) # discard first line of prompt try: self.returncode = int(stdout.pop(-1)) except (IndexError, ValueError): self.returncode = "Unknown" self._done = True stdout = six.b("").join(stdout) stderr = six.b("").join(stderr) return stdout, stderr class ShellSession(object): """An abstraction layer over *shell sessions*. A shell session is the execution of an interactive shell (``/bin/sh`` or something compatible), over which you may run commands (sent over stdin). The output of is then read from stdout and stderr. Shell sessions are less "robust" than executing a process on its own, and they are susseptible to all sorts of malformatted-strings attacks, and there is little benefit from using them locally. However, they can greatly speed up remote connections, and are required for the implementation of :class:`SshMachine `, as they allow us to send multiple commands over a single SSH connection (setting up separate SSH connections incurs a high overhead). Try to avoid using shell sessions, unless you know what you're doing. Instances of this class may be used as *context-managers*. :param proc: The underlying shell process (with open stdin, stdout and stderr) :param encoding: The encoding to use for the shell session. If ``"auto"``, the underlying process' encoding is used. :param isatty: If true, assume the shell has a TTY and that stdout and stderr are unified :param connect_timeout: The timeout to connect to the shell, after which, if no prompt is seen, the shell process is killed """ def __init__(self, proc, encoding = "auto", isatty = False, connect_timeout = 5): self.proc = proc self.encoding = proc.encoding if encoding == "auto" else encoding self.isatty = isatty self._current = None if connect_timeout: def closer(): shell_logger.error("Connection to %s timed out (%d sec)", proc, connect_timeout) self.close() timer = threading.Timer(connect_timeout, self.close) timer.start() self.run("") if connect_timeout: timer.cancel() def __enter__(self): return self def __exit__(self, t, v, tb): self.close() def __del__(self): try: self.close() except Exception: pass def alive(self): """Returns ``True`` if the underlying shell process is alive, ``False`` otherwise""" return self.proc and self.proc.poll() is None def close(self): """Closes (terminates) the shell session""" if not self.alive(): return try: self.proc.stdin.write(six.b("\nexit\n\n\nexit\n\n")) self.proc.stdin.flush() time.sleep(0.05) except (ValueError, EnvironmentError): pass for p in [self.proc.stdin, self.proc.stdout, self.proc.stderr]: try: p.close() except Exception: pass try: self.proc.kill() except EnvironmentError: pass self.proc = None def popen(self, cmd): """Runs the given command in the shell, adding some decoration around it. Only a single command can be executed at any given time. :param cmd: The command (string or :class:`Command ` object) to run :returns: A :class:`SessionPopen ` instance """ if self.proc is None: raise ShellSessionError("Shell session has already been closed") if self._current and not self._current._done: raise ShellSessionError("Each shell may start only one process at a time") if isinstance(cmd, BaseCommand): full_cmd = cmd.formulate(1) else: full_cmd = cmd marker = "--.END%s.--" % (time.time() * random.random(),) if full_cmd.strip(): full_cmd += " ; " else: full_cmd = "true ; " full_cmd += "echo $? ; echo '%s'" % (marker,) if not self.isatty: full_cmd += " ; echo '%s' 1>&2" % (marker,) if self.encoding: full_cmd = full_cmd.encode(self.encoding) shell_logger.debug("Running %r", full_cmd) self.proc.stdin.write(full_cmd + six.b("\n")) self.proc.stdin.flush() self._current = SessionPopen(full_cmd, self.isatty, self.proc.stdin, MarkedPipe(self.proc.stdout, marker), MarkedPipe(self.proc.stderr, marker), self.encoding) return self._current def run(self, cmd, retcode = 0): """Runs the given command :param cmd: The command (string or :class:`Command ` object) to run :param retcode: The expected return code (0 by default). Set to ``None`` in order to ignore erroneous return codes :returns: A tuple of (return code, stdout, stderr) """ return run_proc(self.popen(cmd), retcode) plumbum-1.6.0/plumbum/machines/local.py0000644000232200023220000003517712610225666020504 0ustar debalancedebalanceimport os import sys import subprocess import logging import time import platform import re from functools import partial from plumbum.path.local import LocalPath, LocalWorkdir from tempfile import mkdtemp from contextlib import contextmanager from plumbum.path.remote import RemotePath from plumbum.commands import CommandNotFound, ConcreteCommand from plumbum.machines.session import ShellSession from plumbum.lib import ProcInfo, IS_WIN32, six, StaticProperty from plumbum.commands.daemons import win32_daemonize, posix_daemonize from plumbum.commands.processes import iter_lines from plumbum.machines.base import BaseMachine from plumbum.machines.env import BaseEnv if sys.version_info >= (3, 2): # python 3.2 has the new-and-improved subprocess module from subprocess import Popen, PIPE has_new_subprocess = True else: # otherwise, see if we have subprocess32 try: from subprocess32 import Popen, PIPE has_new_subprocess = True except ImportError: from subprocess import Popen, PIPE has_new_subprocess = False class IterablePopen(Popen): iter_lines = iter_lines def __iter__(self): return self.iter_lines() logger = logging.getLogger("plumbum.local") #=================================================================================================== # Environment #=================================================================================================== class LocalEnv(BaseEnv): """The local machine's environment; exposes a dict-like interface""" __slots__ = [] CASE_SENSITIVE = not IS_WIN32 def __init__(self): # os.environ already takes care of upper'ing on windows self._curr = os.environ.copy() BaseEnv.__init__(self, LocalPath, os.path.pathsep) if IS_WIN32 and "HOME" not in self and self.home is not None: self["HOME"] = self.home def expand(self, expr): """Expands any environment variables and home shortcuts found in ``expr`` (like ``os.path.expanduser`` combined with ``os.path.expandvars``) :param expr: An expression containing environment variables (as ``$FOO``) or home shortcuts (as ``~/.bashrc``) :returns: The expanded string""" prev = os.environ os.environ = self.getdict() try: output = os.path.expanduser(os.path.expandvars(expr)) finally: os.environ = prev return output def expanduser(self, expr): """Expand home shortcuts (e.g., ``~/foo/bar`` or ``~john/foo/bar``) :param expr: An expression containing home shortcuts :returns: The expanded string""" prev = os.environ os.environ = self.getdict() try: output = os.path.expanduser(expr) finally: os.environ = prev return output #=================================================================================================== # Local Commands #=================================================================================================== class LocalCommand(ConcreteCommand): __slots__ = [] QUOTE_LEVEL = 2 def __init__(self, executable, encoding = "auto"): ConcreteCommand.__init__(self, executable, local.encoding if encoding == "auto" else encoding) @property def machine(self): return local def popen(self, args = (), cwd = None, env = None, **kwargs): if isinstance(args, six.string_types): args = (args,) return self.machine._popen(self.executable, self.formulate(0, args), cwd = self.cwd if cwd is None else cwd, env = self.env if env is None else env, **kwargs) #=================================================================================================== # Local Machine #=================================================================================================== class LocalMachine(BaseMachine): """The *local machine* (a singleton object). It serves as an entry point to everything related to the local machine, such as working directory and environment manipulation, command creation, etc. Attributes: * ``cwd`` - the local working directory * ``env`` - the local environment * ``encoding`` - the local machine's default encoding (``sys.getfilesystemencoding()``) """ cwd = StaticProperty(LocalWorkdir) env = LocalEnv() encoding = sys.getfilesystemencoding() uname = platform.uname()[0] def __init__(self): self._as_user_stack = [] if IS_WIN32: _EXTENSIONS = [""] + env.get("PATHEXT", ":.exe:.bat").lower().split(os.path.pathsep) @classmethod def _which(cls, progname): progname = progname.lower() for p in cls.env.path: for ext in cls._EXTENSIONS: fn = p / (progname + ext) if fn.access("x"): return fn return None else: @classmethod def _which(cls, progname): for p in cls.env.path: fn = p / progname if fn.access("x"): return fn return None @classmethod def which(cls, progname): """Looks up a program in the ``PATH``. If the program is not found, raises :class:`CommandNotFound ` :param progname: The program's name. Note that if underscores (``_``) are present in the name, and the exact name is not found, they will be replaced in turn by hyphens (``-``) then periods (``.``), and the name will be looked up again for each alternative :returns: A :class:`LocalPath ` """ alternatives = [progname] if "_" in progname: alternatives.append(progname.replace("_", "-")) alternatives.append(progname.replace("_", ".")) for pn in alternatives: path = cls._which(pn) if path: return path raise CommandNotFound(progname, list(cls.env.path)) def path(self, *parts): """A factory for :class:`LocalPaths `. Usage: ``p = local.path("/usr", "lib", "python2.7")`` """ parts2 = [str(self.cwd)] for p in parts: if isinstance(p, RemotePath): raise TypeError("Cannot construct LocalPath from %r" % (p,)) parts2.append(self.env.expanduser(str(p))) return LocalPath(os.path.join(*parts2)) def __getitem__(self, cmd): """Returns a `Command` object representing the given program. ``cmd`` can be a string or a :class:`LocalPath `; if it is a path, a command representing this path will be returned; otherwise, the program name will be looked up in the system's ``PATH`` (using ``which``). Usage:: ls = local["ls"] """ if isinstance(cmd, LocalPath): return LocalCommand(cmd) elif not isinstance(cmd, RemotePath): if "/" in cmd or "\\" in cmd: # assume path return LocalCommand(local.path(cmd)) else: # search for command return LocalCommand(self.which(cmd)) else: raise TypeError("cmd must not be a RemotePath: %r" % (cmd,)) def _popen(self, executable, argv, stdin = PIPE, stdout = PIPE, stderr = PIPE, cwd = None, env = None, new_session = False, **kwargs): if new_session: if has_new_subprocess: kwargs["start_new_session"] = True elif IS_WIN32: kwargs["creationflags"] = kwargs.get("creationflags", 0) | subprocess.CREATE_NEW_PROCESS_GROUP else: def preexec_fn(prev_fn = kwargs.get("preexec_fn", lambda: None)): os.setsid() prev_fn() kwargs["preexec_fn"] = preexec_fn if IS_WIN32 and "startupinfo" not in kwargs and stdin not in (sys.stdin, None): from plumbum.machines._windows import get_pe_subsystem, IMAGE_SUBSYSTEM_WINDOWS_CUI subsystem = get_pe_subsystem(str(executable)) if subsystem == IMAGE_SUBSYSTEM_WINDOWS_CUI: # don't open a new console sui = subprocess.STARTUPINFO() kwargs["startupinfo"] = sui if hasattr(subprocess, "_subprocess"): sui.dwFlags |= subprocess._subprocess.STARTF_USESHOWWINDOW # @UndefinedVariable sui.wShowWindow = subprocess._subprocess.SW_HIDE # @UndefinedVariable else: sui.dwFlags |= subprocess.STARTF_USESHOWWINDOW # @UndefinedVariable sui.wShowWindow = subprocess.SW_HIDE # @UndefinedVariable if not has_new_subprocess and "close_fds" not in kwargs: if IS_WIN32 and (stdin is not None or stdout is not None or stderr is not None): # we can't close fds if we're on windows and we want to redirect any std handle kwargs["close_fds"] = False else: kwargs["close_fds"] = True if cwd is None: cwd = self.cwd if env is None: env = self.env if isinstance(env, BaseEnv): env = env.getdict() if self._as_user_stack: argv, executable = self._as_user_stack[-1](argv) logger.debug("Running %r", argv) proc = IterablePopen(argv, executable = str(executable), stdin = stdin, stdout = stdout, stderr = stderr, cwd = str(cwd), env = env, **kwargs) # bufsize = 4096 proc._start_time = time.time() proc.encoding = self.encoding proc.argv = argv return proc def daemonic_popen(self, command, cwd = "/", stdout=None, stderr=None, append=True): """ On POSIX systems: Run ``command`` as a UNIX daemon: fork a child process to setpid, redirect std handles to /dev/null, umask, close all fds, chdir to ``cwd``, then fork and exec ``command``. Returns a ``Popen`` process that can be used to poll/wait for the executed command (but keep in mind that you cannot access std handles) On Windows: Run ``command`` as a "Windows daemon": detach from controlling console and create a new process group. This means that the command will not receive console events and would survive its parent's termination. Returns a ``Popen`` object. .. note:: this does not run ``command`` as a system service, only detaches it from its parent. .. versionadded:: 1.3 """ if IS_WIN32: return win32_daemonize(command, cwd, stdout, stderr, append) else: return posix_daemonize(command, cwd, stdout, stderr, append) if IS_WIN32: def list_processes(self): """ Returns information about all running processes (on Windows: using ``tasklist``) .. versionadded:: 1.3 """ import csv tasklist = local["tasklist"] lines = tasklist("/V", "/FO", "CSV").splitlines() rows = csv.reader(lines) header = next(rows) imgidx = header.index('Image Name') pididx = header.index('PID') statidx = header.index('Status') useridx = header.index('User Name') for row in rows: yield ProcInfo(int(row[pididx]), row[useridx], row[statidx], row[imgidx]) else: def list_processes(self): """ Returns information about all running processes (on POSIX systems: using ``ps``) .. versionadded:: 1.3 """ ps = self["ps"] lines = ps("-e", "-o", "pid,uid,stat,args").splitlines() lines.pop(0) # header for line in lines: parts = line.strip().split() yield ProcInfo(int(parts[0]), int(parts[1]), parts[2], " ".join(parts[3:])) def pgrep(self, pattern): """ Process grep: return information about all processes whose command-line args match the given regex pattern """ pat = re.compile(pattern) for procinfo in self.list_processes(): if pat.search(procinfo.args): yield procinfo def session(self, new_session = False): """Creates a new :class:`ShellSession ` object; this invokes ``/bin/sh`` and executes commands on it over stdin/stdout/stderr""" return ShellSession(self["sh"].popen(new_session = new_session)) @contextmanager def tempdir(self): """A context manager that creates a temporary directory, which is removed when the context exits""" dir = self.path(mkdtemp()) # @ReservedAssignment try: yield dir finally: dir.delete() @contextmanager def as_user(self, username = None): """Run nested commands as the given user. For example:: head = local["head"] head("-n1", "/dev/sda1") # this will fail... with local.as_user(): head("-n1", "/dev/sda1") :param username: The user to run commands as. If not given, root (or Administrator) is assumed """ if IS_WIN32: if username is None: username = "Administrator" self._as_user_stack.append(lambda argv: (["runas", "/savecred", "/user:%s" % (username,), '"' + " ".join(str(a) for a in argv) + '"'], self.which("runas"))) else: if username is None: self._as_user_stack.append(lambda argv: (["sudo"] + list(argv), self.which("sudo"))) else: self._as_user_stack.append(lambda argv: (["sudo", "-u", username] + list(argv), self.which("sudo"))) try: yield finally: self._as_user_stack.pop(-1) def as_root(self): """A shorthand for :func:`as_user("root") `""" return self.as_user() python = LocalCommand(sys.executable, encoding) """A command that represents the current python interpreter (``sys.executable``)""" local = LocalMachine() """The *local machine* (a singleton object). It serves as an entry point to everything related to the local machine, such as working directory and environment manipulation, command creation, etc. Attributes: * ``cwd`` - the local working directory * ``env`` - the local environment * ``encoding`` - the local machine's default encoding (``sys.getfilesystemencoding()``) """ plumbum-1.6.0/plumbum/machines/env.py0000644000232200023220000001375612176760322020201 0ustar debalancedebalanceimport os from contextlib import contextmanager class EnvPathList(list): __slots__ = ["_path_factory", "_pathsep"] def __init__(self, path_factory, pathsep): self._path_factory = path_factory self._pathsep = pathsep def append(self, path): list.append(self, self._path_factory(path)) def extend(self, paths): list.extend(self, (self._path_factory(p) for p in paths)) def insert(self, index, path): list.insert(self, index, self._path_factory(path)) def index(self, path): list.index(self, self._path_factory(path)) def __contains__(self, path): return list.__contains__(self, self._path_factory(path)) def remove(self, path): list.remove(self, self._path_factory(path)) def update(self, text): self[:] = [self._path_factory(p) for p in text.split(self._pathsep)] def join(self): return self._pathsep.join(str(p) for p in self) class BaseEnv(object): """The base class of LocalEnv and RemoteEnv""" __slots__ = ["_curr", "_path", "_path_factory"] CASE_SENSITIVE = True def __init__(self, path_factory, pathsep): self._path_factory = path_factory self._path = EnvPathList(path_factory, pathsep) self._update_path() def _update_path(self): self._path.update(self.get("PATH", "")) @contextmanager def __call__(self, *args, **kwargs): """A context manager that can be used for temporal modifications of the environment. Any time you enter the context, a copy of the old environment is stored, and then restored, when the context exits. :param args: Any positional arguments for ``update()`` :param kwargs: Any keyword arguments for ``update()`` """ prev = self._curr.copy() self.update(**kwargs) try: yield finally: self._curr = prev self._update_path() def __iter__(self): """Returns an iterator over the items ``(key, value)`` of current environment (like dict.items)""" return iter(self._curr.items()) def __hash__(self): raise TypeError("unhashable type") def __len__(self): """Returns the number of elements of the current environment""" return len(self._curr) def __contains__(self, name): """Tests whether an environment variable exists in the current environment""" return (name if self.CASE_SENSITIVE else name.upper()) in self._curr def __getitem__(self, name): """Returns the value of the given environment variable from current environment, raising a ``KeyError`` if it does not exist""" return self._curr[name if self.CASE_SENSITIVE else name.upper()] def keys(self): """Returns the keys of the current environment (like dict.keys)""" return self._curr.keys() def items(self): """Returns the items of the current environment (like dict.items)""" return self._curr.items() def values(self): """Returns the values of the current environment (like dict.values)""" return self._curr.values() def get(self, name, *default): """Returns the keys of the current environment (like dict.keys)""" return self._curr.get((name if self.CASE_SENSITIVE else name.upper()), *default) def __delitem__(self, name): """Deletes an environment variable from the current environment""" name = name if self.CASE_SENSITIVE else name.upper() del self._curr[name] if name == "PATH": self._update_path() def __setitem__(self, name, value): """Sets/replaces an environment variable's value in the current environment""" name = name if self.CASE_SENSITIVE else name.upper() self._curr[name] = value if name == "PATH": self._update_path() def pop(self, name, *default): """Pops an element from the current environment (like dict.pop)""" name = name if self.CASE_SENSITIVE else name.upper() res = self._curr.pop(name, *default) if name == "PATH": self._update_path() return res def clear(self): """Clears the current environment (like dict.clear)""" self._curr.clear() self._update_path() def update(self, *args, **kwargs): """Updates the current environment (like dict.update)""" self._curr.update(*args, **kwargs) if not self.CASE_SENSITIVE: for k, v in list(self._curr.items()): self._curr[k.upper()] = v self._update_path() def getdict(self): """Returns the environment as a real dictionary""" self._curr["PATH"] = self.path.join() return dict((k, str(v)) for k, v in self._curr.items()) @property def path(self): """The system's ``PATH`` (as an easy-to-manipulate list)""" return self._path def _get_home(self): if "HOME" in self: return self._path_factory(self["HOME"]) elif "USERPROFILE" in self: return self._path_factory(self["USERPROFILE"]) elif "HOMEPATH" in self: return self._path_factory(self.get("HOMEDRIVE", ""), self["HOMEPATH"]) return None def _set_home(self, p): if "HOME" in self: self["HOME"] = str(p) elif "USERPROFILE" in self: self["USERPROFILE"] = str(p) elif "HOMEPATH" in self: self["HOMEPATH"] = str(p) else: self["HOME"] = str(p) home = property(_get_home, _set_home) """Get or set the home path""" @property def user(self): """Return the user name, or ``None`` if it is not set""" # adapted from getpass.getuser() for name in ('LOGNAME', 'USER', 'LNAME', 'USERNAME'): if name in self: return self[name] try: # POSIX only import pwd except ImportError: return None else: return pwd.getpwuid(os.getuid())[0] # @UndefinedVariable plumbum-1.6.0/plumbum/machines/base.py0000644000232200023220000000307612610225666020315 0ustar debalancedebalancefrom plumbum.commands.processes import CommandNotFound class BaseMachine(object): """This is a base class for other machines. It contains common code to all machines in Plumbum.""" def get(self, cmd, *othercommands): """This works a little like the ``.get`` method with dict's, only it supports an unlimited number of arguments, since later arguments are tried as commands and could also fail. It will try to call the first command, and if that is not found, it will call the next, etc. Will raise if no file named for the executable if a path is given, unlike ``[]`` access. Usage:: best_zip = local.get('pigz','gzip') """ try: command = self[cmd] if not command.executable.exists(): raise CommandNotFound(cmd,command.executable) else: return command except CommandNotFound: if othercommands: return self.get(othercommands[0],*othercommands[1:]) else: raise def __contains__(self, cmd): """Tests for the existance of the command, e.g., ``"ls" in plumbum.local``. ``cmd`` can be anything acceptable by ``__getitem__``. """ try: self[cmd] except CommandNotFound: return False else: return True def daemonic_popen(self, command, cwd = "/", stdout=None, stderr=None, append=True): raise NotImplementedError("This is not implemented on this machine!") plumbum-1.6.0/plumbum/machines/remote.py0000644000232200023220000003553412610225666020702 0ustar debalancedebalanceimport re from contextlib import contextmanager from plumbum.commands import CommandNotFound, shquote, ConcreteCommand from plumbum.lib import _setdoc, ProcInfo, six from plumbum.machines.local import LocalPath from tempfile import NamedTemporaryFile from plumbum.machines.base import BaseMachine from plumbum.machines.env import BaseEnv from plumbum.path.remote import RemotePath, RemoteWorkdir, StatRes class RemoteEnv(BaseEnv): """The remote machine's environment; exposes a dict-like interface""" __slots__ = ["_orig", "remote"] def __init__(self, remote): self.remote = remote session = remote._session # GNU env has a -0 argument; use it if present. Otherwise, # fall back to calling printenv on each (possible) variable # from plain env. env0 = session.run("env -0; echo") if env0[0] == 0 and not env0[2].rstrip(): self._curr = dict(line.split('=', 1) for line in env0[1].split('\x00') if '=' in line) else: lines = session.run("env; echo")[1].splitlines() split = (line.split('=', 1) for line in lines) keys = (line[0] for line in split if len(line)>1) runs = ((key, session.run('printenv "%s"; echo' % key)) for key in keys) self._curr = dict((key, run[1].rstrip('\n')) for (key, run) in runs if run[0] == 0 and run[1].rstrip('\n') and not run[2]) self._orig = self._curr.copy() BaseEnv.__init__(self, self.remote.path, ":") @_setdoc(BaseEnv) def __delitem__(self, name): BaseEnv.__delitem__(self, name) self.remote._session.run("unset %s" % (name,)) @_setdoc(BaseEnv) def __setitem__(self, name, value): BaseEnv.__setitem__(self, name, value) self.remote._session.run("export %s=%s" % (name, shquote(value))) @_setdoc(BaseEnv) def pop(self, name, *default): BaseEnv.pop(self, name, *default) self.remote._session.run("unset %s" % (name,)) @_setdoc(BaseEnv) def update(self, *args, **kwargs): BaseEnv.update(self, *args, **kwargs) self.remote._session.run("export " + " ".join("%s=%s" % (k, shquote(v)) for k, v in self.getdict().items())) def expand(self, expr): """Expands any environment variables and home shortcuts found in ``expr`` (like ``os.path.expanduser`` combined with ``os.path.expandvars``) :param expr: An expression containing environment variables (as ``$FOO``) or home shortcuts (as ``~/.bashrc``) :returns: The expanded string""" return self.remote._session.run("echo %s" % (expr,))[1].strip() def expanduser(self, expr): """Expand home shortcuts (e.g., ``~/foo/bar`` or ``~john/foo/bar``) :param expr: An expression containing home shortcuts :returns: The expanded string""" if not any(part.startswith("~") for part in expr.split("/")): return expr # we escape all $ signs to avoid expanding env-vars return self.remote._session.run("echo %s" % (expr.replace("$", "\\$"),))[1].strip() # def clear(self): # BaseEnv.clear(self, *args, **kwargs) # self.remote._session.run("export %s" % " ".join("%s=%s" % (k, v) for k, v in self.getdict())) def getdelta(self): """Returns the difference between the this environment and the original environment of the remote machine""" self._curr["PATH"] = self.path.join() delta = {} for k, v in self._curr.items(): if k not in self._orig: delta[k] = str(v) for k, v in self._orig.items(): if k not in self._curr: delta[k] = "" else: if v != self._curr[k]: delta[k] = self._curr[k] return delta class RemoteCommand(ConcreteCommand): __slots__ = ["remote", "executable"] QUOTE_LEVEL = 1 def __init__(self, remote, executable, encoding = "auto"): self.remote = remote ConcreteCommand.__init__(self, executable, remote.encoding if encoding == "auto" else encoding) @property def machine(self): return self.remote def __repr__(self): return "RemoteCommand(%r, %r)" % (self.remote, self.executable) def popen(self, args = (), **kwargs): return self.remote.popen(self[args], **kwargs) def nohup(self, cwd='.', stdout='nohup.out', stderr=None, append=True): """Runs a command detached.""" return machine.nohup(self, cwd, stdout, stderr, append) class ClosedRemoteMachine(Exception): pass class ClosedRemote(object): __slots__ = ["_obj"] def __init__(self, obj): self._obj = obj def close(self): pass def __getattr__(self, name): raise ClosedRemoteMachine("%r has been closed" % (self._obj,)) class BaseRemoteMachine(BaseMachine): """Represents a *remote machine*; serves as an entry point to everything related to that remote machine, such as working directory and environment manipulation, command creation, etc. Attributes: * ``cwd`` - the remote working directory * ``env`` - the remote environment * ``encoding`` - the remote machine's default encoding (assumed to be UTF8) * ``connect_timeout`` - the connection timeout """ # allow inheritors to override the RemoteCommand class RemoteCommand = RemoteCommand @property def cwd(self): return RemoteWorkdir(self) def __init__(self, encoding = "utf8", connect_timeout = 10, new_session = False): self.encoding = encoding self.connect_timeout = connect_timeout self._session = self.session(new_session = new_session) self.uname = self._get_uname() self.env = RemoteEnv(self) self._python = None def _get_uname(self): rc, out, _ = self._session.run("uname", retcode = None) if rc == 0: return out.strip() else: rc, out, _ = self._session.run("python -c 'import platform;print(platform.uname()[0])'", retcode = None) if rc == 0: return out.strip() else: # all POSIX systems should have uname. make an educated guess it's Windows return "Windows" def __repr__(self): return "<%s %s>" % (self.__class__.__name__, self) def __enter__(self): return self def __exit__(self, t, v, tb): self.close() def close(self): """closes the connection to the remote machine; all paths and programs will become defunct""" self._session.close() self._session = ClosedRemote(self) def path(self, *parts): """A factory for :class:`RemotePaths `. Usage: ``p = rem.path("/usr", "lib", "python2.7")`` """ parts2 = [str(self.cwd)] for p in parts: if isinstance(p, LocalPath): raise TypeError("Cannot construct RemotePath from %r" % (p,)) p = str(p) if "~" in p: p = self.env.expanduser(p) parts2.append(p) return RemotePath(self, *parts2) def which(self, progname): """Looks up a program in the ``PATH``. If the program is not found, raises :class:`CommandNotFound ` :param progname: The program's name. Note that if underscores (``_``) are present in the name, and the exact name is not found, they will be replaced in turn by hyphens (``-``) then periods (``.``), and the name will be looked up again for each alternative :returns: A :class:`RemotePath ` """ alternatives = [progname] if "_" in progname: alternatives.append(progname.replace("_", "-")) alternatives.append(progname.replace("_", ".")) for name in alternatives: for p in self.env.path: fn = p / name if fn.access("x"): return fn raise CommandNotFound(progname, self.env.path) def __getitem__(self, cmd): """Returns a `Command` object representing the given program. ``cmd`` can be a string or a :class:`RemotePath `; if it is a path, a command representing this path will be returned; otherwise, the program name will be looked up in the system's ``PATH`` (using ``which``). Usage:: r_ls = rem["ls"] """ if isinstance(cmd, RemotePath): if cmd.remote is self: return self.RemoteCommand(self, cmd) else: raise TypeError("Given path does not belong to this remote machine: %r" % (cmd,)) elif not isinstance(cmd, LocalPath): if "/" in cmd or "\\" in cmd: return self.RemoteCommand(self, self.path(cmd)) else: return self.RemoteCommand(self, self.which(cmd)) else: raise TypeError("cmd must not be a LocalPath: %r" % (cmd,)) @property def python(self): """A command that represents the default remote python interpreter""" if not self._python: self._python = self["python"] return self._python def session(self, isatty = False, new_session = False): """Creates a new :class:`ShellSession ` object; this invokes the user's shell on the remote machine and executes commands on it over stdin/stdout/stderr""" raise NotImplementedError() def download(self, src, dst): """Downloads a remote file/directory (``src``) to a local destination (``dst``). ``src`` must be a string or a :class:`RemotePath ` pointing to this remote machine, and ``dst`` must be a string or a :class:`LocalPath `""" raise NotImplementedError() def upload(self, src, dst): """Uploads a local file/directory (``src``) to a remote destination (``dst``). ``src`` must be a string or a :class:`LocalPath `, and ``dst`` must be a string or a :class:`RemotePath ` pointing to this remote machine""" raise NotImplementedError() def popen(self, args, **kwargs): """Spawns the given command on the remote machine, returning a ``Popen``-like object; do not use this method directly, unless you need "low-level" control on the remote process""" raise NotImplementedError() def list_processes(self): """ Returns information about all running processes (on POSIX systems: using ``ps``) .. versionadded:: 1.3 """ ps = self["ps"] lines = ps("-e", "-o", "pid,uid,stat,args").splitlines() lines.pop(0) # header for line in lines: parts = line.strip().split() yield ProcInfo(int(parts[0]), int(parts[1]), parts[2], " ".join(parts[3:])) def pgrep(self, pattern): """ Process grep: return information about all processes whose command-line args match the given regex pattern """ pat = re.compile(pattern) for procinfo in self.list_processes(): if pat.search(procinfo.args): yield procinfo @contextmanager def tempdir(self): """A context manager that creates a remote temporary directory, which is removed when the context exits""" _, out, _ = self._session.run("mktemp -d tmp.XXXXXXXXXX") dir = self.path(out.strip()) # @ReservedAssignment try: yield dir finally: dir.delete() # # Path implementation # def _path_listdir(self, fn): files = self._session.run("ls -a %s" % (shquote(fn),))[1].splitlines() files.remove(".") files.remove("..") return files def _path_glob(self, fn, pattern): matches = self._session.run("for fn in %s/%s; do echo $fn; done" % (fn, pattern))[1].splitlines() if len(matches) == 1 and not self._path_stat(matches[0]): return [] # pattern expansion failed return matches def _path_getuid(self, fn): stat_cmd = "stat -c '%u,%U' " if self.uname not in ('Darwin', 'FreeBSD') else "stat -f '%u,%Su' " return self._session.run(stat_cmd + shquote(fn))[1].strip().split(",") def _path_getgid(self, fn): stat_cmd = "stat -c '%g,%G' " if self.uname not in ('Darwin', 'FreeBSD') else "stat -f '%g,%Sg' " return self._session.run(stat_cmd + shquote(fn))[1].strip().split(",") def _path_stat(self, fn): if self.uname not in ('Darwin', 'FreeBSD'): stat_cmd = "stat -c '%F,%f,%i,%d,%h,%u,%g,%s,%X,%Y,%Z' " else: stat_cmd = "stat -f '%HT,%Xp,%i,%d,%l,%u,%g,%z,%a,%m,%c' " rc, out, _ = self._session.run(stat_cmd + shquote(fn), retcode = None) if rc != 0: return None statres = out.strip().split(",") text_mode = statres.pop(0).lower() res = StatRes((int(statres[0], 16),) + tuple(int(sr) for sr in statres[1:])) res.text_mode = text_mode return res def _path_delete(self, fn): self._session.run("rm -rf %s" % (shquote(fn),)) def _path_move(self, src, dst): self._session.run("mv %s %s" % (shquote(src), shquote(dst))) def _path_copy(self, src, dst): self._session.run("cp -r %s %s" % (shquote(src), shquote(dst))) def _path_mkdir(self, fn): self._session.run("mkdir -p %s" % (shquote(fn),)) def _path_chmod(self, mode, fn): self._session.run("chmod %o %s" % (mode, shquote(fn))) def _path_chown(self, fn, owner, group, recursive): args = ["chown"] if recursive: args.append("-R") if owner is not None and group is not None: args.append("%s:%s" % (owner, group)) elif owner is not None: args.append(str(owner)) elif group is not None: args.append(":%s" % (group,)) args.append(shquote(fn)) self._session.run(" ".join(args)) def _path_read(self, fn): data = self["cat"](fn) if self.encoding and isinstance(data, six.unicode_type): data = data.encode(self.encoding) return data def _path_write(self, fn, data): if self.encoding and isinstance(data, six.unicode_type): data = data.encode(self.encoding) with NamedTemporaryFile() as f: f.write(data) f.flush() f.seek(0) self.upload(f.name, fn) def _path_link(self, src, dst, symlink): self._session.run("ln %s %s %s" % ("-s" if symlink else "", shquote(src), shquote(dst))) plumbum-1.6.0/plumbum/machines/_windows.py0000644000232200023220000000143612462734643021237 0ustar debalancedebalanceimport struct from plumbum.lib import six LFANEW_OFFSET = 30 * 2 FILE_HEADER_SIZE = 5 * 4 SUBSYSTEM_OFFSET = 17 * 4 IMAGE_SUBSYSTEM_WINDOWS_GUI = 2 IMAGE_SUBSYSTEM_WINDOWS_CUI = 3 def get_pe_subsystem(filename): with open(filename, "rb") as f: if f.read(2) != six.b("MZ"): return None f.seek(LFANEW_OFFSET) lfanew = struct.unpack("L", f.read(4))[0] f.seek(lfanew) if f.read(4) != six.b("PE\x00\x00"): return None f.seek(FILE_HEADER_SIZE + SUBSYSTEM_OFFSET, 1) subsystem = struct.unpack("H", f.read(2))[0] return subsystem # print(get_pe_subsystem("c:\\windows\\notepad.exe")) == 2 # print(get_pe_subsystem("c:\\python32\\python.exe")) == 3 # print(get_pe_subsystem("c:\\python32\\pythonw.exe")) == 2 plumbum-1.6.0/plumbum/machines/paramiko_machine.py0000644000232200023220000003570712610225666022700 0ustar debalancedebalanceimport logging import errno import stat import socket from plumbum.machines.remote import BaseRemoteMachine from plumbum.machines.session import ShellSession from plumbum.lib import _setdoc, six from plumbum.path.local import LocalPath from plumbum.path.remote import RemotePath, StatRes from plumbum.commands.processes import iter_lines try: # Sigh... we need to gracefully-import paramiko for Sphinx builds, etc import paramiko except ImportError: class paramiko(object): def __nonzero__(self): return False __bool__ = __nonzero__ def __getattr__(self, name): raise ImportError("No module named paramiko") paramiko = paramiko() logger = logging.getLogger("plumbum.paramiko") class ParamikoPopen(object): def __init__(self, argv, stdin, stdout, stderr, encoding, stdin_file = None, stdout_file = None, stderr_file = None): self.argv = argv self.channel = stdout.channel self.stdin = stdin self.stdout = stdout self.stderr = stderr self.encoding = encoding self.returncode = None self.pid = None self.stdin_file = stdin_file self.stdout_file = stdout_file self.stderr_file = stderr_file def poll(self): if self.returncode is None: if self.channel.exit_status_ready(): return self.wait() return self.returncode def wait(self): if self.returncode is None: self.channel.recv_exit_status() self.returncode = self.channel.exit_status self.close() return self.returncode def close(self): self.channel.shutdown_read() self.channel.shutdown_write() self.channel.close() def kill(self): # possible way to obtain pid: # "(cmd ; echo $?) & echo ?!" # and then client.exec_command("kill -9 %s" % (pid,)) raise EnvironmentError("Cannot kill remote processes, we don't have their PIDs") terminate = kill def send_signal(self, sig): raise NotImplementedError() def communicate(self): stdout = [] stderr = [] infile = self.stdin_file sources = [("1", stdout, self.stdout, self.stdout_file), ("2", stderr, self.stderr, self.stderr_file)] i = 0 while sources: if infile: try: line = infile.readline() except (ValueError, IOError): line = None logger.debug("communicate: %r", line) if not line: infile.close() infile = None self.stdin.close() else: self.stdin.write(line) self.stdin.flush() i = (i + 1) % len(sources) name, coll, pipe, outfile = sources[i] line = pipe.readline() # logger.debug("%s> %r", name, line) if not line: del sources[i] elif outfile: outfile.write(line) outfile.flush() else: coll.append(line) self.wait() stdout = six.b("").join(six.b(s) for s in stdout) stderr = six.b("").join(six.b(s) for s in stderr) return stdout, stderr def iter_lines(self, timeout=None, **kwargs): if timeout is not None: raise NotImplementedError("The 'timeout' parameter is not supported with ParamikoMachine") return iter_lines(self, _iter_lines=_iter_lines, **kwargs) __iter__ = iter_lines class ParamikoMachine(BaseRemoteMachine): """ An implementation of :class:`remote machine ` over Paramiko (a Python implementation of openSSH2 client/server). Invoking a remote command translates to invoking it over SSH :: with ParamikoMachine("yourhostname") as rem: r_ls = rem["ls"] # r_ls is the remote `ls` # executing r_ls() is equivalent to `ssh yourhostname ls`, only without # spawning a new ssh client :param host: the host name to connect to (SSH server) :param user: the user to connect as (if ``None``, the default will be used) :param port: the server's port (if ``None``, the default will be used) :param password: the user's password (if a password-based authentication is to be performed) (if ``None``, key-based authentication will be used) :param keyfile: the path to the identity file (if ``None``, the default will be used) :param load_system_host_keys: whether or not to load the system's host keys (from ``/etc/ssh`` and ``~/.ssh``). The default is ``True``, which means Paramiko behaves much like the ``ssh`` command-line client :param missing_host_policy: the value passed to the underlying ``set_missing_host_key_policy`` of the client. The default is ``None``, which means ``set_missing_host_key_policy`` is not invoked and paramiko's default behavior (reject) is employed :param encoding: the remote machine's encoding (defaults to UTF8) :param look_for_keys: set to False to disable searching for discoverable private key files in ``~/.ssh`` :param connect_timeout: timeout for TCP connection """ class RemoteCommand(BaseRemoteMachine.RemoteCommand): def __or__(self, *_): raise NotImplementedError("Not supported with ParamikoMachine") def __gt__(self, *_): raise NotImplementedError("Not supported with ParamikoMachine") def __rshift__(self, *_): raise NotImplementedError("Not supported with ParamikoMachine") def __ge__(self, *_): raise NotImplementedError("Not supported with ParamikoMachine") def __lt__(self, *_): raise NotImplementedError("Not supported with ParamikoMachine") def __lshift__(self, *_): raise NotImplementedError("Not supported with ParamikoMachine") def __init__(self, host, user = None, port = None, password = None, keyfile = None, load_system_host_keys = True, missing_host_policy = None, encoding = "utf8", look_for_keys = None, connect_timeout = None, keep_alive = 0): self.host = host kwargs = {} if user: self._fqhost = "%s@%s" % (user, host) kwargs['username'] = user else: self._fqhost = host self._client = paramiko.SSHClient() if load_system_host_keys: self._client.load_system_host_keys() if port is not None: kwargs["port"] = port if keyfile is not None: kwargs["key_filename"] = keyfile if password is not None: kwargs["password"] = password if missing_host_policy is not None: self._client.set_missing_host_key_policy(missing_host_policy) if look_for_keys is not None: kwargs["look_for_keys"] = look_for_keys if connect_timeout is not None: kwargs["timeout"] = connect_timeout self._client.connect(host, **kwargs) self._keep_alive = keep_alive self._sftp = None BaseRemoteMachine.__init__(self, encoding, connect_timeout) def __str__(self): return "paramiko://%s" % (self._fqhost,) def close(self): BaseRemoteMachine.close(self) self._client.close() @property def sftp(self): """ Returns an SFTP client on top of the current SSH connection; it can be used to manipulate files directly, much like an interactive FTP/SFTP session """ if not self._sftp: self._sftp = self._client.open_sftp() return self._sftp @_setdoc(BaseRemoteMachine) def session(self, isatty = False, term = "vt100", width = 80, height = 24, new_session = False): # new_session is ignored for ParamikoMachine trans = self._client.get_transport() trans.set_keepalive(self._keep_alive) chan = trans.open_session() if isatty: chan.get_pty(term, width, height) chan.set_combine_stderr() chan.invoke_shell() stdin = chan.makefile('wb', -1) stdout = chan.makefile('rb', -1) stderr = chan.makefile_stderr('rb', -1) proc = ParamikoPopen([""], stdin, stdout, stderr, self.encoding) return ShellSession(proc, self.encoding, isatty) @_setdoc(BaseRemoteMachine) def popen(self, args, stdin = None, stdout = None, stderr = None, new_session = False, cwd = None): # new_session is ignored for ParamikoMachine argv = [] envdelta = self.env.getdelta() argv.extend(["cd", str(cwd or self.cwd), "&&"]) if envdelta: argv.append("env") argv.extend("%s=%s" % (k, v) for k, v in envdelta.items()) argv.extend(args.formulate()) cmdline = " ".join(argv) logger.debug(cmdline) si, so, se = streams = self._client.exec_command(cmdline, 1) return ParamikoPopen(argv, si, so, se, self.encoding, stdin_file = stdin, stdout_file = stdout, stderr_file = stderr) @_setdoc(BaseRemoteMachine) def download(self, src, dst): if isinstance(src, LocalPath): raise TypeError("src of download cannot be %r" % (src,)) if isinstance(src, RemotePath) and src.remote != self: raise TypeError("src %r points to a different remote machine" % (src,)) if isinstance(dst, RemotePath): raise TypeError("dst of download cannot be %r" % (dst,)) return self._download(src if isinstance(src, RemotePath) else self.path(src), dst if isinstance(dst, LocalPath) else LocalPath(dst)) def _download(self, src, dst): if src.is_dir(): if not dst.exists(): self.sftp.mkdir(str(dst)) for fn in src: self._download(fn, dst / fn.name) elif dst.is_dir(): self.sftp.get(str(src), str(dst / src.name)) else: self.sftp.get(str(src), str(dst)) @_setdoc(BaseRemoteMachine) def upload(self, src, dst): if isinstance(src, RemotePath): raise TypeError("src of upload cannot be %r" % (src,)) if isinstance(dst, LocalPath): raise TypeError("dst of upload cannot be %r" % (dst,)) if isinstance(dst, RemotePath) and dst.remote != self: raise TypeError("dst %r points to a different remote machine" % (dst,)) return self._upload(src if isinstance(src, LocalPath) else LocalPath(src), dst if isinstance(dst, RemotePath) else self.path(dst)) def _upload(self, src, dst): if src.is_dir(): if not dst.exists(): self.sftp.mkdir(str(dst)) for fn in src: self._upload(fn, dst / fn.name) elif dst.is_dir(): self.sftp.put(str(src), str(dst / src.name)) else: self.sftp.put(str(src), str(dst)) def connect_sock(self, dport, dhost = "localhost", ipv6 = False): """Returns a Paramiko ``Channel``, connected to dhost:dport on the remote machine. The ``Channel`` behaves like a regular socket; you can ``send`` and ``recv`` on it and the data will pass encrypted over SSH. Usage:: mach = ParamikoMachine("myhost") sock = mach.connect_sock(12345) data = sock.recv(100) sock.send("foobar") sock.close() """ if ipv6 and dhost == "localhost": dhost = "::1" srcaddr = ("::1", 0, 0, 0) if ipv6 else ("127.0.0.1", 0) trans = self._client.get_transport() trans.set_keepalive(self._keep_alive) chan = trans.open_channel('direct-tcpip', (dhost, dport), srcaddr) return SocketCompatibleChannel(chan) # # Path implementation # def _path_listdir(self, fn): return self.sftp.listdir(str(fn)) def _path_read(self, fn): f = self.sftp.open(str(fn), 'rb') data = f.read() f.close() return data def _path_write(self, fn, data): if self.encoding and isinstance(data, six.unicode_type): data = data.encode(self.encoding) f = self.sftp.open(str(fn), 'wb') f.write(data) f.close() def _path_stat(self, fn): try: st = self.sftp.stat(str(fn)) except IOError as e: if e.errno == errno.ENOENT: return None raise OSError(e.errno) res = StatRes((st.st_mode, 0, 0, 0, st.st_uid, st.st_gid, st.st_size, st.st_atime, st.st_mtime, 0)) if stat.S_ISDIR(st.st_mode): res.text_mode = 'directory' if stat.S_ISREG(st.st_mode): res.text_mode = 'regular file' return res ################################################################################################### # Make paramiko.Channel adhere to the socket protocol, namely, send and recv should fail # when the socket has been closed ################################################################################################### class SocketCompatibleChannel(object): def __init__(self, chan): self._chan = chan def __getattr__(self, name): return getattr(self._chan, name) def send(self, s): if self._chan.closed: raise socket.error(errno.EBADF, 'Bad file descriptor') return self._chan.send(s) def recv(self, count): if self._chan.closed: raise socket.error(errno.EBADF, 'Bad file descriptor') return self._chan.recv(count) ################################################################################################### # Custom iter_lines for paramiko.Channel ################################################################################################### def _iter_lines(proc, decode, linesize): try: from selectors import DefaultSelector, EVENT_READ except ImportError: # Pre Python 3.4 implementation from select import select def selector(): while True: rlist, _, _ = select([proc.stdout.channel], [], []) for _ in rlist: yield else: # Python 3.4 implementation def selector(): sel = DefaultSelector() sel.register(proc.stdout.channel, EVENT_READ) while True: for key, mask in sel.select(): yield for _ in selector(): if proc.stdout.channel.recv_ready(): yield 0, decode(six.b(proc.stdout.readline(linesize))) if proc.stdout.channel.recv_stderr_ready(): yield 1, decode(six.b(proc.stderr.readline(linesize))) if proc.poll() is not None: break for line in proc.stdout: yield 0, decode(six.b(line)) for line in proc.stderr: yield 1, decode(six.b(line)) plumbum-1.6.0/plumbum/colorlib/0000755000232200023220000000000012610225742017034 5ustar debalancedebalanceplumbum-1.6.0/plumbum/colorlib/__init__.py0000644000232200023220000000172512610225666021157 0ustar debalancedebalance"""\ The ``ansicolor`` object provides ``bg`` and ``fg`` to access colors, and attributes like bold and underlined text. It also provides ``reset`` to recover the normal font. """ from plumbum.colorlib.factories import StyleFactory from plumbum.colorlib.styles import Style, ANSIStyle, HTMLStyle, ColorNotFound ansicolors = StyleFactory(ANSIStyle) htmlcolors = StyleFactory(HTMLStyle) def load_ipython_extension(ipython): try: from plumbum.colorlib._ipython_ext import OutputMagics except ImportError: print("IPython required for the IPython extension to be loaded.") raise ipython.push({"colors":htmlcolors}) ipython.register_magics(OutputMagics) def main(): """Color changing script entry. Call using python -m plumbum.colors, will reset if no arguments given.""" import sys color = ' '.join(sys.argv[1:]) if len(sys.argv) > 1 else '' ansicolors.use_color=True ansicolors.get_colors_from_string(color).now() plumbum-1.6.0/plumbum/colorlib/styles.py0000644000232200023220000006036612610225666020751 0ustar debalancedebalance""" This file provides two classes, `Color` and `Style`. ``Color`` is rarely used directly, but merely provides the workhorse for finding and manipulating colors. With the ``Style`` class, any color can be directly called or given to a with statement. """ from __future__ import print_function import sys import os import re from copy import copy from plumbum.colorlib.names import color_names, color_html from plumbum.colorlib.names import color_codes_simple, from_html from plumbum.colorlib.names import FindNearest, attributes_ansi from plumbum.lib import six from plumbum import local from abc import abstractmethod import platform ABC = six.ABC __all__ = ['Color', 'Style', 'ANSIStyle', 'HTMLStyle', 'ColorNotFound', 'AttributeNotFound'] _lower_camel_names = [n.replace('_', '') for n in color_names] def get_color_repr(): """Gets best colors for current system.""" if not sys.stdout.isatty(): return False term =local.env.get("TERM", "") # Some terminals set TERM=xterm for compatibility if term == "xterm-256color" or term == "xterm": return 3 if platform.system() == 'Darwin' else 4 elif term == "xterm-16color": return 2 elif os.name == 'nt': return 1 else: return False class ColorNotFound(Exception): """Thrown when a color is not valid for a particular method.""" pass class AttributeNotFound(Exception): """Similar to color not found, only for attributes.""" pass class ResetNotSupported(Exception): """An exception indicating that Reset is not available for this Style.""" pass class Color(ABC): """\ Loaded with ``(r, g, b, fg)`` or ``(color, fg=fg)``. The second signature is a short cut and will try full and hex loading. This class stores the idea of a color, rather than a specific implementation. It provides as many different tools for representations as possible, and can be subclassed to add more representations, though that should not be needed for most situations. ``.from_`` class methods provide quick ways to create colors given different representations. You will not usually interact with this class. Possible colors:: reset = Color() # The reset color by default background_reset = Color(fg=False) # Can be a background color blue = Color(0,0,255) # Red, Green, Blue green = Color.from_full("green") # Case insensitive name, from large colorset red = Color.from_full(1) # Color number white = Color.from_html("#FFFFFF") # HTML supported yellow = Color.from_simple("red") # Simple colorset The attributes are: .. data:: reset True it this is a reset color (following attributes don't matter if True) .. data:: rgb The red/green/blue tuple for this color .. data:: simple If true will stay to 16 color mode. .. data:: number The color number given the mode, closest to rgb if not rgb not exact, gives position of closest name. .. data:: fg This is a foreground color if True. Background color if False. """ __slots__ = ('fg', 'isreset', 'rgb', 'number', 'representation', 'exact') def __init__(self, r_or_color=None, g=None, b=None, fg=True): """This works from color values, or tries to load non-simple ones.""" if isinstance(r_or_color, type(self)): for item in ('fg', 'isreset', 'rgb', 'number', 'representation', 'exact'): setattr(self, item, getattr(r_or_color, item)) return self.fg = fg self.isreset = True # Starts as reset color self.rgb = (0,0,0) self.number = None 'Number of the original color, or closest color' self.representation = 4 '0 for off, 1 for 8 colors, 2 for 16 colors, 3 for 256 colors, 4 for true color' self.exact = True 'This is false if the named color does not match the real color' if None in (g,b): if not r_or_color: return try: self._from_simple(r_or_color) except ColorNotFound: try: self._from_full(r_or_color) except ColorNotFound: self._from_hex(r_or_color) elif None not in (r_or_color, g, b): self.rgb = (r_or_color,g,b) self._init_number() else: raise ColorNotFound("Invalid parameters for a color!") def _init_number(self): """Should always be called after filling in r, g, b, and representation. Color will not be a reset color anymore.""" if self.representation in (0, 1): number = FindNearest(*self.rgb).only_basic() elif self.representation == 2: number = FindNearest(*self.rgb).only_simple() elif self.representation in (3, 4): number = FindNearest(*self.rgb).all_fast() if self.number is None: self.number = number self.isreset = False self.exact = self.rgb == from_html(color_html[self.number]) if not self.exact: self.number = number @classmethod def from_simple(cls, color, fg=True): """Creates a color from simple name or color number""" self = cls(fg=fg) self._from_simple(color) return self def _from_simple(self, color): try: color = color.lower() color = color.replace(' ','') color = color.replace('_','') except AttributeError: pass if color == 'reset': return elif color in _lower_camel_names[:16]: self.number = _lower_camel_names.index(color) self.rgb = from_html(color_html[self.number]) elif isinstance(color, int) and 0 <= color < 16: self.number = color self.rgb = from_html(color_html[color]) else: raise ColorNotFound("Did not find color: " + repr(color)) self.representation = 2 self._init_number() @classmethod def from_full(cls, color, fg=True): """Creates a color from full name or color number""" self = cls(fg=fg) self._from_full(color) return self def _from_full(self, color): try: color = color.lower() color = color.replace(' ','') color = color.replace('_','') except AttributeError: pass if color == 'reset': return elif color in _lower_camel_names: self.number = _lower_camel_names.index(color) self.rgb = from_html(color_html[self.number]) elif isinstance(color, int) and 0 <= color <= 255: self.number = color self.rgb = from_html(color_html[color]) else: raise ColorNotFound("Did not find color: " + repr(color)) self.representation = 3 self._init_number() @classmethod def from_hex(cls, color, fg=True): """Converts #123456 values to colors.""" self = cls(fg=fg) self._from_hex(color) return self def _from_hex(self, color): try: self.rgb = from_html(color) except (TypeError, ValueError): raise ColorNotFound("Did not find htmlcode: " + repr(color)) self.representation = 4 self._init_number() @property def name(self): """The (closest) name of the current color""" if self.isreset: return 'reset' else: return color_names[self.number] @property def name_camelcase(self): """The camelcase name of the color""" return self.name.replace("_", " ").title().replace(" ","") def __repr__(self): """This class has a smart representation that shows name and color (if not unique).""" name = ['Deactivated:', ' Basic:', '', ' Full:', ' True:'][self.representation] name += '' if self.fg else ' Background' name += ' ' + self.name_camelcase name += '' if self.exact else ' ' + self.hex_code return name[1:] def __eq__(self, other): """Reset colors are equal, otherwise rgb have to match.""" if self.isreset: return other.isreset else: return self.rgb == other.rgb @property def ansi_sequence(self): """This is the ansi sequence as a string, ready to use.""" return '\033[' + ';'.join(map(str, self.ansi_codes)) + 'm' @property def ansi_codes(self): """This is the full ANSI code, can be reset, simple, 256, or full color.""" ansi_addition = 30 if self.fg else 40 if self.isreset: return (ansi_addition+9,) elif self.representation < 3: return (color_codes_simple[self.number]+ansi_addition,) elif self.representation == 3: return (ansi_addition+8, 5, self.number) else: return (ansi_addition+8, 2, self.rgb[0], self.rgb[1], self.rgb[2]) @property def hex_code(self): """This is the hex code of the current color, html style notation.""" if self.isreset: return '#000000' else: return '#' + '{0[0]:02X}{0[1]:02X}{0[2]:02X}'.format(self.rgb) def __str__(self): """This just prints it's simple name""" return self.name def to_representation(self, val): """Converts a color to any representation""" other = copy(self) other.representation = val if self.isreset: return other other.number = None other._init_number() return other def limit_representation(self, val): """Only converts if val is lower than representation""" if self.representation <= val: return self else: return self.to_representation(val) class Style(object): """This class allows the color changes to be called directly to write them to stdout, ``[]`` calls to wrap colors (or the ``.wrap`` method) and can be called in a with statement. """ __slots__ = ('attributes','fg', 'bg', 'isreset') color_class = Color """The class of color to use. Never hardcode ``Color`` call when writing a Style method.""" attribute_names = None # should be a dict of valid names _stdout = None end = '\n' """The endline character. Override if needed in subclasses.""" ANSI_REG = re.compile('\033' + r'\[([\d;]+)m') """The regular expression that finds ansi codes in a string.""" @property def stdout(self): """\ This property will allow custom, class level control of stdout. It will use current sys.stdout if set to None (default). Unfortunately, it only works on an instance.. """ return self.__class__._stdout if self.__class__._stdout is not None else sys.stdout @stdout.setter def stdout(self, newout): self.__class__._stdout = newout def __init__(self, attributes=None, fgcolor=None, bgcolor=None, reset=False): """This is usually initialized from a factory.""" if isinstance(attributes, type(self)): for item in ('attributes','fg', 'bg', 'isreset'): setattr(self, item, copy(getattr(attributes, item))) return self.attributes = attributes if attributes is not None else dict() self.fg = fgcolor self.bg = bgcolor self.isreset = reset invalid_attributes = set(self.attributes) - set(self.attribute_names) if len(invalid_attributes) > 0: raise AttributeNotFound("Attribute(s) not valid: " + ", ".join(invalid_attributes)) @classmethod def from_color(cls, color): if color.fg: self = cls(fgcolor=color) else: self = cls(bgcolor=color) return self def invert(self): """This resets current color(s) and flips the value of all attributes present""" other = self.__class__() # Opposite of reset is reset if self.isreset: other.isreset = True return other # Flip all attributes for attribute in self.attributes: other.attributes[attribute] = not self.attributes[attribute] # Reset only if color present if self.fg: other.fg = self.fg.__class__() if self.bg: other.bg = self.bg.__class__() return other @property def reset(self): """Shortcut to access reset as a property.""" return self.invert() def __copy__(self): """Copy is supported, will make dictionary and colors unique.""" result = self.__class__() result.isreset = self.isreset result.fg = copy(self.fg) result.bg = copy(self.bg) result.attributes = copy(self.attributes) return result def __invert__(self): """This allows ~color.""" return self.invert() def __add__(self, other): """Adding two matching Styles results in a new style with the combination of both. Adding with a string results in the string concatenation of a style. Addition is non-commutative, with the rightmost Style property being taken if both have the same property. (Not safe)""" if type(self) == type(other): result = copy(other) result.isreset = self.isreset or other.isreset for attribute in self.attributes: if attribute not in result.attributes: result.attributes[attribute] = self.attributes[attribute] if not result.fg: result.fg = self.fg if not result.bg: result.bg = self.bg return result else: return other.__class__(self) + other def __radd__(self, other): """This only gets called if the string is on the left side. (Not safe)""" return other + other.__class__(self) def wrap(self, wrap_this): """Wrap a sting in this style and its inverse.""" return self + wrap_this + ~self def __and__(self, other): """This class supports ``color & color2`` syntax, and ``color & "String" syntax too.``""" if type(self) == type(other): return self + other else: return self.wrap(other) def __rand__(self, other): """This class supports ``"String:" & color`` syntax, excpet in Python 2.6 due to bug with that Python.""" return self.wrap(other) def __ror__(self, other): """Support for "String" | color syntax""" return self.wrap(other) def __or__(self, other): """This class supports ``color | color2`` syntax. It also supports ``"color | "String"`` syntax too. """ return self.__and__(other) def __call__(self): """\ This is a shortcut to print color immediately to the stdout. (Not safe) """ self.now() def now(self): '''Immediately writes color to stdout. (Not safe)''' self.stdout.write(str(self)) def print(self, *printables, **kargs): """\ This acts like print; will print that argument to stdout wrapped in Style with the same syntax as the print function in 3.4.""" end = kargs.get('end', self.end) sep = kargs.get('sep', ' ') file = kargs.get('file', self.stdout) flush = kargs.get('flush', False) file.write(self.wrap(sep.join(map(str,printables))) + end) if flush: file.flush() print_ = print """Shortcut just in case user not using __future__""" def __getitem__(self, wrapped): """The [] syntax is supported for wrapping""" return self.wrap(wrapped) def __enter__(self): """Context manager support""" self.stdout.write(str(self)) def __exit__(self, type, value, traceback): """Runs even if exception occurred, does not catch it.""" self.stdout.write(str(~self)) return False @property def ansi_codes(self): """Generates the full ANSI code sequence for a Style""" if self.isreset: return [0] codes = [] for attribute in self.attributes: if self.attributes[attribute]: codes.append(attributes_ansi[attribute]) else: # Fixing bold inverse being 22 instead of 21 on some terminals: codes.append(attributes_ansi[attribute] + 20 if attributes_ansi[attribute]!=1 else 22 ) if self.fg: codes.extend(self.fg.ansi_codes) if self.bg: self.bg.fg = False codes.extend(self.bg.ansi_codes) return codes @property def ansi_sequence(self): """This is the string ANSI sequence.""" codes = self.ansi_codes if codes: return '\033[' + ';'.join(map(str, self.ansi_codes)) + 'm' else: return '' def __repr__(self): name = self.__class__.__name__ attributes = ', '.join(a for a in self.attributes if self.attributes[a]) neg_attributes = ', '.join('-'+a for a in self.attributes if not self.attributes[a]) colors = ', '.join(repr(c) for c in [self.fg, self.bg] if c) string = '; '.join(s for s in [attributes, neg_attributes, colors] if s) if self.isreset: string = 'reset' return "<{0}: {1}>".format(name, string if string else 'empty') def __eq__(self, other): """Equality is true only if reset, or if attributes, fg, and bg match.""" if type(self) == type(other): if self.isreset: return other.isreset else: return (self.attributes == other.attributes and self.fg == other.fg and self.bg == other.bg) else: return str(self) == other @abstractmethod def __str__(self): """Base Style does not implement a __str__ representation. This is the one required method of a subclass.""" @classmethod def from_ansi(cls, ansi_string, filter_resets = False): """This generated a style from an ansi string. Will ignore resets if filter_resets is True.""" result = cls() res = cls.ANSI_REG.search(ansi_string) for group in res.groups(): sequence = map(int,group.split(';')) result.add_ansi(sequence, filter_resets) return result def add_ansi(self, sequence, filter_resets = False): """Adds a sequence of ansi numbers to the class. Will ignore resets if filter_resets is True.""" values = iter(sequence) try: while True: value = next(values) if value == 38 or value == 48: fg = value == 38 value = next(values) if value == 5: value = next(values) if fg: self.fg = self.color_class.from_full(value) else: self.bg = self.color_class.from_full(value, fg=False) elif value == 2: r = next(values) g = next(values) b = next(values) if fg: self.fg = self.color_class(r, g, b) else: self.bg = self.color_class(r, g, b, fg=False) else: raise ColorNotFound("the value 5 or 2 should follow a 38 or 48") elif value==0: if filter_resets is False: self.isreset = True elif value in attributes_ansi.values(): for name in attributes_ansi: if value == attributes_ansi[name]: self.attributes[name] = True elif value in (20+n for n in attributes_ansi.values()): if filter_resets is False: for name in attributes_ansi: if value == attributes_ansi[name] + 20: self.attributes[name] = False elif 30 <= value <= 37: self.fg = self.color_class.from_simple(value-30) elif 40 <= value <= 47: self.bg = self.color_class.from_simple(value-40, fg=False) elif 90 <= value <= 97: self.fg = self.color_class.from_simple(value-90+8) elif 100 <= value <= 107: self.bg = self.color_class.from_simple(value-100+8, fg=False) elif value == 39: if filter_resets is False: self.fg = self.color_class() elif value == 49: if filter_resets is False: self.bg = self.color_class(fg=False) else: raise ColorNotFound("The code {0} is not recognised".format(value)) except StopIteration: return @classmethod def string_filter_ansi(cls, colored_string): """Filters out colors in a string, returning only the name.""" return cls.ANSI_REG.sub('', colored_string) @classmethod def string_contains_colors(cls, colored_string): """Checks to see if a string contains colors.""" return len(cls.ANSI_REG.findall(colored_string)) > 0 def to_representation(self, rep): """This converts both colors to a specific representation""" other = copy(self) if other.fg: other.fg = other.fg.to_representation(rep) if other.bg: other.bg = other.bg.to_representation(rep) return other def limit_representation(self, rep): """This only converts if true representation is higher""" if rep is True or rep is False: return self other = copy(self) if other.fg: other.fg = other.fg.limit_representation(rep) if other.bg: other.bg = other.bg.limit_representation(rep) return other @property def basic(self): """The color in the 8 color representation.""" return self.to_representation(1) @property def simple(self): """The color in the 16 color representation.""" return self.to_representation(2) @property def full(self): """The color in the 256 color representation.""" return self.to_representation(3) @property def true(self): """The color in the true color representation.""" return self.to_representation(4) class ANSIStyle(Style): """This is a subclass for ANSI styles. Use it to get color on sys.stdout tty terminals on posix systems. Set ``use_color = True/False`` if you want to control color for anything using this Style.""" __slots__ = () use_color = get_color_repr() attribute_names = attributes_ansi def __str__(self): if not self.use_color: return '' else: return self.limit_representation(self.use_color).ansi_sequence class HTMLStyle(Style): """This was meant to be a demo of subclassing Style, but actually can be a handy way to quickly color html text.""" __slots__ = () attribute_names = dict(bold='b', em='em', italics='i', li='li', underline='span style="text-decoration: underline;"', code='code', ol='ol start=0', strikeout='s') end = '
\n' def __str__(self): if self.isreset: raise ResetNotSupported("HTML does not support global resets!") result = '' if self.bg and not self.bg.isreset: result += ''.format(self.bg.hex_code) if self.fg and not self.fg.isreset: result += ''.format(self.fg.hex_code) for attr in sorted(self.attributes): if self.attributes[attr]: result += '<' + self.attribute_names[attr] + '>' for attr in reversed(sorted(self.attributes)): if not self.attributes[attr]: result += '' if self.fg and self.fg.isreset: result += '' if self.bg and self.bg.isreset: result += '' return result plumbum-1.6.0/plumbum/colorlib/factories.py0000644000232200023220000001530712610225666021400 0ustar debalancedebalance""" Color-related factories. They produce Styles. """ from __future__ import print_function import sys from functools import reduce from plumbum.colorlib.names import color_names, default_styles from plumbum.colorlib.styles import ColorNotFound __all__ = ['ColorFactory', 'StyleFactory'] class ColorFactory(object): """This creates color names given fg = True/False. It usually will be called as part of a StyleFactory.""" def __init__(self, fg, style): self._fg = fg self._style = style self.reset = style.from_color(style.color_class(fg=fg)) # Adding the color name shortcuts for foreground colors for item in color_names[:16]: setattr(self, item, style.from_color(style.color_class.from_simple(item, fg=fg))) def __getattr__(self, item): """Full color names work, but do not populate __dir__.""" try: return self._style.from_color(self._style.color_class(item, fg=self._fg)) except ColorNotFound: raise AttributeError(item) def full(self, name): """Gets the style for a color, using standard name procedure: either full color name, html code, or number.""" return self._style.from_color(self._style.color_class.from_full(name, fg=self._fg)) def simple(self, name): """Return the extended color scheme color for a value or name.""" return self._style.from_color(self._style.color_class.from_simple(name, fg=self._fg)) def rgb(self, r, g=None, b=None): """Return the extended color scheme color for a value.""" if g is None and b is None: return self.hex(r) else: return self._style.from_color(self._style.color_class(r, g, b, fg=self._fg)) def hex(self, hexcode): """Return the extended color scheme color for a value.""" return self._style.from_color(self._style.color_class.from_hex(hexcode, fg=self._fg)) def ansi(self, ansiseq): """Make a style from an ansi text sequence""" return self._style.from_ansi(ansiseq) def __getitem__(self, val): """\ Shortcut to provide way to access colors numerically or by slice. If end <= 16, will stay to simple ANSI version.""" if isinstance(val, slice): (start, stop, stride) = val.indices(256) if stop <= 16: return [self.simple(v) for v in range(start, stop, stride)] else: return [self.full(v) for v in range(start, stop, stride)] elif isinstance(val, tuple): return self.rgb(*val) try: return self.full(val) except ColorNotFound: return self.hex(val) def __call__(self, val_or_r=None, g = None, b = None): """Shortcut to provide way to access colors.""" if val_or_r is None or (isinstance(val_or_r, str) and val_or_r == ''): return self._style() if isinstance(val_or_r, self._style): return self._style(val_or_r) if isinstance(val_or_r, str) and '\033' in val_or_r: return self.ansi(val_or_r) return self._style.from_color(self._style.color_class(val_or_r, g, b, fg=self._fg)) def __iter__(self): """Iterates through all colors in extended colorset.""" return (self.full(i) for i in range(256)) def __invert__(self): """Allows clearing a color with ~""" return self.reset def __enter__(self): """This will reset the color on leaving the with statement.""" return self def __exit__(self, type, value, traceback): """This resets a FG/BG color or all styles, due to different definition of RESET for the factories.""" self.reset.now() return False def __repr__(self): """Simple representation of the class by name.""" return "<{0}>".format(self.__class__.__name__) class StyleFactory(ColorFactory): """Factory for styles. Holds font styles, FG and BG objects representing colors, and imitates the FG ColorFactory to a large degree.""" def __init__(self, style): super(StyleFactory,self).__init__(True, style) self.fg = ColorFactory(True, style) self.bg = ColorFactory(False, style) self.do_nothing = style() self.reset = style(reset=True) for item in style.attribute_names: setattr(self, item, style(attributes={item:True})) self.load_stylesheet(default_styles) @property def use_color(self): """Shortcut for setting color usage on Style""" return self._style.use_color @use_color.setter def use_color(self, val): self._style.use_color = val def from_ansi(self, ansi_sequence): """Calling this is a shortcut for creating a style from an ANSI sequence.""" return self._style.from_ansi(ansi_sequence) @property def stdout(self): """This is a shortcut for getting stdout from a class without an instance.""" return self._style._stdout if self._style._stdout is not None else sys.stdout @stdout.setter def stdout(self, newout): self._style._stdout = newout def get_colors_from_string(self, color=''): """ Sets color based on string, use `.` or space for separator, and numbers, fg/bg, htmlcodes, etc all accepted (as strings). """ names = color.replace('.', ' ').split() prev = self styleslist = [] for name in names: try: prev = getattr(prev, name) except AttributeError: try: prev = prev(int(name)) except (ColorNotFound, ValueError): prev = prev(name) if isinstance(prev, self._style): styleslist.append(prev) prev = self if styleslist: prev = reduce(lambda a,b: a & b, styleslist) return prev if isinstance(prev, self._style) else prev.reset def filter(self, colored_string): """Filters out colors in a string, returning only the name.""" if isinstance(colored_string, self._style): return colored_string return self._style.string_filter_ansi(colored_string) def contains_colors(self, colored_string): """Checks to see if a string contains colors.""" return self._style.string_contains_colors(colored_string) def extract(self, colored_string): """Gets colors from an ansi string, returns those colors""" return self._style.from_ansi(colored_string, True) def load_stylesheet(self, stylesheet=default_styles): for item in stylesheet: setattr(self, item, self.get_colors_from_string(stylesheet[item])) plumbum-1.6.0/plumbum/colorlib/__main__.py0000644000232200023220000000026212610225666021133 0ustar debalancedebalance""" This is provided as a quick way to recover your terminal. Simply run ``python -m plumbum.colorlib`` to recover terminal color. """ from plumbum.colorlib import main main() plumbum-1.6.0/plumbum/colorlib/names.py0000644000232200023220000001715112610225666020523 0ustar debalancedebalance''' Names for the standard and extended color set. Extended set is similar to `vim wiki `_, `colored `_, etc. Colors based on `wikipedia `_. You can access the index of the colors with names.index(name). You can access the rgb values with ``r=int(html[n][1:3],16)``, etc. ''' from __future__ import division, print_function color_names = '''\ black red green yellow blue magenta cyan light_gray dark_gray light_red light_green light_yellow light_blue light_magenta light_cyan white grey_0 navy_blue dark_blue blue_3 blue_3a blue_1 dark_green deep_sky_blue_4 deep_sky_blue_4a deep_sky_blue_4b dodger_blue_3 dodger_blue_2 green_4 spring_green_4 turquoise_4 deep_sky_blue_3 deep_sky_blue_3a dodger_blue_1 green_3 spring_green_3 dark_cyan light_sea_green deep_sky_blue_2 deep_sky_blue_1 green_3a spring_green_3a spring_green_2 cyan_3 dark_turquoise turquoise_2 green_1 spring_green_2a spring_green_1 medium_spring_green cyan_2 cyan_1 dark_red deep_pink_4 purple_4 purple_4a purple_3 blue_violet orange_4 grey_37 medium_purple_4 slate_blue_3 slate_blue_3a royal_blue_1 chartreuse_4 dark_sea_green_4 pale_turquoise_4 steel_blue steel_blue_3 cornflower_blue chartreuse_3 dark_sea_green_4a cadet_blue cadet_blue_a sky_blue_3 steel_blue_1 chartreuse_3a pale_green_3 sea_green_3 aquamarine_3 medium_turquoise steel_blue_1a chartreuse_2a sea_green_2 sea_green_1 sea_green_1a aquamarine_1 dark_slate_gray_2 dark_red_a deep_pink_4a dark_magenta dark_magenta_a dark_violet purple orange_4a light_pink_4 plum_4 medium_purple_3 medium_purple_3a slate_blue_1 yellow_4 wheat_4 grey_53 light_slate_grey medium_purple light_slate_blue yellow_4_a dark_olive_green_3 dark_sea_green light_sky_blue_3 light_sky_blue_3a sky_blue_2 chartreuse_2 dark_olive_green_3a pale_green_3a dark_sea_green_3 dark_slate_gray_3 sky_blue_1 chartreuse_1 light_green_a light_green_b pale_green_1 aquamarine_1a dark_slate_gray_1 red_3 deep_pink_4b medium_violet_red magenta_3 dark_violet_a purple_a dark_orange_3 indian_red hot_pink_3 medium_orchid_3 medium_orchid medium_purple_2 dark_goldenrod light_salmon_3 rosy_brown grey_63 medium_purple_2a medium_purple_1 gold_3 dark_khaki navajo_white_3 grey_69 light_steel_blue_3 light_steel_blue yellow_3 dark_olive_green_3b dark_sea_green_3a dark_sea_green_2 light_cyan_3 light_sky_blue_1 green_yellow dark_olive_green_2 pale_green_1a dark_sea_green_2a dark_sea_green_1 pale_turquoise_1 red_3a deep_pink_3 deep_pink_3a magenta_3a magenta_3b magenta_2 dark_orange_3a indian_red_a hot_pink_3a hot_pink_2 orchid medium_orchid_1 orange_3 light_salmon_3a light_pink_3 pink_3 plum_3 violet gold_3a light_goldenrod_3 tan misty_rose_3 thistle_3 plum_2 yellow_3a khaki_3 light_goldenrod_2 light_yellow_3 grey_84 light_steel_blue_1 yellow_2 dark_olive_green_1 dark_olive_green_1a dark_sea_green_1a honeydew_2 light_cyan_1 red_1 deep_pink_2 deep_pink_1 deep_pink_1a magenta_2a magenta_1 orange_red_1 indian_red_1 indian_red_1a hot_pink hot_pink_a medium_orchid_1a dark_orange salmon_1 light_coral pale_violet_red_1 orchid_2 orchid_1 orange_1 sandy_brown light_salmon_1 light_pink_1 pink_1 plum_1 gold_1 light_goldenrod_2a light_goldenrod_2b navajo_white_1 misty_rose_1 thistle_1 yellow_1 light_goldenrod_1 khaki_1 wheat_1 cornsilk_1 grey_10_0 grey_3 grey_7 grey_11 grey_15 grey_19 grey_23 grey_27 grey_30 grey_35 grey_39 grey_42 grey_46 grey_50 grey_54 grey_58 grey_62 grey_66 grey_70 grey_74 grey_78 grey_82 grey_85 grey_89 grey_93'''.split() _greys = (3.4, 7.4, 11, 15, 19, 23, 26.7, 30.49, 34.6, 38.6, 42.4, 46.4, 50, 54, 58, 62, 66, 69.8, 73.8, 77.7, 81.6, 85.3, 89.3, 93) _grey_vals = [int(x/100.0*16*16) for x in _greys] _grey_html = ['#' + format(x,'02x')*3 for x in _grey_vals] _normals = [int(x,16) for x in '0 5f 87 af d7 ff'.split()] _normal_html = ['#' + format(_normals[n//36],'02x') + format(_normals[n//6%6],'02x') + format(_normals[n%6],'02x') for n in range(16-16,232-16)] _base_pattern = [(n//4,n//2%2,n%2) for n in range(8)] _base_html = (['#{2:02x}{1:02x}{0:02x}'.format(x[0]*192,x[1]*192,x[2]*192) for x in _base_pattern] + ['#808080'] + ['#{2:02x}{1:02x}{0:02x}'.format(x[0]*255,x[1]*255,x[2]*255) for x in _base_pattern][1:]) color_html = _base_html + _normal_html + _grey_html color_codes_simple = list(range(8)) + list(range(60,68)) """Simple colors, remember that reset is #9, second half is non as common.""" # Attributes attributes_ansi = dict( bold=1, dim=2, italics=3, underline=4, reverse=7, hidden=8, strikeout=9, ) # Stylesheet default_styles = dict( warn="fg red", title="fg cyan underline bold", fatal="fg red bold", highlight="bg yellow", info="fg blue", success="fg green", ) #Functions to be used for color name operations class FindNearest(object): """This is a class for finding the nearest color given rgb values. Different find methods are available.""" def __init__(self, r, g, b): self.r = r self.b = b self.g = g def only_basic(self): """This will only return the first 8 colors! Breaks the colorspace into cubes, returns color""" midlevel = 0x40 # Since bright is not included # The colors are organised so that it is a # 3D cube, black at 0,0,0, white at 1,1,1 # Compressed to linear_integers r,g,b # [[[0,1],[2,3]],[[4,5],[6,7]]] # r*1 + g*2 + b*4 return (self.r>=midlevel)*1 + (self.g>=midlevel)*2 + (self.b>=midlevel)*4 def all_slow(self, color_slice=slice(None, None, None)): """This is a slow way to find the nearest color.""" distances = [self._distance_to_color(color) for color in color_html[color_slice]] return min(range(len(distances)), key=distances.__getitem__) def _distance_to_color(self, color): """This computes the distance to a color, should be minimized.""" rgb = (int(color[1:3],16), int(color[3:5],16), int(color[5:7],16)) return (self.r-rgb[0])**2 + (self.g-rgb[1])**2 + (self.b-rgb[2])**2 def _distance_to_color_number(self, n): color = color_html[n] return self._distance_to_color(color) def only_colorblock(self): """This finds the nearest color based on block system, only works for 17-232 color values.""" rint = min(range(len(_normals)), key=[abs(x-self.r) for x in _normals].__getitem__) bint = min(range(len(_normals)), key=[abs(x-self.b) for x in _normals].__getitem__) gint = min(range(len(_normals)), key=[abs(x-self.g) for x in _normals].__getitem__) return (16 + 36 * rint + 6 * gint + bint) def only_simple(self): """Finds the simple color-block color.""" return self.all_slow(slice(0,16,None)) def only_grey(self): """Finds the greyscale color.""" rawval = (self.r + self.b + self.g) / 3 n = min(range(len(_grey_vals)), key=[abs(x-rawval) for x in _grey_vals].__getitem__) return n+232 def all_fast(self): """Runs roughly 8 times faster than the slow version.""" colors = [self.only_simple(), self.only_colorblock(), self.only_grey()] distances = [self._distance_to_color_number(n) for n in colors] return colors[min(range(len(distances)), key=distances.__getitem__)] def from_html(color): """Convert html hex code to rgb.""" if len(color) != 7 or color[0] != '#': raise ValueError("Invalid length of html code") return (int(color[1:3],16), int(color[3:5],16), int(color[5:7],16)) def to_html(r, g, b): """Convert rgb to html hex code.""" return "#{0:02x}{1:02x}{2:02x}".format(r, g, b) plumbum-1.6.0/plumbum/colorlib/_ipython_ext.py0000644000232200023220000000210212610225666022117 0ustar debalancedebalancefrom IPython.core.magic import (Magics, magics_class, cell_magic, needs_local_scope) import IPython.display try: from io import StringIO except ImportError: try: from cStringIO import StringIO except ImportError: from StringIO import StringIO import sys valid_choices = [x[8:] for x in dir(IPython.display) if 'display_' == x[:8]] @magics_class class OutputMagics(Magics): @needs_local_scope @cell_magic def to(self, line, cell, local_ns=None): choice = line.strip() assert choice in valid_choices, "Valid choices for '%%to' are: "+str(valid_choices) display_fn = getattr(IPython.display, "display_"+choice) "Captures stdout and renders it in the notebook with some ." with StringIO() as out: old_out = sys.stdout try: sys.stdout = out exec(cell, self.shell.user_ns, local_ns) out.seek(0) display_fn(out.getvalue(), raw=True) finally: sys.stdout = old_out plumbum-1.6.0/plumbum/cli/0000755000232200023220000000000012610225742015776 5ustar debalancedebalanceplumbum-1.6.0/plumbum/cli/__init__.py0000644000232200023220000000040112610225666020107 0ustar debalancedebalancefrom plumbum.cli.switches import SwitchError, switch, autoswitch, SwitchAttr, Flag, CountOf, positional from plumbum.cli.switches import Range, Set, ExistingDirectory, ExistingFile, NonexistentPath, Predicate from plumbum.cli.application import Application plumbum-1.6.0/plumbum/cli/switches.py0000644000232200023220000004370012610225666020212 0ustar debalancedebalancefrom plumbum.lib import six, getdoc from plumbum import local from abc import abstractmethod class SwitchError(Exception): """A general switch related-error (base class of all other switch errors)""" pass class PositionalArgumentsError(SwitchError): """Raised when an invalid number of positional arguments has been given""" pass class SwitchCombinationError(SwitchError): """Raised when an invalid combination of switches has been given""" pass class UnknownSwitch(SwitchError): """Raised when an unrecognized switch has been given""" pass class MissingArgument(SwitchError): """Raised when a switch requires an argument, but one was not provided""" pass class MissingMandatorySwitch(SwitchError): """Raised when a mandatory switch has not been given""" pass class WrongArgumentType(SwitchError): """Raised when a switch expected an argument of some type, but an argument of a wrong type has been given""" pass class SubcommandError(SwitchError): """Raised when there's something wrong with subcommands""" pass #=================================================================================================== # The switch decorator #=================================================================================================== class SwitchInfo(object): def __init__(self, **kwargs): for k, v in kwargs.items(): setattr(self, k, v) def switch(names, argtype = None, argname = None, list = False, mandatory = False, requires = (), excludes = (), help = None, overridable = False, group = "Switches", envname=None): """ A decorator that exposes functions as command-line switches. Usage:: class MyApp(Application): @switch(["-l", "--log-to-file"], argtype = str) def log_to_file(self, filename): handler = logging.FileHandler(filename) logger.addHandler(handler) @switch(["--verbose"], excludes=["--terse"], requires=["--log-to-file"]) def set_debug(self): logger.setLevel(logging.DEBUG) @switch(["--terse"], excludes=["--verbose"], requires=["--log-to-file"]) def set_terse(self): logger.setLevel(logging.WARNING) :param names: The name(s) under which the function is reachable; it can be a string or a list of string, but at least one name is required. There's no need to prefix the name with ``-`` or ``--`` (this is added automatically), but it can be used for clarity. Single-letter names are prefixed by ``-``, while longer names are prefixed by ``--`` :param envname: Name of environment variable to extract value from, as alternative to argv :param argtype: If this function takes an argument, you need to specify its type. The default is ``None``, which means the function takes no argument. The type is more of a "validator" than a real type; it can be any callable object that raises a ``TypeError`` if the argument is invalid, or returns an appropriate value on success. If the user provides an invalid value, :func:`plumbum.cli.WrongArgumentType` :param argname: The name of the argument; if ``None``, the name will be inferred from the function's signature :param list: Whether or not this switch can be repeated (e.g. ``gcc -I/lib -I/usr/lib``). If ``False``, only a single occurrence of the switch is allowed; if ``True``, it may be repeated indefinitely. The occurrences are collected into a list, so the function is only called once with the collections. For instance, for ``gcc -I/lib -I/usr/lib``, the function will be called with ``["/lib", "/usr/lib"]``. :param mandatory: Whether or not this switch is mandatory; if a mandatory switch is not given, :class:`MissingMandatorySwitch ` is raised. The default is ``False``. :param requires: A list of switches that this switch depends on ("requires"). This means that it's invalid to invoke this switch without also invoking the required ones. In the example above, it's illegal to pass ``--verbose`` or ``--terse`` without also passing ``--log-to-file``. By default, this list is empty, which means the switch has no prerequisites. If an invalid combination is given, :class:`SwitchCombinationError ` is raised. Note that this list is made of the switch *names*; if a switch has more than a single name, any of its names will do. .. note:: There is no guarantee on the (topological) order in which the actual switch functions will be invoked, as the dependency graph might contain cycles. :param excludes: A list of switches that this switch forbids ("excludes"). This means that it's invalid to invoke this switch if any of the excluded ones are given. In the example above, it's illegal to pass ``--verbose`` along with ``--terse``, as it will result in a contradiction. By default, this list is empty, which means the switch has no prerequisites. If an invalid combination is given, :class:`SwitchCombinationError ` is raised. Note that this list is made of the switch *names*; if a switch has more than a single name, any of its names will do. :param help: The help message (description) for this switch; this description is used when ``--help`` is given. If ``None``, the function's docstring will be used. :param overridable: Whether or not the names of this switch are overridable by other switches. If ``False`` (the default), having another switch function with the same name(s) will cause an exception. If ``True``, this is silently ignored. :param group: The switch's *group*; this is a string that is used to group related switches together when ``--help`` is given. The default group is ``Switches``. :returns: The decorated function (with a ``_switch_info`` attribute) """ if isinstance(names, six.string_types): names = [names] names = [n.lstrip("-") for n in names] requires = [n.lstrip("-") for n in requires] excludes = [n.lstrip("-") for n in excludes] def deco(func): if argname is None: argspec = six.getfullargspec(func).args if len(argspec) == 2: argname2 = argspec[1] else: argname2 = "VALUE" else: argname2 = argname help2 = getdoc(func) if help is None else help if not help2: help2 = str(func) func._switch_info = SwitchInfo(names = names, envname=envname, argtype = argtype, list = list, func = func, mandatory = mandatory, overridable = overridable, group = group, requires = requires, excludes = excludes, argname = argname2, help = help2) return func return deco def autoswitch(*args, **kwargs): """A decorator that exposes a function as a switch, "inferring" the name of the switch from the function's name (converting to lower-case, and replacing underscores with hyphens). The arguments are the same as for :func:`switch `.""" def deco(func): return switch(func.__name__.replace("_", "-"), *args, **kwargs)(func) return deco #=================================================================================================== # Switch Attributes #=================================================================================================== class SwitchAttr(object): """ A switch that stores its result in an attribute (descriptor). Usage:: class MyApp(Application): logfile = SwitchAttr(["-f", "--log-file"], str) def main(self): if self.logfile: open(self.logfile, "w") :param names: The switch names :param argtype: The switch argument's (and attribute's) type :param default: The attribute's default value (``None``) :param argname: The switch argument's name (default is ``"VALUE"``) :param kwargs: Any of the keyword arguments accepted by :func:`switch ` """ ATTR_NAME = '__plumbum_switchattr_dict__' def __init__(self, names, argtype = str, default = None, list = False, argname = "VALUE", **kwargs): self.__doc__ = "Sets an attribute" # to prevent the help message from showing SwitchAttr's docstring if "help" in kwargs and default and argtype is not None: kwargs["help"] += "; the default is %r" % (default,) switch(names, argtype = argtype, argname = argname, list = list, **kwargs)(self) listtype = type([]) if list: if default is None: self._default_value = [] elif isinstance(default, (tuple, listtype)): self._default_value = listtype(default) else: self._default_value = [default] else: self._default_value = default def __call__(self, inst, val): self.__set__(inst, val) def __get__(self, inst, cls): if inst is None: return self else: return getattr(inst, self.ATTR_NAME, {}).get(self, self._default_value) def __set__(self, inst, val): if inst is None: raise AttributeError("cannot set an unbound SwitchAttr") else: if not hasattr(inst, self.ATTR_NAME): setattr(inst, self.ATTR_NAME, {self : val}) else: getattr(inst, self.ATTR_NAME)[self] = val class Flag(SwitchAttr): """A specialized :class:`SwitchAttr ` for boolean flags. If the flag is not given, the value of this attribute is the ``default``; if it is given, the value changes to ``not default``. Usage:: class MyApp(Application): verbose = Flag(["-v", "--verbose"], help = "If given, I'll be very talkative") :param names: The switch names :param default: The attribute's initial value (``False`` by default) :param kwargs: Any of the keyword arguments accepted by :func:`switch `, except for ``list`` and ``argtype``. """ def __init__(self, names, default = False, **kwargs): SwitchAttr.__init__(self, names, argtype = None, default = default, list = False, **kwargs) def __call__(self, inst): self.__set__(inst, not self._default_value) class CountOf(SwitchAttr): """A specialized :class:`SwitchAttr ` that counts the number of occurrences of the switch in the command line. Usage:: class MyApp(Application): verbosity = CountOf(["-v", "--verbose"], help = "The more, the merrier") If ``-v -v -vv`` is given in the command-line, it will result in ``verbosity = 4``. :param names: The switch names :param default: The default value (0) :param kwargs: Any of the keyword arguments accepted by :func:`switch `, except for ``list`` and ``argtype``. """ def __init__(self, names, default = 0, **kwargs): SwitchAttr.__init__(self, names, argtype = None, default = default, list = True, **kwargs) self._default_value = default # issue #118 def __call__(self, inst, v): self.__set__(inst, len(v)) #=================================================================================================== # Decorator for function that adds argument checking #=================================================================================================== class positional(object): """ Runs a validator on the main function for a class. This should be used like this:: class MyApp(cli.Application): @cli.positional(cli.Range(1,10), cli.ExistingFile) def main(self, x, *f): # x is a range, f's are all ExistingFile's) Or, Python 3 only:: class MyApp(cli.Application): def main(self, x : cli.Range(1,10), *f : cli.ExistingFile): # x is a range, f's are all ExistingFile's) If you do not want to validate on the annotations, use this decorator ( even if empty) to override annotation validation. Validators should be callable, and should have a ``.choices()`` function with possible choices. (For future argument completion, for example) Default arguments do not go through the validator. #TODO: Check with MyPy """ def __init__(self, *args, **kargs): self.args = args self.kargs = kargs def __call__(self, function): m = six.getfullargspec(function) args_names = list(m.args[1:]) positional = [None]*len(args_names) varargs = None for i in range(min(len(positional),len(self.args))): positional[i] = self.args[i] if len(args_names) + 1 == len(self.args): varargs = self.args[-1] # All args are positional, so convert kargs to positional for item in self.kargs: if item == m.varargs: varargs = self.kargs[item] else: positional[args_names.index(item)] = self.kargs[item] function.positional = positional function.positional_varargs = varargs return function class Validator(six.ABC): __slots__ = () @abstractmethod def __call__(self, obj): "Must be implemented for a Validator to work" def choices(self, partial=""): """Should return set of valid choices, can be given optional partial info""" return set() def __repr__(self): """If not overridden, will print the slots as args""" slots = {} for cls in self.__mro__: for prop in getattr(cls, "__slots__", ()): if prop[0] != '_': slots[prop] = getattr(self, prop) mystrs = ("{0} = {1}".format(name, slots[name]) for name in slots) return "{0}({1})".format(self.__class__.__name__, ", ".join(mystrs)) #=================================================================================================== # Switch type validators #=================================================================================================== class Range(Validator): """ A switch-type validator that checks for the inclusion of a value in a certain range. Usage:: class MyApp(Application): age = SwitchAttr(["--age"], Range(18, 120)) :param start: The minimal value :param end: The maximal value """ __slots__ = ("start", "end") def __init__(self, start, end): self.start = start self.end = end def __repr__(self): return "[%d..%d]" % (self.start, self.end) def __call__(self, obj): obj = int(obj) if obj < self.start or obj > self.end: raise ValueError("Not in range [%d..%d]" % (self.start, self.end)) return obj def choices(self, partial=""): # TODO: Add partial handling return set(range(self.start, self.end+1)) class Set(Validator): """ A switch-type validator that checks that the value is contained in a defined set of values. Usage:: class MyApp(Application): mode = SwitchAttr(["--mode"], Set("TCP", "UDP", case_sensitive = False)) :param values: The set of values (strings) :param case_sensitive: A keyword argument that indicates whether to use case-sensitive comparison or not. The default is ``False`` """ def __init__(self, *values, **kwargs): self.case_sensitive = kwargs.pop("case_sensitive", False) if kwargs: raise TypeError("got unexpected keyword argument(s): %r" % (kwargs.keys(),)) self.values = dict(((v if self.case_sensitive else v.lower()), v) for v in values) def __repr__(self): return "{%s}" % (", ".join(repr(v) for v in self.values.values())) def __call__(self, obj): if not self.case_sensitive: obj = obj.lower() if obj not in self.values: raise ValueError("Expected one of %r" % (list(self.values.values()),)) return self.values[obj] def choices(self, partial=""): # TODO: Add case sensitive/insensitive parital completion return set(self.values) class Predicate(object): """A wrapper for a single-argument function with pretty printing""" def __init__(self, func): self.func = func def __str__(self): return self.func.__name__ def __call__(self, val): return self.func(val) def choices(self, partial=""): return set() @Predicate def ExistingDirectory(val): """A switch-type validator that ensures that the given argument is an existing directory""" p = local.path(val) if not p.is_dir(): raise ValueError("%r is not a directory" % (val,)) return p @Predicate def ExistingFile(val): """A switch-type validator that ensures that the given argument is an existing file""" p = local.path(val) if not p.is_file(): raise ValueError("%r is not a file" % (val,)) return p @Predicate def NonexistentPath(val): """A switch-type validator that ensures that the given argument is a nonexistent path""" p = local.path(val) if p.exists(): raise ValueError("%r already exists" % (val,)) return p plumbum-1.6.0/plumbum/cli/progress.py0000644000232200023220000001707012610225666020226 0ustar debalancedebalance""" Progress bar ------------ """ from __future__ import print_function, division import warnings from abc import abstractmethod import datetime from plumbum.lib import six from plumbum.cli.termsize import get_terminal_size class ProgressBase(six.ABC): """Base class for progress bars. Customize for types of progress bars. :param iterator: The iterator to wrap with a progress bar :param length: The length of the iterator (will use ``__len__`` if None) :param timer: Try to time the completion status of the iterator :param body: True if the slow portion occurs outside the iterator (in a loop, for example) """ def __init__(self, iterator=None, length=None, timer=True, body=False, has_output=False): if length is None: length = len(iterator) elif iterator is None: iterator = range(length) elif length is None and iterator is None: raise TypeError("Expected either an iterator or a length") self.length = length self.iterator = iterator self.timer = timer self.body = body self.has_output = has_output def __len__(self): return self.length def __iter__(self): self.start() return self @abstractmethod def start(self): """This should initialize the progress bar and the iterator""" self.iter = iter(self.iterator) self.value = -1 if self.body else 0 self._start_time = datetime.datetime.now() def __next__(self): try: rval = next(self.iter) self.increment() except StopIteration: self.done() raise return rval def next(self): return self.__next__() @property def value(self): """This is the current value, as a property so setting it can be customized""" return self._value @value.setter def value(self, val): self._value = val @abstractmethod def display(self): """Called to update the progress bar""" pass def increment(self): """Sets next value and displays the bar""" self.value += 1 self.display() def time_remaining(self): """Get the time remaining for the progress bar, guesses""" if self.value < 1: return None, None elapsed_time = datetime.datetime.now() - self._start_time time_each = elapsed_time / self.value time_remaining = time_each * (self.length - self.value) return elapsed_time, time_remaining def str_time_remaining(self): """Returns a string version of time remaining""" if self.value < 1: return "Starting... " else: elapsed_time, time_remaining = list(map(str,self.time_remaining())) return "{0} completed, {1} remaining".format(elapsed_time.split('.')[0], time_remaining.split('.')[0]) @abstractmethod def done(self): """Is called when the iterator is done.""" pass @classmethod def range(cls, value, **kargs): """Fast shortcut to create a range based progress bar, assumes work done in body""" return cls(range(value), value, body=True, **kargs) @classmethod def wrap(cls, iterator, length=None, **kargs): """Shortcut to wrap an iterator that does not do all the work internally""" return cls(iterator, length, body = True, **kargs) class Progress(ProgressBase): def start(self): super(Progress, self).start() self.display() def done(self): self.value = self.length self.display() if self.has_output: print() def __str__(self): percent = max(self.value,0)/self.length width = get_terminal_size(default=(0,0))[0] ending = ' ' + (self.str_time_remaining() if self.timer else '{0} of {1} complete'.format(self.value, self.length)) if width - len(ending) < 10 or self.has_output: self.width = 0 if self.timer: return "{0:g}% complete: {1}".format(100*percent, self.str_time_remaining()) else: return "{0:g}% complete".format(100*percent) else: self.width = width - len(ending) - 2 - 1 nstars = int(percent*self.width) pbar = '[' + '*'*nstars + ' '*(self.width-nstars) + ']' + ending str_percent = ' {0:.0f}% '.format(100*percent) return pbar[:self.width//2 - 2] + str_percent + pbar[self.width//2+len(str_percent) - 2:] def display(self): disptxt = str(self) if self.width == 0 or self.has_output: print(disptxt) else: print("\r", end='') print(disptxt, end='', flush=True) class ProgressIPy(ProgressBase): HTMLBOX = '
{}
' def __init__(self, *args, **kargs): # Ipython gives warnings when using widgets about the API potentially changing with warnings.catch_warnings(): warnings.simplefilter("ignore") try: from ipywidgets import IntProgress, HTML, HBox except ImportError: # Support IPython < 4.0 from IPython.html.widgets import IntProgress, HTML, HBox super(ProgressIPy, self).__init__(*args, **kargs) self.prog = IntProgress(max=self.length) self._label = HTML() self._box = HBox((self.prog, self._label)) def start(self): from IPython.display import display display(self._box) super(ProgressIPy, self).start() @property def value(self): """This is the current value, -1 allowed (automatically fixed for display)""" return self._value @value.setter def value(self, val): self._value = val self.prog.value = max(val, 0) self.prog.description = "{0:.2f}%".format(100*self.value / self.length) if self.timer and val > 0: self._label.value = self.HTMLBOX.format(self.str_time_remaining()) def display(self): pass def done(self): self._box.close() class ProgressAuto(ProgressBase): """Automatically selects the best progress bar (IPython HTML or text). Does not work with qtconsole (as that is correctly identified as identical to notebook, since the kernel is the same); it will still iterate, but no graphical indication will be diplayed. :param iterator: The iterator to wrap with a progress bar :param length: The length of the iterator (will use ``__len__`` if None) :param timer: Try to time the completion status of the iterator :param body: True if the slow portion occurs outside the iterator (in a loop, for example) """ def __new__(cls, *args, **kargs): """Uses the generator trick that if a cls instance is returned, the __init__ method is not called.""" try: __IPYTHON__ try: from traitlets import TraitError except ImportError: # Support for IPython < 4.0 from IPython.utils.traitlets import TraitError try: return ProgressIPy(*args, **kargs) except TraitError: raise NameError() except (NameError, ImportError): return Progress(*args, **kargs) ProgressAuto.register(ProgressIPy) ProgressAuto.register(Progress) def main(): import time tst = Progress.range(20) for i in tst: time.sleep(1) if __name__ == '__main__': main() plumbum-1.6.0/plumbum/cli/terminal.py0000644000232200023220000001460512610225666020176 0ustar debalancedebalance""" Terminal-related utilities -------------------------- """ from __future__ import division, print_function, absolute_import import sys import os from plumbum import local from plumbum.cli.termsize import get_terminal_size from plumbum.cli.progress import Progress def readline(message = ""): """Gets a line of input from the user (stdin)""" sys.stdout.write(message) return sys.stdin.readline() def ask(question, default = None): """ Presents the user with a yes/no question. :param question: The question to ask :param default: If ``None``, the user must answer. If ``True`` or ``False``, lack of response is interpreted as the default option :returns: the user's choice """ question = question.rstrip().rstrip("?").rstrip() + "?" if default is None: question += " (y/n) " elif default: question += " [Y/n] " else: question += " [y/N] " while True: try: answer = readline(question).strip().lower() except EOFError: answer = None if answer in ("y", "yes"): return True elif answer in ("n", "no"): return False elif not answer and default is not None: return default else: sys.stdout.write("Invalid response, please try again\n") def choose(question, options, default = None): """Prompts the user with a question and a set of options, from which the user need choose. :param question: The question to ask :param options: A set of options. It can be a list (of strings or two-tuples, mapping text to returned-object) or a dict (mapping text to returned-object).`` :param default: If ``None``, the user must answer. Otherwise, lack of response is interpreted as this answer :returns: The user's choice Example:: ans = choose("What is your favorite color?", ["blue", "yellow", "green"], default = "yellow") # `ans` will be one of "blue", "yellow" or "green" ans = choose("What is your favorite color?", {"blue" : 0x0000ff, "yellow" : 0xffff00 , "green" : 0x00ff00}, default = 0x00ff00) # this will display "blue", "yellow" and "green" but return a numerical value """ if hasattr(options, "items"): options = options.items() sys.stdout.write(question.rstrip() + "\n") choices = {} defindex = None for i, item in enumerate(options): i = i + 1 # python2.5 if isinstance(item, (tuple, list)) and len(item) == 2: text = item[0] val = item[1] else: text = item val = item choices[i] = val if default is not None and default == val: defindex = i sys.stdout.write("(%d) %s\n" % (i, text)) if default is not None: if defindex is None: msg = "Choice [%s]: " % (default,) else: msg = "Choice [%d]: " % (defindex,) else: msg = "Choice: " while True: try: choice = readline(msg).strip() except EOFError: choice = "" if not choice and default: return default try: choice = int(choice) if choice not in choices: raise ValueError() except ValueError: sys.stdout.write("Invalid choice, please try again\n") continue return choices[choice] def prompt(question, type = int, default = NotImplemented, validator = lambda val: True): question = question.rstrip(" \t:") if default is not NotImplemented: question += " [%s]" % (default,) question += ": " while True: try: ans = readline(question).strip() except EOFError: ans = "" if not ans: if default is not NotImplemented: #sys.stdout.write("\b%s\n" % (default,)) return default else: continue try: ans = type(ans) except (TypeError, ValueError) as ex: sys.stdout.write("Invalid value (%s), please try again\n" % (ex,)) continue try: validator(ans) except ValueError as ex: sys.stdout.write("%s, please try again\n" % (ex,)) return ans def hexdump(data_or_stream, bytes_per_line = 16, aggregate = True): """Convert the given bytes (or a stream with a buffering ``read()`` method) to hexdump-formatted lines, with possible aggregation of identical lines. Returns a generator of formatted lines. """ if hasattr(data_or_stream, "read"): def read_chunk(): while True: buf = data_or_stream.read(bytes_per_line) if not buf: break yield buf else: def read_chunk(): for i in range(0, len(data_or_stream), bytes_per_line): yield data_or_stream[i:i + bytes_per_line] prev = None skipped = False for i, chunk in enumerate(read_chunk()): hexd = " ".join("%02x" % (ord(ch),) for ch in chunk) text = "".join(ch if 32 <= ord(ch) < 127 else "." for ch in chunk) if aggregate and prev == chunk: skipped = True continue prev = chunk if skipped: yield "*" yield "%06x | %s| %s" % (i * bytes_per_line, hexd.ljust(bytes_per_line * 3, " "), text) skipped = False def pager(rows, pagercmd = None): """Opens a pager (e.g., ``less``) to display the given text. Requires a terminal. :param rows: a ``bytes`` or a list/iterator of "rows" (``bytes``) :param pagercmd: the pager program to run. Defaults to ``less -RSin`` """ if not pagercmd: pagercmd = local["less"]["-RSin"] if hasattr(rows, "splitlines"): rows = rows.splitlines() pg = pagercmd.popen(stdout = None, stderr = None) try: for row in rows: line = "%s\n" % (row,) try: pg.stdin.write(line) pg.stdin.flush() except IOError: break pg.stdin.close() pg.wait() finally: try: rows.close() except Exception: pass if pg and pg.poll() is None: try: pg.terminate() except Exception: pass os.system("reset") plumbum-1.6.0/plumbum/cli/termsize.py0000644000232200023220000000500412610225666020216 0ustar debalancedebalance""" Terminal size utility --------------------- """ from __future__ import division, print_function, absolute_import import os import platform from struct import Struct def get_terminal_size(default=(80, 25)): """ Get width and height of console; works on linux, os x, windows and cygwin Adapted from https://gist.github.com/jtriley/1108174 Originally from: http://stackoverflow.com/questions/566746/how-to-get-console-window-width-in-python """ current_os = platform.system() if current_os == 'Windows': size = _get_terminal_size_windows() if not size: # needed for window's python in cygwin's xterm! size = _get_terminal_size_tput() elif current_os in ('Linux', 'Darwin', 'FreeBSD') or current_os.startswith('CYGWIN'): size = _get_terminal_size_linux() if size is None: # we'll assume the standard 80x25 if for any reason we don't know the terminal size size = default return size def _get_terminal_size_windows(): try: from ctypes import windll, create_string_buffer STDERR_HANDLE = -12 h = windll.kernel32.GetStdHandle(STDERR_HANDLE) csbi_struct = Struct("hhhhHhhhhhh") csbi = create_string_buffer(csbi_struct.size) res = windll.kernel32.GetConsoleScreenBufferInfo(h, csbi) if res: _, _, _, _, _, left, top, right, bottom, _, _ = csbi_struct.unpack(csbi.raw) return right - left + 1, bottom - top + 1 return None except Exception: return None def _get_terminal_size_tput(): # get terminal width # src: http://stackoverflow.com/questions/263890/how-do-i-find-the-width-height-of-a-terminal-window try: from plumbum.cmd import tput cols = int(tput('cols')) rows = int(tput('lines')) return (cols, rows) except Exception: return None def _ioctl_GWINSZ(fd): yx = Struct("hh") try: import fcntl import termios return yx.unpack(fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234')) except Exception: return None def _get_terminal_size_linux(): cr = _ioctl_GWINSZ(0) or _ioctl_GWINSZ(1) or _ioctl_GWINSZ(2) if not cr: try: fd = os.open(os.ctermid(), os.O_RDONLY) cr = _ioctl_GWINSZ(fd) os.close(fd) except Exception: pass if not cr: try: cr = (int(os.environ['LINES']), int(os.environ['COLUMNS'])) except Exception: return None return cr[1], cr[0] plumbum-1.6.0/plumbum/cli/application.py0000644000232200023220000006562212610225666020673 0ustar debalancedebalancefrom __future__ import division, print_function, absolute_import import os import sys import functools from textwrap import TextWrapper from collections import defaultdict from plumbum.lib import six, getdoc from plumbum.cli.terminal import get_terminal_size from plumbum.cli.switches import (SwitchError, UnknownSwitch, MissingArgument, WrongArgumentType, MissingMandatorySwitch, SwitchCombinationError, PositionalArgumentsError, switch, SubcommandError, Flag, CountOf) from plumbum import colors, local class ShowHelp(SwitchError): pass class ShowHelpAll(SwitchError): pass class ShowVersion(SwitchError): pass class SwitchParseInfo(object): __slots__ = ["swname", "val", "index"] def __init__(self, swname, val, index): self.swname = swname self.val = val self.index = index class Subcommand(object): def __init__(self, name, subapplication): self.name = name self.subapplication = subapplication def get(self): if isinstance(self.subapplication, str): modname, clsname = self.subapplication.rsplit(".", 1) mod = __import__(modname, None, None, "*") try: cls = getattr(mod, clsname) except AttributeError: raise ImportError("cannot import name %s" % (clsname,)) self.subapplication = cls return self.subapplication def __repr__(self): return "Subcommand(%r, %r)" % (self.name, self.subapplication) #=================================================================================================== # CLI Application base class #=================================================================================================== class Application(object): """ The base class for CLI applications; your "entry point" class should derive from it, define the relevant switch functions and attributes, and the ``main()`` function. The class defines two overridable "meta switches" for version (``-v``, ``--version``) and help (``-h``, ``--help``). The signature of the main function matters: any positional arguments (e.g., non-switch arguments) given on the command line are passed to the ``main()`` function; if you wish to allow unlimited number of positional arguments, use varargs (``*args``). The names of the arguments will be shown in the help message. The classmethod ``run`` serves as the entry point of the class. It parses the command-line arguments, invokes switch functions and enter ``main``. You should **not override** this method. Usage:: class FileCopier(Application): stat = Flag("p", "copy stat info as well") def main(self, src, dst): if self.stat: shutil.copy2(src, dst) else: shutil.copy(src, dst) if __name__ == "__main__": FileCopier.run() There are several class-level attributes you may set: * ``PROGNAME`` - the name of the program; if ``None`` (the default), it is set to the name of the executable (``argv[0]``), can be in color. If only a color, will be applied to the name. * ``VERSION`` - the program's version (defaults to ``1.0``, can be in color) * ``DESCRIPTION`` - a short description of your program (shown in help). If not set, the class' ``__doc__`` will be used. Can be in color. * ``USAGE`` - the usage line (shown in help) * ``COLOR_USAGE`` - The color of the usage line * ``COLOR_GROUPS`` - A dictionary that sets colors for the groups, like Meta-switches, Switches, and Subcommands A note on sub-commands: when an application is the root, its ``parent`` attribute is set to ``None``. When it is used as a nested-command, ``parent`` will point to be its direct ancestor. Likewise, when an application is invoked with a sub-command, its ``nested_command`` attribute will hold the chosen sub-application and its command-line arguments (a tuple); otherwise, it will be set to ``None`` """ PROGNAME = None DESCRIPTION = None VERSION = None USAGE = None COLOR_USAGE = None COLOR_GROUPS = None CALL_MAIN_IF_NESTED_COMMAND = True parent = None nested_command = None _unbound_switches = () def __init__(self, executable): # Filter colors if self.PROGNAME is None: self.PROGNAME = os.path.basename(executable) elif isinstance(self.PROGNAME, colors._style): self.PROGNAME = self.PROGNAME | os.path.basename(executable) elif colors.filter(self.PROGNAME) == '': self.PROGNAME = colors.extract(self.PROGNAME) | os.path.basename(executable) if self.DESCRIPTION is None: self.DESCRIPTION = getdoc(self) # Allow None for the colors self.COLOR_GROUPS=defaultdict(lambda:colors.do_nothing, dict() if type(self).COLOR_GROUPS is None else type(self).COLOR_GROUPS ) if type(self).COLOR_USAGE is None: self.COLOR_USAGE=colors.do_nothing self.executable = executable self._switches_by_name = {} self._switches_by_func = {} self._switches_by_envar = {} self._subcommands = {} for cls in reversed(type(self).mro()): for obj in cls.__dict__.values(): if isinstance(obj, Subcommand): name = colors.filter(obj.name) if name.startswith("-"): raise SubcommandError("Subcommand names cannot start with '-'") # it's okay for child classes to override subcommands set by their parents self._subcommands[name] = obj continue swinfo = getattr(obj, "_switch_info", None) if not swinfo: continue for name in swinfo.names: if name in self._unbound_switches: continue if name in self._switches_by_name and not self._switches_by_name[name].overridable: raise SwitchError("Switch %r already defined and is not overridable" % (name,)) self._switches_by_name[name] = swinfo self._switches_by_func[swinfo.func] = swinfo if swinfo.envname: self._switches_by_envar[swinfo.envname] = swinfo @property def root_app(self): return self.parent.root_app if self.parent else self @classmethod def unbind_switches(cls, *switch_names): """Unbinds the given switch names from this application. For example :: class MyApp(cli.Application): pass MyApp.unbind_switches("--version") """ cls._unbound_switches += tuple(name.lstrip("-") for name in switch_names if name) @classmethod def subcommand(cls, name, subapp = None): """Registers the given sub-application as a sub-command of this one. This method can be used both as a decorator and as a normal ``classmethod``:: @MyApp.subcommand("foo") class FooApp(cli.Application): pass Or :: MyApp.subcommand("foo", FooApp) .. versionadded:: 1.1 .. versionadded:: 1.3 The subcommand can also be a string, in which case it is treated as a fully-qualified class name and is imported on demand. For examples, MyApp.subcommand("foo", "fully.qualified.package.FooApp") """ def wrapper(subapp): attrname = "_subcommand_%s" % (subapp if isinstance(subapp, str) else subapp.__name__,) setattr(cls, attrname, Subcommand(name, subapp)) return subapp return wrapper(subapp) if subapp else wrapper def _parse_args(self, argv): tailargs = [] swfuncs = {} index = 0 while argv: index += 1 a = argv.pop(0) val = None if a == "--": # end of options, treat the rest as tailargs tailargs.extend(argv) break if a in self._subcommands: subcmd = self._subcommands[a].get() self.nested_command = (subcmd, [self.PROGNAME + " " + self._subcommands[a].name] + argv) break elif a.startswith("--") and len(a) >= 3: # [--name], [--name=XXX], [--name, XXX], [--name, ==, XXX], # [--name=, XXX], [--name, =XXX] eqsign = a.find("=") if eqsign >= 0: name = a[2:eqsign] argv.insert(0, a[eqsign:]) else: name = a[2:] swname = "--" + name if name not in self._switches_by_name: raise UnknownSwitch("Unknown switch %s" % (swname,)) swinfo = self._switches_by_name[name] if swinfo.argtype: if not argv: raise MissingArgument("Switch %s requires an argument" % (swname,)) a = argv.pop(0) if a and a[0] == "=": if len(a) >= 2: val = a[1:] else: if not argv: raise MissingArgument("Switch %s requires an argument" % (swname)) val = argv.pop(0) else: val = a elif a.startswith("-") and len(a) >= 2: # [-a], [-a, XXX], [-aXXX], [-abc] name = a[1] swname = "-" + name if name not in self._switches_by_name: raise UnknownSwitch("Unknown switch %s" % (swname,)) swinfo = self._switches_by_name[name] if swinfo.argtype: if len(a) >= 3: val = a[2:] else: if not argv: raise MissingArgument("Switch %s requires an argument" % (swname,)) val = argv.pop(0) elif len(a) >= 3: argv.insert(0, "-" + a[2:]) else: if a.startswith("-"): raise UnknownSwitch("Unknown switch %s" % (a,)) tailargs.append(a) continue # handle argument val = self._handle_argument(val, swinfo.argtype, name) if swinfo.func in swfuncs: if swinfo.list: swfuncs[swinfo.func].val[0].append(val) else: if swfuncs[swinfo.func].swname == swname: raise SwitchError("Switch %r already given" % (swname,)) else: raise SwitchError("Switch %r already given (%r is equivalent)" % ( swfuncs[swinfo.func].swname, swname)) else: if swinfo.list: swfuncs[swinfo.func] = SwitchParseInfo(swname, ([val],), index) elif val is NotImplemented: swfuncs[swinfo.func] = SwitchParseInfo(swname, (), index) else: swfuncs[swinfo.func] = SwitchParseInfo(swname, (val,), index) # Extracting arguments from environment variables envindex = 0 for env, swinfo in self._switches_by_envar.items(): envindex -= 1 envval = local.env.get(env) if envval is None: continue if swinfo.func in swfuncs: continue # skip if overridden by command line arguments val = self._handle_argument(envval, swinfo.argtype, env) envname = "$%s" % (env,) if swinfo.list: # multiple values over environment variables are not supported, # this will require some sort of escaping and separator convention swfuncs[swinfo.func] = SwitchParseInfo(envname, ([val],), envindex) elif val is NotImplemented: swfuncs[swinfo.func] = SwitchParseInfo(envname, (), envindex) else: swfuncs[swinfo.func] = SwitchParseInfo(envname, (val,), envindex) return swfuncs, tailargs @classmethod def autocomplete(cls, argv): """This is supplied to make subclassing and testing argument completion methods easier""" pass @staticmethod def _handle_argument(val, argtype, name): if argtype: try: return argtype(val) except (TypeError, ValueError): ex = sys.exc_info()[1] # compat raise WrongArgumentType("Argument of %s expected to be %r, not %r:\n %r" % ( name, argtype, val, ex)) else: return NotImplemented def _validate_args(self, swfuncs, tailargs): if six.get_method_function(self.help) in swfuncs: raise ShowHelp() if six.get_method_function(self.helpall) in swfuncs: raise ShowHelpAll() if six.get_method_function(self.version) in swfuncs: raise ShowVersion() requirements = {} exclusions = {} for swinfo in self._switches_by_func.values(): if swinfo.mandatory and not swinfo.func in swfuncs: raise MissingMandatorySwitch("Switch %s is mandatory" % ("/".join(("-" if len(n) == 1 else "--") + n for n in swinfo.names),)) requirements[swinfo.func] = set(self._switches_by_name[req] for req in swinfo.requires) exclusions[swinfo.func] = set(self._switches_by_name[exc] for exc in swinfo.excludes) # TODO: compute topological order gotten = set(swfuncs.keys()) for func in gotten: missing = set(f.func for f in requirements[func]) - gotten if missing: raise SwitchCombinationError("Given %s, the following are missing %r" % (swfuncs[func].swname, [self._switches_by_func[f].names[0] for f in missing])) invalid = set(f.func for f in exclusions[func]) & gotten if invalid: raise SwitchCombinationError("Given %s, the following are invalid %r" % (swfuncs[func].swname, [swfuncs[f].swname for f in invalid])) m = six.getfullargspec(self.main) max_args = six.MAXSIZE if m.varargs else len(m.args) - 1 min_args = len(m.args) - 1 - (len(m.defaults) if m.defaults else 0) if len(tailargs) < min_args: raise PositionalArgumentsError("Expected at least %d positional arguments, got %r" % (min_args, tailargs)) elif len(tailargs) > max_args: raise PositionalArgumentsError("Expected at most %d positional arguments, got %r" % (max_args, tailargs)) # Positional arguement validataion if hasattr(self.main, 'positional'): tailargs = self._positional_validate(tailargs, self.main.positional, self.main.positional_varargs, m.args[1:], m.varargs) elif hasattr(m, 'annotations'): args_names = list(m.args[1:]) positional = [None]*len(args_names) varargs = None # All args are positional, so convert kargs to positional for item in m.annotations: if item == m.varargs: varargs = m.annotations[item] else: positional[args_names.index(item)] = m.annotations[item] tailargs = self._positional_validate(tailargs, positional, varargs, m.args[1:], m.varargs) ordered = [(f, a) for _, f, a in sorted([(sf.index, f, sf.val) for f, sf in swfuncs.items()])] return ordered, tailargs def _positional_validate(self, args, validator_list, varargs, argnames, varargname): """Makes sure args follows the validation given input""" out_args = list(args) for i in range(min(len(args),len(validator_list))): if validator_list[i] is not None: out_args[i] = self._handle_argument(args[i], validator_list[i], argnames[i]) if len(args) > len(validator_list): if varargs is not None: out_args[len(validator_list):] = [ self._handle_argument(a, varargs, varargname) for a in args[len(validator_list):]] else: out_args[len(validator_list):] = args[len(validator_list):] return out_args @classmethod def run(cls, argv = None, exit = True): # @ReservedAssignment """ Runs the application, taking the arguments from ``sys.argv`` by default if nothing is passed. If ``exit`` is ``True`` (the default), the function will exit with the appropriate return code; otherwise it will return a tuple of ``(inst, retcode)``, where ``inst`` is the application instance created internally by this function and ``retcode`` is the exit code of the application. .. note:: Setting ``exit`` to ``False`` is intendend for testing/debugging purposes only -- do not override it other situations. """ if argv is None: argv = sys.argv cls.autocomplete(argv) argv = list(argv) inst = cls(argv.pop(0)) retcode = 0 try: swfuncs, tailargs = inst._parse_args(argv) ordered, tailargs = inst._validate_args(swfuncs, tailargs) except ShowHelp: inst.help() except ShowHelpAll: inst.helpall() except ShowVersion: inst.version() except SwitchError: ex = sys.exc_info()[1] # compatibility with python 2.5 print("Error: %s" % (ex,)) print("------") inst.help() retcode = 2 else: for f, a in ordered: f(inst, *a) cleanup = None if not inst.nested_command or inst.CALL_MAIN_IF_NESTED_COMMAND: retcode = inst.main(*tailargs) cleanup = functools.partial(inst.cleanup, retcode) if not retcode and inst.nested_command: subapp, argv = inst.nested_command subapp.parent = inst inst, retcode = subapp.run(argv, exit = False) if cleanup: cleanup() if retcode is None: retcode = 0 if exit: sys.exit(retcode) else: return inst, retcode @classmethod def invoke(cls, *args, **switches): """Invoke this application programmatically (as a function), in the same way ``run()`` would. There are two key differences: the return value of ``main()`` is not converted to an integer (returned as-is), and exceptions are not swallowed either. :param args: any positional arguments for ``main()`` :param switches: command-line switches are passed as keyword arguments, e.g., ``foo=5`` for ``--foo=5`` """ inst = cls("") swfuncs = inst._parse_kwd_args(switches) ordered, tailargs = inst._validate_args(swfuncs, args) for f, a in ordered: f(inst, *a) cleanup = None if not inst.nested_command or inst.CALL_MAIN_IF_NESTED_COMMAND: retcode = inst.main(*tailargs) cleanup = functools.partial(inst.cleanup, retcode) if not retcode and inst.nested_command: subapp, argv = inst.nested_command subapp.parent = inst inst, retcode = subapp.run(argv, exit = False) if cleanup: cleanup() return inst, retcode def _parse_kwd_args(self, switches): """Parses keywords (positional arguments), used by invoke.""" swfuncs = {} for index, (swname, val) in enumerate(switches.items(), 1): switch = getattr(type(self), swname) swinfo = self._switches_by_func[switch._switch_info.func] if isinstance(switch, CountOf): p = (range(val),) elif swinfo.list and not hasattr(val, "__iter__"): raise SwitchError("Switch %r must be a sequence (iterable)" % (swname,)) elif not swinfo.argtype: # a flag if val not in (True, False, None, Flag): raise SwitchError("Switch %r is a boolean flag" % (swname,)) p = () else: p = (val,) swfuncs[swinfo.func] = SwitchParseInfo(swname, p, index) return swfuncs def main(self, *args): """Implement me (no need to call super)""" if self._subcommands: if args: print("Unknown sub-command %r" % (args[0],)) print("------") self.help() return 1 if not self.nested_command: print("No sub-command given") print("------") self.help() return 1 else: print("main() not implemented") return 1 def cleanup(self, retcode): """Called after ``main()`` and all subapplications have executed, to perform any necessary cleanup. :param retcode: the return code of ``main()`` """ @switch(["--help-all"], overridable = True, group = "Meta-switches") def helpall(self): """Print help messages of all subcommands and quit""" self.help() print("") if self._subcommands: for name, subcls in sorted(self._subcommands.items()): subapp = (subcls.get())("%s %s" % (self.PROGNAME, name)) subapp.parent = self for si in subapp._switches_by_func.values(): if si.group == "Meta-switches": si.group = "Hidden-switches" subapp.helpall() @switch(["-h", "--help"], overridable = True, group = "Meta-switches") def help(self): # @ReservedAssignment """Prints this help message and quits""" if self._get_prog_version(): self.version() print("") if self.DESCRIPTION: print(self.DESCRIPTION.strip() + '\n') m = six.getfullargspec(self.main) tailargs = m.args[1:] # skip self if m.defaults: for i, d in enumerate(reversed(m.defaults)): tailargs[-i - 1] = "[%s=%r]" % (tailargs[-i - 1], d) if m.varargs: tailargs.append("%s..." % (m.varargs,)) tailargs = " ".join(tailargs) with self.COLOR_USAGE: print("Usage:") if not self.USAGE: if self._subcommands: self.USAGE = " %(progname)s [SWITCHES] [SUBCOMMAND [SWITCHES]] %(tailargs)s\n" else: self.USAGE = " %(progname)s [SWITCHES] %(tailargs)s\n" print(self.USAGE % {"progname": colors.filter(self.PROGNAME), "tailargs": tailargs}) by_groups = {} for si in self._switches_by_func.values(): if si.group not in by_groups: by_groups[si.group] = [] by_groups[si.group].append(si) def switchs(by_groups, show_groups): for grp, swinfos in sorted(by_groups.items(), key = lambda item: item[0]): if show_groups: print(self.COLOR_GROUPS[grp] | grp) for si in sorted(swinfos, key = lambda si: si.names): swnames = ", ".join(("-" if len(n) == 1 else "--") + n for n in si.names if n in self._switches_by_name and self._switches_by_name[n] == si) if si.argtype: if isinstance(si.argtype, type): typename = si.argtype.__name__ else: typename = str(si.argtype) argtype = " %s:%s" % (si.argname.upper(), typename) else: argtype = "" prefix = swnames + argtype yield si, prefix, self.COLOR_GROUPS[grp] if show_groups: print("") sw_width = max(len(prefix) for si, prefix, color in switchs(by_groups, False)) + 4 cols, _ = get_terminal_size() description_indent = " %s%s%s" wrapper = TextWrapper(width = max(cols - min(sw_width, 60), 50) - 6) indentation = "\n" + " " * (cols - wrapper.width) for si, prefix, color in switchs(by_groups, True): help = si.help # @ReservedAssignment if si.list: help += "; may be given multiple times" if si.mandatory: help += "; required" if si.requires: help += "; requires %s" % (", ".join((("-" if len(s) == 1 else "--") + s) for s in si.requires)) if si.excludes: help += "; excludes %s" % (", ".join((("-" if len(s) == 1 else "--") + s) for s in si.excludes)) msg = indentation.join(wrapper.wrap(" ".join(l.strip() for l in help.splitlines()))) if len(prefix) + wrapper.width >= cols: padding = indentation else: padding = " " * max(cols - wrapper.width - len(prefix) - 4, 1) print(description_indent % (color | prefix, padding, color | msg)) if self._subcommands: gc = self.COLOR_GROUPS["Subcommands"] print(gc | "Subcommands:") for name, subcls in sorted(self._subcommands.items()): with gc: subapp = subcls.get() doc = subapp.DESCRIPTION if subapp.DESCRIPTION else getdoc(subapp) help = doc + "; " if doc else "" # @ReservedAssignment help += "see '%s %s --help' for more info" % (self.PROGNAME, name) msg = indentation.join(wrapper.wrap(" ".join(l.strip() for l in help.splitlines()))) if len(name) + wrapper.width >= cols: padding = indentation else: padding = " " * max(cols - wrapper.width - len(name) - 4, 1) print(description_indent % (subcls.name, padding, gc | colors.filter(msg))) def _get_prog_version(self): ver = None curr = self while curr is not None: ver = getattr(curr, "VERSION", None) if ver is not None: return ver curr = curr.parent return ver @switch(["-v", "--version"], overridable = True, group = "Meta-switches") def version(self): """Prints the program's version and quits""" ver = self._get_prog_version() ver_name = ver if ver is not None else "(version not set)" print('{0} {1}'.format(self.PROGNAME, ver_name)) plumbum-1.6.0/plumbum/version.py0000644000232200023220000000013412610225666017271 0ustar debalancedebalanceversion = (1, 6, 0) version_string = ".".join(map(str,version)) release_date = "2015.10.16"