sclapp-0.5.3/0000755000175000017500000000000011041121232011026 5ustar fabfabsclapp-0.5.3/CHANGELOG0000644000175000017500000000475211041121232012250 0ustar fabfabChanges since 0.3.4 =================== * Added sub-module redirection.py to __all__; it is public. * Updated XSLT to include documentation from the sub-module redirection.py. * Added a docstring to function redirectFds(). Changes since 0.3.3 =================== * Added announce_signals argument to enableSignalHandling(); this restores the functionality of the old enableAnnounceSignals(), disableAnnounceSignals() functions. * Changed ALL from 0 to 1. * Fixed announce_signals not bracketing with enableSignalHandling(), disableSignalHandling(). * Moved redirectFds() into its own module: redirection.py. Changes since 0.3.2 =================== * doc/Makefile: parameterized teudng as TEUDNG. * doc/xsl/docbook/teudng.xsl: Removed unnecessary localization. Changes since 0.3.1 =================== * Fixed bug in Makefile: /usr/lib/python-2.4 -> /usr/lib/python2.4 . Changes since 0.3.0 =================== * Fixed bug when assigning signal handler for some signals on some systems: sclapp now catches the RuntimeError that can result. Changes since 0.2.3 =================== * Implemented BackgroundProcess class. * Major changes in API -- see documentation. Changes since 0.2.2 =================== * Fixed a bug where sclapp prints documentation twice for -h/--help option, unnecessarily prints documenation for -v/--version option. Changes since 0.2.1 =================== * Added sclapp.main() option ignore_unrecognized_options. * Raising a UsageError now causes sclapp to print doc (as if -h had been specified on the command line), if it is specified. * Tweaked Makefile: doc target now depends on version file. Changes since 0.2.0 =================== * Fixed bug where sclapp.__version__ was bound to module sclapp.version instead of string sclapp.version.version. Changes since 0.1.1 =================== * Major rewrite; no stone left unturned: - more reliable signal handling - substantially improved documentation - more & better tests - output protection is now transparent (sys.stdout and sys.stderr are replaced with compatible objects) - sclapp will now handle help and version command line options if desired - daemonization functionality included for those who want it - all sclapp messages are now customizable (version, bug, doc) - many other small changes Changes since 0.1.0 =================== * Renamed SimpleCommandLineApp to sclapp to reduce naming convention compatibility problems. Changes since 0.0.0 =================== * Initial release. sclapp-0.5.3/COPYING0000644000175000017500000004313311041121232012065 0ustar fabfab GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. sclapp-0.5.3/NEWS0000644000175000017500000001103211041121232011522 0ustar fabfab==================== sclapp Release Notes ==================== .. contents:: sclapp 0.5.3 2008-07-21 ======================= * BackgroundFunction: make constructor arguments ``args``, ``kwargs`` optional. They default to an empty tuple, dict (respectively). (Forest Bond) * Shell: rather than wrapping ``pexpect.spawn``, sub-class it. (Forest Bond) * Shell: call ``self.exit()`` in ``__del__`` in case caller didn't clean up properly. (Forest Bond) * Shell: Don't print "Exiting interactive shell..." when exiting interactive shell. (Forest Bond) * Shell: implement method ``interact_with``, which starts an interactive session with a command executed within a sub-shell. (Forest Bond) * ``Shell.interact``, ``Shell.interact_with``: check exit status of sub-shell. If ``failure_exceptions`` or ``signal_exceptions`` are True, raise ``CommandFailed`` or ``CommandSignalled`` exceptions as appropriate. Otherwise, return the exit status of the command/shell executed. (Forest Bond) sclapp 0.5.2 2008-05-19 ======================= * Shell: save interpolated command as last command. This makes CommandFailed and CommandSignalled exceptions present the fully interpolated command, and causes functions that return the command (execute, follow, etc.) to return that instead of the uninterpolated version. (Forest Bond) * Shell: use pre-adjusted command for trace. This causes curly braces that are added to the command before execution to be omitted from the trace output. (Forest Bond) * Shell: follow* methods now support keyword argument ``follow_input``. If this argument is True, the executed command will be present in the output stream of characters. The default is False. (Forest Bond) * Shell.interact: wrap underlying pexpect.spawn.interact in a way that makes the resulting interactive shell useful to humans. Previously, calling this method presented a fragile interactive shell lacking much usability. (Forest Bond) * Shell: the default pexpect behavior is now overridden for ``__del__``. pexpect closes the pty device from ``__del__``, but this mechanism is unreliable, as ``__del__`` is not guaranteed to be called under all circumstances. (Forest Bond) * Shell.interact: support keyword argument ``fitted``. This causes the shell to be passed SIGWINCH each time such a signal is received, as well as for the initial window size of the shell to be set to that of sys.stdout. This argument defaults to False. (Forest Bond) * logWrapper: correctly pass along return value from wrapped function. (Forest Bond) * Shell now supports keyword argument delaybeforesend. This sets a delay that is used before any input is sent to the shell process. The default delay is determined by the underlying pexpect library. (Forest Bond) * Integrate tests into setup.py. Tests can now be run with ``./setup.py test``. (Forest Bond) * Provide sclapp.locale, which is just the standard locale module, however, it is imported in such a way as to force Darwin systems to respect the LC_* and LANG environment variables. Callers are encouraged to import the locale module from sclapp, rather than importing it directly. (Forest Bond) * Shell.exit now calls ``close``, which closes the pty device. (Forest Bond) * Provide ``--tests`` option to setup.py test sub-command to enable specific tests to be run, rather than always running the entire test suite. sclapp 0.5.1 2008-03-07 ======================= * Fixed Shell trace output does not include interpolated values. (Forest Bond) * Implemented ``decode_argv`` for ``main_function``, ``mainWrapper``. Members of the ``argv`` argument to main function will be decoded to Unicode unless ``decode_argv`` is set to False. (Forest Bond) * Fixed UnicodeEncodeError with Shell.followWrite, Shell.followWriteReturn. (Forest Bond) * Fixed bug in setup.py causing version information to be excluded from generated .egg file. (Forest Bond) * Fleshed out more metadata in setup.py; the following fields were added: - author - author_email - url - license - description (Forest Bond) * Shell: Fix UnicodeEncodeError when interpolating unicode strings into byte strings. (Forest Bond) * ``Shell.exit``: fix call to wrongly-named method ``sendEOF``; should have been ``sendeof``. (Forest Bond) sclapp 0.5.0 2007-12-16 ======================= * Added NEWS file; sorry for the previous absence. (Forest Bond) * [NEWS file not present] sclapp-0.5.3/README0000644000175000017500000000272011041121232011707 0ustar fabfab============= sclapp README ============= ---------------------------------------------- http://www.alittletooquiet.net/software/sclapp ---------------------------------------------- sclapp is a Python module that makes it easier to write well-behaved command-line applications. This file may be distributed under the same license as sclapp itself. Installing ========== Before installing from source, check if your distribution has packages available. It is not normally recommended that you install packages from source in system-wide directories, unless you know what you're doing. To build:: ./setup.py build To install:: ./setup.py install To clean up temporary files created while building or testing:: ./setup.py clean To clean all files, including built files that are required for installation:: ./setup.py clean --all Running Tests ============= Tests can be run via setup.py:: ./setup.py test Specific tests can be specified on the command-line. For instance, to only run tests defined in module tests.background_command:: ./setup.py test --tests tests.background_command To only run tests defined by test case BackgroundCommandTestCase:: ./setup.py test --tests tests.background_command.BackgroundCommandTestCase To only run a specific test defined by that test case:: ./setup.py test --tests tests.background_command.BackgroundCommandTestCase.test_basic_operation Multiple identifiers can be specified using a comma-separated list. sclapp-0.5.3/release0000644000175000017500000000000611041121232012365 0ustar fabfab0.5.3 sclapp-0.5.3/sclapp/0000755000175000017500000000000011041121232012310 5ustar fabfabsclapp-0.5.3/sclapp/__init__.py0000644000175000017500000000650611041121232014430 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. '''An easy-to-use framework for python command line applications.''' # XXX: This work-around causes darwin to use the LC_* and LANG environment # variables as on other Unix-like systems. Without this, # locale.getpreferredencoding always returns "mac-roman". See: # # http://www.selenic.com/mercurial/wiki/index.cgi/Character_Encoding_On_OSX import sys if sys.platform == 'darwin': sys.platform = 'generic' import locale sys.platform = 'darwin' else: import locale __all__ = ( '__version__', # from sclapp.exceptions: 'CriticalError', 'UsageError', 'SignalError', 'ExitSignalError', # from sclapp.protected_output: 'protectOutput', 'unprotectOutput', # from sclapp.error_output: 'printDebug', 'printInfo', 'printWarning', 'printError', 'printCritical', 'setErrorOutputLevel', 'ALL', 'SCLAPP_DEBUG', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL', # from sclapp.signals: 'getExitSignals', 'getNotifySignals', 'getDefaultSignals', 'getIgnoreSignals', 'getCaughtSignals', 'enableSignalHandling', 'disableSignalHandling', 'ALL_SIGNALS', 'STD_EXIT_SIGNALS', 'STD_NOTIFY_SIGNALS', 'STD_DEFAULT_SIGNALS', 'STD_IGNORE_SIGNALS', # from sclapp.main: 'mainWrapper', 'main_function', 'makeSubCommandMain', # public sub-modules: 'daemonize', 'processes', 'redirection', 'termcontrol', 'services', 'stdio_encoding', 'shinterp', # Allow callers to import locale from sclapp (see work-around above): 'locale', ) from sclapp.main import mainWrapper, main_function, makeSubCommandMain from sclapp import termcontrol, services, daemonize, processes, redirection, \ stdio_encoding, shinterp try: import shell except ImportError: pass else: __all__ = __all__ + ('shell',) from sclapp.exceptions import CriticalError, UsageError, SignalError, \ ExitSignalError from sclapp.protected_output import protectOutput, unprotectOutput from sclapp.error_output import printDebug, printInfo, printWarning, \ printError, printCritical, setErrorOutputLevel, \ ALL, SCLAPP_DEBUG, DEBUG, INFO, WARNING, ERROR, CRITICAL from sclapp.signals import getExitSignals, getNotifySignals, \ getDefaultSignals, getIgnoreSignals, getCaughtSignals, \ enableSignalHandling, disableSignalHandling, \ ALL_SIGNALS, STD_EXIT_SIGNALS, STD_NOTIFY_SIGNALS, STD_DEFAULT_SIGNALS, \ STD_IGNORE_SIGNALS from sclapp import signals try: from sclapp import version except ImportError: __version__ = '0.0.0' else: __version__ = version.version def debug(): '''Causes sclapp to enter debug mode. Currently, this means that sclapp will wrap it's signal handlers with the logWrapper() function from the sclapp.debug_logging module. ''' from sclapp.debug_logging import logWrapper signals._exitSignalHandler = logWrapper( signals._exitSignalHandler) signals._notifySignalHandler = logWrapper( signals._notifySignalHandler) signals._ignoreExitSignalHandler = logWrapper( signals._ignoreExitSignalHandler) sclapp-0.5.3/sclapp/daemonize.py0000644000175000017500000000211311041121232014632 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys from sclapp.error_output import printInfo from sclapp.processes import redirectFds def getRootDirectory(): '''Tries to find the root directory by moving up the directory tree starting from the current working directory. ''' p = os.getcwd() _p = None while p != _p: _p = p p = os.path.dirname(p) return p def daemonize(): '''Performs traditional *nix daemonization. For more information, see http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16. ''' printInfo('daemonizing...') if os.fork(): os._exit(0) os.setsid() if os.fork(): os._exit(0) os.chdir(getRootDirectory()) os.umask(0) redirectFds(os.devnull, os.devnull, os.devnull) sclapp-0.5.3/sclapp/debug_logging.py0000644000175000017500000000200211041121232015450 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os DEBUG_LOGFILE = u'logfile' def logMessage(message): logfile = file(DEBUG_LOGFILE, 'a') logfile.write(u'%s\n' % message) logfile.close() def readLogFile(): logfile = file(DEBUG_LOGFILE, 'r') contents = logfile.read() logfile.close() return contents def removeLogFile(): return os.remove(DEBUG_LOGFILE) def logWrapper(fn): def newFn(*args, **kwargs): logMessage(u'entering %s' % fn.__name__) logMessage(u'args: %s' % unicode(args)) logMessage(u'kwargs: %s' % unicode(kwargs)) try: return fn(*args, **kwargs) finally: logMessage(u'exiting %s' % fn.__name__) return newFn sclapp-0.5.3/sclapp/error_output.py0000644000175000017500000000414011041121232015432 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import logging def setErrorOutputLevel(level): '''setErrorOutputLevel(level) -> None level: int minimum priority of error messages that are displayed Sets the minimum priority required for an error output message to actually be seen. sclapp defines the following constants for convenience: ALL = 1 SCLAPP_DEBUG = 5 DEBUG = 10 INFO = 20 WARNING = 30 ERROR = 40 CRITICAL = 50 ''' return _logger.setLevel(level) ALL = 1 SCLAPP_DEBUG = 5 DEBUG = logging.DEBUG INFO = logging.INFO WARNING = logging.WARNING ERROR = logging.ERROR CRITICAL = logging.CRITICAL _logger = logging.getLogger('sclapp') _console = logging.StreamHandler() _formatter = logging.Formatter('%(message)s') _console.setFormatter(_formatter) _logger.addHandler(_console) setErrorOutputLevel(DEBUG) def _printSclappDebug(message): '''_printSclappDebug(message) -> None Prints a message to stderr with priority SCLAPP_DEBUG. ''' return _logger.log(SCLAPP_DEBUG, message) def printDebug(message): '''printDebug(message) -> None Prints a message to stderr with priority DEBUG. ''' return _logger.debug(message) def printInfo(message): '''printInfo(message) -> None Prints a message to stderr with priority INFO. ''' return _logger.info(message) def printWarning(message): '''printWarning(message) -> None Prints a message to stderr with priority WARNING. ''' return _logger.warning(message) def printError(message): '''printError(message) -> None Prints a message to stderr with priority ERROR. ''' return _logger.error(message) def printCritical(message): '''printCritical(message) -> None Prints a message to stderr with priority CRITICAL. ''' return _logger.critical(message) sclapp-0.5.3/sclapp/exceptions.py0000644000175000017500000000346711041121232015055 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. '''This module implements sclapp exceptions.''' from sclapp.util import safe_encode, safe_decode class Error(Exception): '''Base exception for other exceptions defined in this module.''' class CriticalError(Error): '''Indicates a fatal error.''' code = None message = None def __init__(self, code = 0, message = u''): if type(code) != int: raise ValueError, ( u'first argument to CriticalError initializer ' 'should be an integer' ) self.code = code self.message = message Error.__init__(self) def __str__(self): return safe_encode(self.message) def __unicode__(self): return safe_decode(self.message) class UsageError(CriticalError): '''Indicates a usage error.''' def __init__(self, message = None): if message is not None: code = -1 else: code = 0 CriticalError.__init__(self, code, message) class SignalError(Error): '''Indicates that a notify signal has been caught.''' def __init__(self, signum): self.signum = signum Error.__init__(self) def __str__(self): return 'caught signal %i' % self.signum class ExitSignalError(CriticalError): '''Indicates that an exit signal has been caught.''' def __init__(self, signum): self.signum = signum CriticalError.__init__(self) def __str__(self): return 'caught exit signal %i' % self.signum sclapp-0.5.3/sclapp/legacy_support.py0000644000175000017500000001305111041121232015722 0ustar fabfab'''Miscellaenous functionality required for support of legacy Python versions. ''' # The following two functions are copyright 2007 Forest Bond, and can be # distributed under the same terms as the rest of the sclapp package. def wraps(wrapped): '''A replacement implementation for functools.wraps, which was introduced in Python 2.5. ''' def decorator(fn): try: fn.__name__ = wrapped.__name__ except TypeError: pass try: fn.__doc__ = wrapped.__doc__ except TypeError: pass try: fn.__module__ = wrapped.__module__ except TypeError: pass return fn return decorator def reversed(l1): l2 = list(l1) l2.reverse() return l2 # The rest of the code in this file was taken from the Python 2.4 standard # library. As such, it is distributed under the terms of the Python Software # Foundation License Version 2. The full text of this license is distributed # with Python itself, and is also available from the Python website: # http://www.python.org import sys from traceback import format_exception def format_exc(limit=None): """Like print_exc() but return a string.""" try: etype, value, tb = sys.exc_info() return ''.join(format_exception(etype, value, tb, limit)) finally: etype = value = tb = None import re as _re class _multimap: """Helper class for combining multiple mappings. Used by .{safe_,}substitute() to combine the mapping and keyword arguments. """ def __init__(self, primary, secondary): self._primary = primary self._secondary = secondary def __getitem__(self, key): try: return self._primary[key] except KeyError: return self._secondary[key] class _TemplateMetaclass(type): pattern = r""" %(delim)s(?: (?P%(delim)s) | # Escape sequence of two delimiters (?P%(id)s) | # delimiter and a Python identifier {(?P%(id)s)} | # delimiter and a braced identifier (?P) # Other ill-formed delimiter exprs ) """ def __init__(cls, name, bases, dct): super(_TemplateMetaclass, cls).__init__(name, bases, dct) if 'pattern' in dct: pattern = cls.pattern else: pattern = _TemplateMetaclass.pattern % { 'delim' : _re.escape(cls.delimiter), 'id' : cls.idpattern, } cls.pattern = _re.compile(pattern, _re.IGNORECASE | _re.VERBOSE) class Template: """A string class for supporting $-substitutions.""" __metaclass__ = _TemplateMetaclass delimiter = '$' idpattern = r'[_a-z][_a-z0-9]*' def __init__(self, template): self.template = template # Search for $$, $identifier, ${identifier}, and any bare $'s def _invalid(self, mo): i = mo.start('invalid') lines = self.template[:i].splitlines(True) if not lines: colno = 1 lineno = 1 else: colno = i - len(''.join(lines[:-1])) lineno = len(lines) raise ValueError('Invalid placeholder in string: line %d, col %d' % (lineno, colno)) def substitute(self, *args, **kws): if len(args) > 1: raise TypeError('Too many positional arguments') if not args: mapping = kws elif kws: mapping = _multimap(kws, args[0]) else: mapping = args[0] # Helper function for .sub() def convert(mo): # Check the most common path first. named = mo.group('named') or mo.group('braced') if named is not None: val = mapping[named] # We use this idiom instead of str() because the latter will # fail if val is a Unicode containing non-ASCII characters. return '%s' % val if mo.group('escaped') is not None: return self.delimiter if mo.group('invalid') is not None: self._invalid(mo) raise ValueError('Unrecognized named group in pattern', self.pattern) return self.pattern.sub(convert, self.template) def safe_substitute(self, *args, **kws): if len(args) > 1: raise TypeError('Too many positional arguments') if not args: mapping = kws elif kws: mapping = _multimap(kws, args[0]) else: mapping = args[0] # Helper function for .sub() def convert(mo): named = mo.group('named') if named is not None: try: # We use this idiom instead of str() because the latter # will fail if val is a Unicode containing non-ASCII return '%s' % mapping[named] except KeyError: return self.delimiter + named braced = mo.group('braced') if braced is not None: try: return '%s' % mapping[braced] except KeyError: return self.delimiter + '{' + braced + '}' if mo.group('escaped') is not None: return self.delimiter if mo.group('invalid') is not None: return self.delimiter raise ValueError('Unrecognized named group in pattern', self.pattern) return self.pattern.sub(convert, self.template) sclapp-0.5.3/sclapp/main.py0000644000175000017500000002315211041121232013611 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. '''sclapp's mainWrapper()''' import os, sys, logging, errno, signal, locale from sclapp.exceptions import CriticalError, UsageError from sclapp.signals import enableSignalHandling, disableSignalHandling, \ getExitSignals from sclapp.error_output import printDebug, printCritical, setErrorOutputLevel from sclapp.protected_output import protectOutput, unprotectOutput import sclapp.daemonize from sclapp.stdio_encoding import enableStdioEncoding def mainWrapper( realmain, name = None, author = u'the author', version = None, doc = None, handle_signals = True, exit_signals = None, notify_signals = None, default_signals = None, ignore_signals = None, protect_output = True, daemonize = False, bug_message = ( u'${traceback}\n' u'Something bad happened, and is most likely a bug.\n' u'Please file a bug report to ${author}.\n' u'Include the error message(s) printed here.' ), version_message = u'${name} version ${version}', ignore_unrecognized_options = True, error_output_level = None, announce_signals = True, decode_stdin = True, encode_stdout = True, encode_stderr = True, decode_argv = True, ): ''' Some of the optional parameters mentioned above cause sclapp to parse argv and respond appropriately: * If version_message is not None, sclapp will respond to the -v command line switch by printing the version_message to stdout. * If doc is not None, sclapp will respond to the -h command line switch by printing doc to stdout. Both bug_message and version_message are parsed for substitution strings prior to printing. Specifically, sclapp uses the Template class of the standard library's string module to make the following substitutions: ${name} is replaced by the name parameter ${author} is replaced by the author parameter ${version} is replaced by the version parameter ${doc} is replaced by the doc parameters The doc parameter is parsed for all other substitutions before it is itself substituted. Thus, callers should feel free to add ${name}, ${author}, and ${version} to the doc parameter, and they will be replaced appropriately before printing. Be default, sclapp installs signal handlers that are easily configurable using module functions. Setting handle_signals to False will disable this behavior. sclapp can also be used to write simple daemons. To cause the program to daemonize before launching the wrapped main() function, set daemonize to True. bug_message, if not None, will be printed in the event that an unhandled exception is caught by the main wrapper. The default message simply informs the user that a likely bug has been encountered, and asks them to file a bug report to the author. If ignore_unrecognized_options is False, sclapp will complain about options it isn't expecting. If your program does not explicitly handle command line options, you probably want this. Otherwise, you definately don't want this, as any of your options will cause a UsageError to be reported before you have a chance to handle them. decode_stdin, encode_stdout, and encode_stderr determine whether or not sclapp enables on-the-fly stdio en/decoding. They default to True. If decode_argv is True, members of the argv argument passed to the main function will be decoded to Unicode. It defaults to True. If sclapp finds that it can't do anything useful with -h/--help or -v/--version (if version, doc are None, or contain substitutions sclapp can't fulfill), sclapp does nothing with command line options. The new main() function will return the wrapped function's return value, unless an exception is raised (including exceptions resulting from an exit signal being received), in which case the return value is an integer intended to act as the program's termination status. ''' # functools was added in Python 2.5 try: from functools import wraps except ImportError: from sclapp.legacy_support import wraps # Template was added in Python 2.4 try: from string import Template except ImportError: from sclapp.legacy_support import Template try: from traceback import format_exc except ImportError: from sclapp.legacy_support import format_exc encoding = locale.getpreferredencoding() argv = sys.argv if decode_argv: argv = [unicode(a, encoding) for a in argv] if name is None: name = os.path.basename(argv[0]) def _sub(message): mapping = { } if name is not None: mapping['name'] = name if author is not None: mapping['author'] = author if version is not None: mapping['version'] = version mapping['traceback'] = format_exc() return Template(message).safe_substitute(mapping) def _handleArgv(argv): for arg in argv: if arg.startswith('-') and ( arg not in ('-h', '-v', '--help', '--version')): if not ignore_unrecognized_options: raise UsageError, u'unrecognized option: %s' % arg if (version_message is not None) and (version is not None): try: version_message_out = _sub(version_message) except (KeyError, ValueError), e: pass else: if ('-v' in argv) or ('--version' in argv): print version_message_out raise CriticalError, (0, None) if doc is not None: doc_out = _sub(doc) if doc_out is not None: if ('-h' in argv) or ('--help' in argv): print doc_out raise CriticalError, (0, None) #@wraps(realmain) def main(argv = argv): try: enableStdioEncoding( decode_stdin = decode_stdin, encode_stdout = encode_stdout, encode_stderr = encode_stderr ) try: if error_output_level is not None: setErrorOutputLevel(error_output_level) if os.name == 'posix' and handle_signals: enableSignalHandling( exit_signals = exit_signals, notify_signals = notify_signals, default_signals = default_signals, ignore_signals = ignore_signals, announce_signals = announce_signals ) if ( not protect_output ) and ( hasattr(signal, 'SIGPIPE') ) and ( signal.SIGPIPE in getExitSignals() ): raise AssertionError, ( 'Protected output must be enabled to reliably ' 'handle SIGPIPE as an exit signal.' ) if daemonize: sclapp.daemonize.daemonize() if protect_output: protectOutput() _handleArgv(argv) status = realmain(argv) except KeyboardInterrupt, e: raise CriticalError, (0, unicode(e)) except UsageError, e: if doc is not None: doc_out = _sub(doc) print doc_out raise except SystemExit, e: raise except CriticalError, e: if e.code != 0: printCritical(unicode(e)) status = e.code except: if bug_message is not None: msg = _sub(bug_message) else: msg = format_exc() try: printCritical(msg) except: print msg raise status = 255 if protect_output: unprotectOutput() logging.shutdown() if status is not None and status != 0: printDebug(u'exiting with status %i' % int(status)) return status main = wraps(realmain)(main) return main def main_function(*args, **kwargs): '''Decorator for main functions. Both of these variations should work: @main_function def main(argv): return 0 @main_function(name = 'myprog') def main(argv): return 0 ''' if (len(args) == 1) and (len(kwargs) == 0): return mainWrapper(args[0]) elif (len(args) > 1): raise ValueError, ( 'main_function accepts either a signle positional arg or zero or ' 'more keyword args' ) def decorator(fn): return mainWrapper(fn, **kwargs) return decorator def makeSubCommandMain(cmds, initialize = None, default = None): def main(argv): if len(argv) < 2: if default is None: raise UsageError, ( u'expected: one of %s' % ', '.join(cmds.keys())) else: cmd = default else: cmd = argv[1] try: cmd_fn = cmds[cmd] except KeyError: raise UsageError, u'no such sub-command: %s' % cmd if initialize is not None: initialize(argv) return cmd_fn(argv) return main sclapp-0.5.3/sclapp/pipes.py0000644000175000017500000000715611041121232014013 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. '''A module that implements some strange functions for turning Python functions into sub-processes that can be strung together with pipes. Not recommended for mass-consumption at this time. ''' import os, sys from processes import waitPid from redirection import redirectFds def pipeFnsRecursive(fns, argses = None, kwargses = None, pipe_out = True, pipe_err = False): if argses is None: argses = len(fns) * ( ( ), ) if kwargses is None: kwargses = len(fns) * ( { }, ) if len(fns) > 1: next_pipe_w = pipeFnsRecursive(fns[1:], argses[1:], kwargses[1:]) pipe_r, pipe_w = os.pipe() pid = os.fork() if not pid: #child os.close(pipe_w) if len(fns) > 1: stdout = None stderr = None if pipe_out: stdout = next_pipe_w if pipe_err: stderr = next_pipe_w execFnFds(fns[0], argses[0], kwargses[0], stdin = pipe_r, stdout = stdout, stderr = stderr) else: execFnFds(fns[0], argses[0], kwargses[0], stdin = pipe_r) else: #parent os.close(pipe_r) return pipe_w def pipeFns(fns, argses = None, kwargses = None, pipe_out = True, pipe_err = False, stdin = None, stdout = None, stderr = None): pid = os.fork() if not pid: redirectFds(stdin = stdin, stdout = stdout, stderr = stderr) if argses is None: argses = len(fns) * ( ( ), ) if kwargses is None: kwargses = len(fns) * ( { }, ) if len(fns) > 1: pipe_w = pipeFnsRecursive( fns[1:], argses[1:], kwargses[1:], pipe_out = pipe_out, pipe_err = pipe_err) else: pipe_w = None stdout = None stderr = None if pipe_out: stdout = pipe_w if pipe_err: stderr = pipe_w execFnFds( fns[0], argses[0], kwargses[0], stdout = stdout, stderr = stderr) return pid def pipeCmds(cmds, argses, pipe_out = True, pipe_err = False, stdin = None, stdout = None, stderr = None): def execWrapper(*args, **kwargs): print 'execing %s' % args[0] print >>sys.stderr, 'execing %s' % args[0] os.execvp(*args, **kwargs) fns = len(cmds) * ( execWrapper, ) fn_argses = zip(cmds, argses) return pipeFns(fns, fn_argses, pipe_out = pipe_out, pipe_err = pipe_err, stdin = stdin, stdout = stdout, stderr = stderr) def execFnFds(fn, args = None, kwargs = None, stdin = None, stdout = None, stderr = None): redirectFds(stdin = stdin, stdout = stdout, stderr = stderr) if args is None: args = [ ] if kwargs is None: kwargs = { } status = fn(*args, **kwargs) try: os._exit(status) except TypeError: os._exit(-1) if __name__ == '__main__': def test01(): print pipeCmds(['echo'], [['echo', 'test01']]) print pipeCmds(['echo', 'grep'], [['echo', 'test01'], ['grep', 'hi']]) print pipeCmds( ['echo', 'grep'], [['echo', 'test01'], ['grep', 'test01']]) def test02(): def f(x): print x pid = os.fork() if not pid: pipeFns([f, os.execvp], [['test02'], ['grep', ['grep', 'test02']]]) print pid test01() test02() test01() sclapp-0.5.3/sclapp/processes.py0000644000175000017500000002234511041121232014676 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. '''Classes for easily managing background processes.''' import os, signal, traceback from datetime import datetime from sclapp import error_output from sclapp.redirection import redirectFds from sclapp.util import safe_encode def waitPid(pid, block = True): error_output._printSclappDebug(u'waitPid') import errno if pid is None: status = None signal = None else: try: if block: (os_pid, os_status) = os.waitpid(pid, 0) else: (os_pid, os_status) = os.waitpid(pid, os.WNOHANG) except OSError, e: if e.errno == errno.ESRCH: error_output._printSclappDebug(u'waitpid (%i): ESRCH' % pid) status = None signal = None if e.errno == errno.ECHILD: error_output._printSclappDebug(u'waitpid (%i): ECHILD' % pid) status = None signal = None else: raise e else: if os_pid: if os.WIFSIGNALED(os_status): status = None signal = os.WTERMSIG(os_status) error_output._printSclappDebug(u'signaled: %s' % signal) elif os.WIFEXITED(os_status): status = os.WEXITSTATUS(os_status) error_output._printSclappDebug(u'exited: %s' % status) signal = None else: error_output._printSclappDebug(u'not signaled, not exited') status = None signal = None else: error_output._printSclappDebug(u'os_pid is %s' % os_pid) status = None signal = None return status, signal def _runFunction(fn, args, kwargs, stdin = None, stdout = None, stderr = None): pid = os.fork() if pid: return pid try: os.setsid() redirectFds(stdin, stdout, stderr) os._exit(fn(*args, **kwargs)) except Exception: traceback.print_exc() os._exit(254) def _runCommand(command, args, stdin = None, stdout = None, stderr = None): pid = os.fork() if pid: return pid try: os.setsid() redirectFds(stdin, stdout, stderr) os.execvp(command, args) except Exception: traceback.print_exc() os._exit(254) def average(l): return (sum(l) / float(len(l))) class Event(object): name = None timestamp = None def __init__(self, name, *args, **kwargs): self.name = name self.timestamp = datetime.now() super(Event, self).__init__(*args, **kwargs) class EventHistory(list): maxlength = None def __init__(self, *args, **kwargs): try: self.maxlength = kwargs['maxlength'] del kwargs['maxlength'] except KeyError: self.maxlength = 100 def averagePeriod(self, name, samplesize): if samplesize < 2: raise ValueError, u'samplesize must be greater than 1' samples = [ event for event in self if (event.name == name) ][-samplesize:] if len(samples) < 2: return None deltas = [ ] for i in range(1, len(samples)): deltas.append(samples[i].timestamp - samples[i-1].timestamp) deltas_seconds = [ d.seconds + (d.microseconds / 1000000.0) for d in deltas ] return average(deltas_seconds) def averageFrequency(self, name, samplesize): avg_period = self.averagePeriod(name, samplesize) if avg_period is None: return 0.0 return (1.0 / avg_period) def __setitem__(self, index, value): raise TypeError def __delitem__(self, index): raise TypeError def pop(self, value): raise TypeError def append(self, value): super(EventHistory, self).append(value) if len(self) > self.maxlength: super(EventHistory, self).__delitem__(0) class _BackgroundProcess(object): _pid = None _status = None _signal = None _stdin = None _stdout = None _stderr = None _history = None def __init__(self, stdin = None, stdout = None, stderr = None): self._stdin = stdin self._stdout = stdout self._stderr = stderr self._history = EventHistory() def run(self): '''Runs the process.''' self._history.append(Event('run')) def getRunFrequency(self, samplesize = 10): '''Returns runs/second over the last samplesize runs.''' return self._history.averageFrequency('run', samplesize) def getRunCount(self): return len([ ev for ev in self._history if ev.name == 'run' ]) def stop(self): '''Stops (pauses) the process (usually by sending SIGSTOP).''' return self.kill(signal.SIGSTOP) def cont(self): '''Re-starts a stopped (paused) process (usually by sending SIGCONT).''' return self.kill(signal.SIGCONT) def kill(self, signum = signal.SIGINT): '''Kills the process, or sends an arbitrary signal (specified by the signum argument) to the process. Returns False if the process doesn't appear to be running in the first place, True otherwise. ''' if self._pid is None: return False os.kill(self._pid, signum) return True def reap(self): '''Tries to reap the process. Returns True if successful, False otherwise. ''' return self.wait(False) def wait(self, block = True): '''Blocks until the process has terminated. If the block argument is False, process will simply be reaped (if it has terminated), and the function will return. Returns True if process is reaped by this call, False otherwise. ''' error_output._printSclappDebug(u'wait') if self._pid is None: error_output._printSclappDebug(u'_pid is None') return False self._status, self._signal = waitPid(self._pid, block) if (self._status is not None) or (self._signal is not None): self._pid = None return True return False def isRunning(self): '''Returns True if process is running, False otherwise.''' self.reap() return (self._pid is not None) def isStopped(self): '''Returns True if process is stopped, False otherwise.''' self.reap() return self._stopped def getExitStatus(self): '''Returns the exit status of the process.''' self.reap() return self._status def getExitSignal(self): '''Returns the signal that caused the process to terminate.''' self.reap() return self._signal def getPid(self): '''Returns the PID (process ID) of the process.''' return self._pid class BackgroundFunction(_BackgroundProcess): '''Runs a function in a forked background process. The status of the forked process can be monitored by the caller during function execution. ''' _function = None _args = None _kwargs = None def __init__( self, function, args = (), kwargs = {}, stdin = None, stdout = None, stderr = None, ): self._function = function self._args = args self._kwargs = dict(kwargs) super(BackgroundFunction, self).__init__(stdin, stdout, stderr) def run(self): self._pid = _runFunction(self._function, self._args, self._kwargs, self._stdin, self._stdout, self._stderr) return super(BackgroundFunction, self).run() def __str__(self): return '%s: %s (pid=%s)' % ( self.__class__.__name__, self._function.__name__, self._pid ) class BackgroundCommand(_BackgroundProcess): '''Runs an external command in a forked process. The status of the forked process can be monitored by the caller. ''' _command = None _args = None def __init__(self, command, args, stdin = None, stdout = None, stderr = None): '''command will be run with arguments args. If stdin, stdout, or stderr are specified, the standard I/O file descriptors for the sub-process will be redirected. stdin, stdout, and stderr should be specified as for redirectFds(), or left unspecified if no I/O redirection is desired. ''' self._command = command self._args = args super(BackgroundCommand, self).__init__(stdin, stdout, stderr) def run(self): '''Runs the command.''' self._pid = _runCommand(self._command, self._args, self._stdin, self._stdout, self._stderr) return super(BackgroundCommand, self).run() def __str__(self): return safe_encode(unicode(self)) def __unicode__(self): return u'%s: %s (pid=%s)' % ( self.__class__.__name__, self._command, self._pid ) sclapp-0.5.3/sclapp/protected_output.py0000644000175000017500000000620211041121232016273 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import sys, os, signal, errno from sclapp.exceptions import CriticalError import sclapp.signals from sclapp.util import DelegateWrapper _protected_output_enabled = False class _FileWrapper(DelegateWrapper): buffer = [ ] def __init__(self, wrapped_obj): self.wrapped_obj = wrapped_obj def write(self, message): pass def flush(self): pass def signalOutput(self): pass def restoreOutput(self): pass def disableOutput(self): self.__class__ = _NullFileWrapper class _NullFileWrapper(_FileWrapper): def disableOutput(self): pass class _SafeFileWrapper(_FileWrapper): def write(self, message): try: self.wrapped_obj.write(message) self.wrapped_obj.flush() except IOError, e: if e.errno != errno.EPIPE: raise if hasattr(signal, 'SIGPIPE') and ( signal.SIGPIPE in sclapp.signals.getExitSignals()): # We want to disable output for stdout, stderr if they are # triggering IOError EPIPE exceptions, but we need to wait # until a SIGPIPE signal is caught on POSIX systems. if signal.SIGPIPE in sclapp.signals.getCaughtSignals(): self.disableOutput() # The IOError we just caught may have interrupted handling of # an ExitSignalError. If so, we should re-raise it to ensure # proper handling of that exception. if sclapp.signals._exit_signal_exception is not None: raise sclapp.signals._exit_signal_exception else: self.disableOutput() raise CriticalError, (e.errno, unicode(e)) def signalOutput(self): self.__class__ = _SignalFileWrapper class _SignalFileWrapper(_FileWrapper): def write(self, message): self.buffer.append(message) self.flush() def restoreOutput(self): self.__class__ = _SafeFileWrapper while len(self.buffer) > 0: self.write(self.buffer.pop()) def flush(self): if sclapp.signals._handling_signal is None: self.restoreOutput() def _protectOutput(): global _protected_output_enabled sys.stdout = _SafeFileWrapper(sys.stdout) sys.stderr = _SafeFileWrapper(sys.stderr) _protected_output_enabled = True def protectOutput(): if not _protected_output_enabled: return _protectOutput() def _unprotectOutput(): global _protected_output_enabled sys.stdout = sys.stdout.wrapped_obj sys.stderr = sys.stderr.wrapped_obj _protected_output_enabled = False def unprotectOutput(): if _protected_output_enabled: _unprotectOutput() def _signalOutput(): sys.stdout.signalOutput() sys.stderr.signalOutput() sclapp-0.5.3/sclapp/redirection.py0000644000175000017500000000323011041121232015167 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import sys, os, types from sclapp import error_output def _dup(n, f, flags = None): os.close(n) try: return os.dup(f.fileno()) except (AttributeError, TypeError): try: return os.dup(f) except TypeError: try: return os.open(f, flags) except TypeError: raise TypeError, ( 'f is neither a usable file-like object, ' 'an integer file descriptor, nor a filename.' ) def redirectFds(stdin = None, stdout = None, stderr = None): '''Redirects the standard I/O file descriptors for the current process. stdin, stdout, and stderr can be either a filename, an fd (integer), or None (to indicate no redirection). ''' pid = os.getpid() if stdin is not None: error_output._printSclappDebug( u'pid %u: redirecting stdin to %s' % (pid, stdin)) _dup(0, stdin, os.O_RDONLY) if stdout is not None: error_output._printSclappDebug( u'pid %u: redirecting stdout to %s' % (pid, stdout)) _dup(1, stdout, os.O_RDWR | os.O_APPEND | os.O_CREAT) if stderr is not None: error_output._printSclappDebug( u'pid %u: redirecting stderr to %s' % (pid, stderr)) _dup(2, stderr, os.O_RDWR | os.O_APPEND | os.O_CREAT) sclapp-0.5.3/sclapp/services.py0000644000175000017500000000430711041121232014511 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import sys, os, signal from sclapp.daemonize import daemonize from sclapp.exceptions import * from sclapp.error_output import * class InvalidPidFileError(Error): pass def readPidFile(pid_file_name): try: f = open(pid_file_name, 'r') except IOError: return None try: contents = f.read().strip() assert(len(contents.split(u'\n')) < 2) f.close() pid = int(contents) except (AssertionError, ValueError): raise InvalidPidFileError return pid def writePidFile(pid_file_name, pid = None): if pid is None: pid = os.getpid() pid_file = open(pid_file_name, 'w') pid_file.write(unicode(os.getpid())) pid_file.close() def removePidFile(pid_file_name): try: os.unlink(pid_file_name) except OSError, e: if e.errno != 2: raise def startService(pid_file_name, fn, args = None, kwargs = None): if args is None: args = [ ] if kwargs is None: kwargs = { } # TODO: verify that this pid actually belongs to an instance of the service. pid = readPidFile(pid_file_name) if pid is not None: raise CriticalError, (1, u'already running? process ID %u' % pid) pid = os.fork() if not pid: # child process daemonize() try: writePidFile(pid_file_name, pid = pid) fn(*args, **kwargs) finally: try: removePidFile(pid_file_name) except Exception, e: printCritical(unicode(e)) os._exit(0) def stopService(pid_file_name): try: pid_file = open(pid_file_name, 'r') except IOError: return pid = int(pid_file.read().strip()) pid_file.close() # FIXME: wait for process to die, try to kill again if necessary try: os.kill(pid, signal.SIGINT) except OSError: pass sclapp-0.5.3/sclapp/shell.py0000644000175000017500000004241011041121232013772 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys, re, random import string, locale, struct, fcntl, termios, signal from pexpect import ( spawn, TIMEOUT, EOF, ) from sclapp import shinterp from sclapp.exceptions import Error from sclapp.util import DelegateWrapper, CallbackMapping from sclapp.error_output import printError RAND_PROMPT_SIZE = 32 MAX_DELAY = 1200 def randomAlphaNumeric(length): return ''.join(random.sample(string.letters + string.digits, length)) class CommandFailed(Error): pass class CommandSignalled(Error): pass class CommandFinished(Error): pass class ExitInteract(Error): pass def lengthOfLongestLine(s): length = 0 for line in s.split('\n'): len_line = len(line) if len_line > length: length = len_line return length class Shell(spawn): shell = None _prompt = None failure_exceptions = None signal_exceptions = None trace = None interpolate = staticmethod(shinterp.interpolate) env = None timeout_cache = None dir_cache = None shell_encoding = 'utf-8' always_group_cmd = None interactive_ps1 = '$ ' interactive_ps2 = '> ' interactive_ps3 = '' interactive_ps4 = '+ ' interactive_exit_string = 'DEBSYS_INTERACTIVE_SHELL_EXITING' def __init__(self, shell = '/bin/sh', prompt = None, failure_exceptions = True, signal_exceptions = True, trace = False, timeout = 30, shell_encoding = None, always_group_cmd = True, delaybeforesend = None, **kwargs ): if prompt is None: prompt = randomAlphaNumeric(RAND_PROMPT_SIZE) self.failure_exceptions = failure_exceptions self.signal_exceptions = signal_exceptions self.trace = trace self.shell = shell super(Shell, self).__init__(self.shell, **kwargs) if delaybeforesend is not None: self.delaybeforesend = delaybeforesend self.timeout_cache = [ ] self._pushTimeout(timeout) self._pre_prompt() self.setecho(False) self.prompt = prompt self.env = CallbackMapping( self._envKeys, self._envGet, self._envSet, self._envDel) self.dir_cache = [ ] if shell_encoding is None: self.shell_encoding = self.detectShellEncoding() else: self.shell_encoding = shell_encoding self.env['PS2'] = '' self.env['PS3'] = '' self.env['PS4'] = '' if 'TERM' in os.environ: self.env['TERM'] = os.environ['TERM'] self.always_group_cmd = always_group_cmd self.setupShell() def __del__(self): # Make an effort to clean up here just in case our caller neglected to # do so. If we don't, bad things (like not closing the pty) can # happen. try: self.exit() except Exception, e: printError(e) def setupShell(self): self.execute('set +m') def detectShellEncoding(self): self.sendline('echo $LANG') lang = self._getLastOutput().strip() try: language, encoding = lang.split('.') except ValueError: raise ValueError, 'Invalid value for LANG: "%s"' % lang try: ''.decode(encoding) except LookupError: raise ValueError( 'invalid encoding obtained by splitting language string "%s"' % lang ) return encoding def _pre_prompt(self): pass def _pushTimeout(self, timeout): self.timeout_cache.append(self.timeout) self.timeout = timeout def _popTimeout(self): old_timeout = self.timeout try: new_timeout = self.timeout_cache.pop() except IndexError: pass else: self.timeout = new_timeout return old_timeout def _interpolate_command(self, cmd, *values, **kwargs): return self.interpolate(cmd.strip(), *values) def _adjust_command(self, interpolated_cmd, *values, **kwargs): force = kwargs.get('force', False) adjusted_cmd = interpolated_cmd if self.always_group_cmd: adjusted_cmd = '{ %s\n}' % adjusted_cmd if isinstance(adjusted_cmd, unicode): adjusted_cmd = self.encode(adjusted_cmd) if (not force) and (lengthOfLongestLine(adjusted_cmd) > 4095): raise ValueError( 'Command contains a line longer than 4095 characters, ' 'and may not execute reliably. ' 'Use force = True if you really want this.' ) return adjusted_cmd def _execute(self, cmd, *values, **kwargs): interpolated_cmd = self._interpolate_command( cmd, *values, **kwargs) adjusted_cmd = self._adjust_command( interpolated_cmd, *values, **kwargs) if self.trace: trace_cmd = interpolated_cmd if not isinstance(trace_cmd, unicode): trace_cmd = self.decode(trace_cmd) print trace_cmd bytes_sent = 0 try: bytes_sent = self.sendline(adjusted_cmd) finally: if bytes_sent > len(adjusted_cmd): self._last_cmd = interpolated_cmd def _getLastOutput(self): self._pushTimeout(MAX_DELAY) try: self.expect(self.prompt_sep) return self.before.replace('\r\n', '\n').replace('\r', '\n') finally: self._popTimeout() def _getLastStatus(self): try: self._execute('echo $??') output = '' while not output.strip(): output = self._getLastOutput() status = int(output) return status except Exception: self.interrupt() self.expect(self.prompt_sep) raise def _checkLastStatus(self, output): last_cmd = self._last_cmd status = self._getLastStatus() if self.failure_exceptions and (status > 0) and (status <= 128): raise CommandFailed(last_cmd, status, output) if self.signal_exceptions and (status > 128): raise CommandSignalled(last_cmd, status, output) return status def interrupt(self): # Send Control-C self.send('') def execute(self, cmd, *values, **kwargs): try: self._execute(cmd, *values, **kwargs) output = self._getLastOutput() except Exception: self.interrupt() self.expect(self.prompt_sep) raise output = self.decode(output) status = self._checkLastStatus(output) return status, output def encode(self, s): return s.encode(self.shell_encoding) def decode(self, s): return s.decode(self.shell_encoding) def _follow_chars(self, buffer): chars = [ ] while buffer and (not self.prompt.startswith(buffer)): chars.append(buffer[0]) buffer = buffer[1:] if re.match(self.prompt_sep, buffer) is not None: raise CommandFinished return chars def _follow_bytes_iter(self, follow_input): if follow_input: for b in self.encode(self._last_cmd): yield b yield self.encode('\n') while True: yield self.read(1) def _follow(self, cmd, *values, **kwargs): buffer = '' follow_input = kwargs.get('follow_input', False) try: del kwargs['follow_input'] except KeyError: pass try: self._execute(cmd, *values, **kwargs) self._pushTimeout(MAX_DELAY) try: for b in self._follow_bytes_iter(follow_input): buffer = '%s%s' % (buffer, b) try: chars = self._follow_chars(buffer) except CommandFinished: break for ch in chars: if ch != '\r': try: yield ch except Exception: buffer = buffer[1:] raise buffer = buffer[1:] # BEGIN Python 2.3 compat #finally: except Exception: self._popTimeout() raise else: self._popTimeout() # END Python2.3 compat except Exception: self.interrupt() self.expect(self.prompt_sep) raise def _follow_decode(self, *args, **kwargs): buffer = '' for ch in self._follow(*args, **kwargs): buffer = '%s%s' % (buffer, ch) try: buffer_decoded = self.decode(buffer) except UnicodeDecodeError: continue else: for ch_decoded in buffer_decoded: yield ch_decoded buffer = '' continue def follow(self, cmd, *values, **kwargs): for ch in self._follow_decode(cmd, *values, **kwargs): yield ch self._checkLastStatus(None) def followCallback(self, cmd, *values, **kwargs): try: callback = kwargs['callback'] del kwargs['callback'] except KeyError: raise TypeError, 'missing required argument "callback"' for ch in self._follow_decode(cmd, *values, **kwargs): callback(ch) return self._checkLastStatus(None) def followWrite(self, cmd, *values, **kwargs): outfile = kwargs.get('outfile', sys.stdout) try: del kwargs['outfile'] except KeyError: pass def f(ch): outfile.write(ch) kwargs['callback'] = f return self.followCallback(cmd, *values, **kwargs) def followWriteReturn(self, cmd, *values, **kwargs): outfile = kwargs.get('outfile', sys.stdout) try: del kwargs['outfile'] except KeyError: pass buffer = [ ] def f(ch): outfile.write(ch) buffer.append(ch) kwargs['callback'] = f try: status = self.followCallback(cmd, *values, **kwargs) except CommandFailed, e: output = ''.join(buffer) e.args = (e.args[0], e.args[1], output) raise e output = ''.join(buffer) return status, output def exit(self): self.sendeof() self.expect(EOF) self.close() def _getPrompt(self): return self._prompt def _setPrompt(self, value): self._prompt = value try: self._execute('PS1=?; export PS1', value) # FIXME: It'd be nice to do some sanity checking on the returned # output here: self._getLastOutput() except Exception: self.interrupt() # We can't expect anything reasonable in terms of prompt at this # point. The shell is essentially unusable. We simply raise the # exception and hope the caller can sort it out. #self.expect(self.prompt_sep) raise prompt = property(_getPrompt, _setPrompt) def _getPromptSep(self): return re.escape(self.prompt) prompt_sep = property(_getPromptSep) def _envValues_iter(self): try: self._execute('env') output = '%s%s%s' % ( self._getLastOutput(), self.prompt, self._getLastOutput(), ) except Exception: self.interrupt() self.expect(self.prompt_sep) raise output = self.decode(output) for line in output.strip().split('\n'): i = line.index('=') n, v = line[:i], line[i+1:] yield n, v def _envKeys(self): return list([n_v[0] for n_v in self._envValues_iter()]) def _envGet(self, name): for n, v in self._envValues_iter(): if n == name: return v raise KeyError def _envSet(self, name, value): self.execute('%s=?; export %s' % (name, name), value) def _envDel(self, name): self.execute('unset ?', name) def pwd(self): status, output = self.execute('echo $PWD') return output.strip() def pushd(self, dir): old_dir = self.pwd() self.dir_cache.append(self.pwd()) try: self.cd(dir) except Exception: self.dir_cache.pop() raise return old_dir def popd(self): old_dir = self.pwd() new_dir = self.dir_cache.pop() try: self.cd(new_dir) except Exception: self.dir_cache.push(new_dir) raise return old_dir def cd(self, dir): return self.execute('cd ?', dir) def _interact_output_filter(self, s): if self.interactive_exit_string in s: raise ExitInteract return s def _sigwinch_handler(self, signum, frame): self.fit_window() def fit_window(self): s = struct.pack('HHHH', 0, 0, 0, 0) a = struct.unpack( 'hhhh', fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, s)) self.setwinsize(a[0], a[1]) def interact(self, fitted = False): if fitted: return self._interact_fitted() return self._interact() def interact_with(self, *args, **kwargs): fitted = kwargs.pop('fitted', False) if fitted: return self._interact_fitted(*args, **kwargs) return self._interact(*args, **kwargs) def _interact_fitted(self, *args, **kwargs): old_handler = signal.signal(signal.SIGWINCH, self._sigwinch_handler) try: self.fit_window() return self._interact(*args, **kwargs) finally: signal.signal(signal.SIGWINCH, old_handler) def _interact(self, cmd = None, *values, **kwargs): old_prompt = self.prompt self.execute('sh') self.setupShell() self.prompt = self.interactive_ps1 self.execute('trap "echo \'%s\'" EXIT' % self.interactive_exit_string) # We're in a subshell, so we can do this without causing any parsing # issues or even having to clean up: self.sendline('PS2="%s"; export PS2' % self.interactive_ps2) self.expect(self.prompt_sep) self.sendline('PS3="%s"; export PS3' % self.interactive_ps3) self.expect(self.prompt_sep) self.sendline('PS4="%s"; export PS4' % self.interactive_ps4) self.expect(self.prompt_sep) if cmd is not None: cmd = u'%s; exit $??' % cmd interpolated_cmd = self._interpolate_command( cmd, *values, **kwargs) adjusted_cmd = self._adjust_command( interpolated_cmd, *values, **kwargs) self.sendline(adjusted_cmd) self._last_cmd = interpolated_cmd else: # Trigger first prompt to be displayed: self.sendline() self.setecho(True) try: try: super(Shell, self).interact( escape_character = chr(0), output_filter = self._interact_output_filter, ) except ExitInteract: pass finally: print '' self.setecho(False) self._prompt = old_prompt try: self.expect(self.prompt_sep, timeout = 2) except TIMEOUT: self.sendline() self.expect(self.prompt_sep, timeout = 2) return self._checkLastStatus(None) SORRY_TRY_AGAIN = 'Sorry, try again.' PASSWORD_PROMPT = 'Password:' ROOT_PROMPT = '#' class SudoShell(Shell): password = None interactive_ps1 = '# ' def __init__(self, shell = '/bin/sh', password = None, **kwargs): self.password = password # FIXME: This will blow up if either PASSWORD_PROMPT or shell contains # double quote characters. super(SudoShell, self).__init__( shell = 'sudo -p "%s" "%s"' % (PASSWORD_PROMPT, shell), **kwargs) def _pre_prompt(self): root_prompt_regex = re.compile(re.escape(ROOT_PROMPT)) password_prompt_regex = re.compile(re.escape(PASSWORD_PROMPT)) sorry_try_again_regex = re.compile(re.escape(SORRY_TRY_AGAIN)) self.expect([password_prompt_regex, root_prompt_regex]) if password_prompt_regex.match(self.after): if self.password is None: raise ValueError, 'Password required' self.sendline(self.password) self.expect([root_prompt_regex, sorry_try_again_regex, EOF]) if sorry_try_again_regex.match(self.after) or (self.after == EOF): raise ValueError, 'Password incorrect' if not root_prompt_regex.match(self.after): raise AssertionError, u'Expected string matching "%s", got "%s"' % ( ROOT_PROMPT, self.after) del self.password sclapp-0.5.3/sclapp/shinterp.py0000644000175000017500000000667311041121232014532 0ustar fabfab# coding: utf-8 # Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import sys, itertools def interpolate(s, *values, **kwargs): # FIXME: Should this be a unicode doc-string? r''' >>> interpolate('echo ?', 'foo') "echo 'foo'" >>> interpolate('echo ? ?', 'foo') Traceback (most recent call last): ... ValueError: Too few values to interpolate >>> interpolate('echo ? ?', 'foo', 'bar', 'baz') Traceback (most recent call last): ... ValueError: Too many values to interpolate >>> interpolate('echo ?', 'he\'llo') "echo 'he'\\''llo'" >>> interpolate('echo ? ?', 'hi', 'there') "echo 'hi' 'there'" >>> interpolate("echo 'foo'") "echo 'foo'" >>> interpolate(u'echo ?', 'foo') u"echo 'foo'" >>> interpolate('echo ?', u'foo') u"echo 'foo'" >>> interpolate('echo ?', u'•').encode('utf-8') "echo '\xc3\xa2\xc2\x80\xc2\xa2'" ''' divisions = find_divisions(s) values = [ coerceToStringType(v).replace('\'', '\'\\\'\'') for v in values ] len_divisions, len_values = len(divisions), len(values) if len_divisions < (len_values + 1): raise ValueError, 'Too many values to interpolate' elif len_divisions > (len_values + 1): raise ValueError, 'Too few values to interpolate' total = zip_together(divisions, values) total.append(divisions[-1]) return '\''.join(total) def coerceToStringType(x): # If x is (like) a byte string, leave it as is. try: if str(x) == x: return x except UnicodeEncodeError: pass # If x is (like) a unicode string, leave it as is. try: if unicode(x) == x: return x except UnicodeDecodeError: pass # If we can make it a byte string, return that. try: return str(x) except UnicodeEncodeError: pass # If we can make it a unicode string, return that. try: return unicode(x) except UnicodeDecodeError: pass # Give up and return x as is, hoping that things work out. Best of luck. return x def izip_together(i1, i2): for t in itertools.izip(i1, i2): for x in t: yield x def zip_together(l1, l2): return list(izip_together(l1, l2)) def find_divisions(s): ''' >>> find_divisions('??one?two?three??') ['?one', 'two', 'three?'] >>> find_divisions('??one??two??three??') ['?one?two?three?'] >>> find_divisions('?one??two??three?') ['', 'one?two?three', ''] ''' # If we are given unicode, return unicode. # If we are given byte-strings, return byte-strings. s_type = type(s) parts = s.split(s_type('?')) new_parts = [ parts[0] ] iterable = iter(parts[1:]) for part in iterable: if part != '': new_parts.append(part) else: first_half = new_parts.pop() try: new_parts.append( s_type('%s?%s') % (first_half, iterable.next())) except StopIteration: new_parts.append(first_half) new_parts.append(s_type('')) return new_parts if __name__ == '__main__': import doctest doctest.testmod() sclapp-0.5.3/sclapp/signals.py0000644000175000017500000002553711041121232014336 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, signal from copy import copy try: set except NameError: from sets import Set as set try: reversed except NameError: from sclapp.legacy_support import reversed from sclapp.exceptions import ExitSignalError, SignalError from sclapp.error_output import printWarning, printDebug def _getAllSignals_iter(): for name in dir(signal): if name.startswith('SIG') \ and (name != 'SIG_IGN') \ and (name != 'SIG_DFL'): signum = getattr(signal, name) yield signum def _getRealSignals_iter(names): for name in names: if hasattr(signal, name): signum = getattr(signal, name) yield signum ALL_SIGNALS = tuple(list(_getAllSignals_iter())) _STD_EXIT_SIGNALS = ( ) _STD_NOTIFY_SIGNALS = ( ) _STD_DEFAULT_SIGNALS = ( 'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGILL', 'SIGABRT', 'SIGFPE', 'SIGPIPE', 'SIGTERM', 'SIGBUS', 'SIGPROF', 'SIGSYS', 'SIGTRAP', 'SIGXCPU', 'SIGXFSZ', ) _STD_IGNORE_SIGNALS = ( 'SIGUSR1', 'SIGUSR2', 'SIGALRM', ) STD_EXIT_SIGNALS = tuple(list(_getRealSignals_iter(_STD_EXIT_SIGNALS))) STD_NOTIFY_SIGNALS = tuple(list(_getRealSignals_iter(_STD_NOTIFY_SIGNALS))) STD_DEFAULT_SIGNALS = tuple(list(_getRealSignals_iter(_STD_DEFAULT_SIGNALS))) STD_IGNORE_SIGNALS = tuple(list(_getRealSignals_iter(_STD_IGNORE_SIGNALS))) _announce_signals = True _announce_signals_stack = [ ] # store raised SignalError here _exit_signal_exception = None # signal current being handled _handling_signal = None # for keeping track of signal handlers; old handlers are pushed here, and # popped back into place later _signal_mapping_stack = [ ] def _addToMapping(d, signums, handler_factory): if signums is not None: for signum in signums: assert (signum not in d), 'Signal %u mapped more than once' % signum d[signum] = handler_factory(signum) def _pushSignalMapping( exit_signals = None, notify_signals = None, default_signals = None, ignore_signals = None, announce_signals = True, custom_handlers = None ): global _signal_mapping_stack d = { } _addToMapping(d, custom_handlers, lambda signum: custom_handlers[signum]) _addToMapping(d, exit_signals, lambda signum: _exitSignalHandler) _addToMapping(d, notify_signals, lambda signum: _notifySignalHandler) _addToMapping(d, default_signals, lambda signum: signal.SIG_DFL) _addToMapping(d, ignore_signals, lambda signum: signal.SIG_IGN) for signum in d.keys(): try: signal.signal(signum, d[signum]) except RuntimeError, e: if e[0] == 22: printDebug( 'Setting signal handler for signum %u: Invalid argument' % \ signum) d[signum] = signal.getsignal(signum) else: raise _signal_mapping_stack.append(d) global _announce_signals, _announce_signals_stack _announce_signals_stack.append(_announce_signals) _announce_signals = announce_signals def _popSignalMapping(): global _signal_mapping_stack d = _signal_mapping_stack.pop() for signum in d: _restoreSignalHandler(signum) global _announce_signals, _announce_signals_stack _announce_signals = _announce_signals_stack.pop() return d def _pushCurrentSignalMapping(): global _signal_mapping_stack d = { } for signum in ALL_SIGNALS: d[signum] = signal.getsignal(signum) _signal_mapping_stack.append(d) def _restoreSignalHandler(signum): for mapping in reversed(_signal_mapping_stack): if signum in mapping: signal.signal(signum, mapping[signum]) def _getSignalsByHandler_iter(handler): for signum in ALL_SIGNALS: if signal.getsignal(signum) == handler: yield signum def _getSignalHandlers_iter(): for signum in ALL_SIGNALS: yield signal.getsignal(signum) # stores list of caught signal numbers _caught_signals = set([ ]) def _ignoreExitSignals(): for signum in getExitSignals(): signal.signal(signum, _ignoreExitSignalHandler) def _ignoreExitSignalHandler(signum, frame): handler = signal.signal(signum, signal.SIG_IGN) global _caught_signals, _exit_signal_exception, _handling_signal if _handling_signal is None: _handling_signal = signum else: _handleLameSignal(signum, handler) return _signalPrint('caught signal %i (ignored)' % signum) _caught_signals.add(signum) _handling_signal = None signal.signal(signum, handler) def _exitSignalHandler(signum, frame): signal.signal(signum, signal.SIG_IGN) global _handling_signal, _exit_signal_exception if _exit_signal_exception is not None: _handleLameSignal(signum, _ignoreExitSignalHandler) return _exit_signal_exception = ExitSignalError(signum) _handling_signal = signum _signalPrint('caught exit signal %i' % signum) _caught_signals.add(signum) _handling_signal = None _ignoreExitSignals() raise _exit_signal_exception def _notifySignalHandler(signum, frame): handler = signal.signal(signum, signal.SIG_IGN) global _exit_signal_exception, _handling_signal if _exit_signal_exception is not None: _handleLameSignal(signum, handler) return if _handling_signal is None: _handling_signal = signum elif _handling_signal in getNotifySignals(): _handleLameSignal(signum, handler) return _signalPrint('caught signal %i' % signum) _caught_signals.add(signum) _handling_signal = None signal.signal(signum, handler) raise SignalError, signum def _handleLameSignal(signum, handler): _signalPrint('caught signal %i (ignored)' % signum) _caught_signals.add(signum) signal.signal(signum, handler) def _signalPrint(message): _signal_print_fn = _getSignalPrintFn() if _signal_print_fn is not None: from sclapp.protected_output import _protected_output_enabled, \ _signalOutput if _protected_output_enabled: _signalOutput() _signal_print_fn(message) def _getSignalPrintFn(): global _announce_signals if _announce_signals: return printWarning else: return None def _enableSignals(signums): for signum in signums: if signum in getExitSignals(): if _exit_signal_exception is not None: signal.signal(signum, _ignoreExitSignalHandler) else: signal.signal(signum, _exitSignalHandler) elif signum in getNotifySignals(): signal.signal(signum, _notifySignalHandler) elif signum in getDefaultSignals(): signal.signal(signum, signal.SIG_DFL) elif signum in getIgnoreSignals(): signal.signal(signum, signal.SIG_IGN) def getExitSignals(): '''getExitSignals() -> list Returns a list containing all signal numbers that are currently being handled by sclapp as exit signals. ''' return set( list(_getSignalsByHandler_iter(_exitSignalHandler)) + list(_getSignalsByHandler_iter(_ignoreExitSignalHandler)) ) def getNotifySignals(): '''getNotifySignals() -> list Returns a list containing all signal numbers that are currently being handled by sclapp as notify signals. ''' return set(list(_getSignalsByHandler_iter(_notifySignalHandler))) def getDefaultSignals(): '''getDefaultSignals() -> list Returns a list containing all signal numbers that are currently being handled by sclapp as default signals (SIG_DFL). ''' return set(list(_getSignalsByHandler_iter(signal.SIG_DFL))) def getIgnoreSignals(): '''getIgnoreSignals() -> list Returns a list containing all signal numbers that are currently being handled by sclapp as ignore signals (SIG_IGN). ''' return set(list(_getSignalsByHandler_iter(signal.SIG_IGN))) def getCaughtSignals(): '''getCaughtSignals() -> list Returns a list containing all signal numbers that have been caught by sclapp signal handlers. ''' return copy(_caught_signals) def _parseSignal(sig): if str(sig) == sig: if hasattr(signal, sig): return getattr(signal, sig) return None return sig def _parseSignals(sigs): return [signum for signum in [_parseSignal(sig) for sig in sigs] if signum] def enableSignalHandling(exit_signals = None, notify_signals = None, default_signals = None, ignore_signals = None, announce_signals = True, custom_handlers = None): '''enableSignalHandling() -> None Enables sclapp signal handling. ''' # Note that we use an empty list if a group is not specified: exit_signals = _parseSignals(exit_signals or [ ]) notify_signals = _parseSignals(notify_signals or [ ]) default_signals = _parseSignals(default_signals or [ ]) ignore_signals = _parseSignals(ignore_signals or [ ]) if os.name == 'posix': global _signal_mapping_stack if len(_signal_mapping_stack) < 1: _pushCurrentSignalMapping() specified_signals = [] if custom_handlers: specified_signals = specified_signals + custom_handlers.keys() specified_signals = specified_signals + exit_signals specified_signals = specified_signals + notify_signals specified_signals = specified_signals + default_signals specified_signals = specified_signals + ignore_signals for signum in STD_EXIT_SIGNALS: if signum not in specified_signals: exit_signals.append(signum) for signum in STD_NOTIFY_SIGNALS: if signum not in specified_signals: notify_signals.append(signum) for signum in STD_DEFAULT_SIGNALS: if signum not in specified_signals: default_signals.append(signum) for signum in STD_IGNORE_SIGNALS: if signum not in specified_signals: ignore_signals.append(signum) _pushSignalMapping( exit_signals = exit_signals, notify_signals = notify_signals, default_signals = default_signals, ignore_signals = ignore_signals, announce_signals = announce_signals, custom_handlers = custom_handlers ) def disableSignalHandling(): '''disableSignalHandling() -> None Disables sclapp signal handling. ''' if os.name == 'posix': _popSignalMapping() def signalHandlingEnabled(): handlers = list(_getSignalHandlers_iter()) if _exitSignalHandler in handlers: return True if _notifySignalHandler in handlers: return True if _ignoreExitSignalHandler in handlers: return True return False sclapp-0.5.3/sclapp/stdio_encoding.py0000644000175000017500000000513011041121232015651 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import sys, codecs, locale def enableStdioEncoding(decode_stdin = True, encode_stdout = True, encode_stderr = True): '''Replace sys.stdin, sys.stdout, sys.stderr with stream readers & writers that encode/decode to/from the preferred character encoding. ''' encoding = locale.getpreferredencoding() encode, decode, streamreader, streamwriter = codecs.lookup(encoding) streamreader.errors = 'replace' streamwriter.errors = 'replace' if decode_stdin: sys.stdin = streamreader(sys.stdin) sys.stdin.encoding = encoding if encode_stdout: sys.stdout = streamwriter(sys.stdout) sys.stdout.encoding = encoding if encode_stderr: sys.stderr = streamwriter(sys.stderr) sys.stderr.encoding = encoding def disableStdioEncoding(): '''Restore sys.stdin, sys.stdout, sys.stderr to their original objects.''' restoreStdin() restoreStdout() restoreStderr() def restoreStdin(): '''Restore sys.stdin to the original object that existed when _setStdinEncoding was called. Return the replacement so that it can be reinstated afterwards. ''' previous_stdin = sys.stdin if hasattr(sys.stdin, 'stream'): sys.stdin = sys.stdin.stream return previous_stdin def restoreStdout(): '''Restore sys.stdout to the original object that existed when _setStdoutEncoding was called. Return the replacement so that it can be reinstated afterwards. ''' previous_stdout = sys.stdout if hasattr(sys.stdout, 'stream'): sys.stdout = sys.stdout.stream return previous_stdout def restoreStderr(): '''Restore sys.stderr to the original object that existed when _setStderrEncoding was called. Return the replacement so that it can be reinstated afterwards. ''' previous_stderr = sys.stderr if hasattr(sys.stderr, 'stream'): sys.stderr = sys.stderr.stream return previous_stderr def raw_input(q = ''): '''Decoded input-friendly replacement for the built-in raw_input function''' sys.stdout.write(q) sys.stdout.flush() result = '' ch = sys.stdin.read(1) while ch != '\n': result = '%s%s' % (result, ch) ch = sys.stdin.read(1) return result def input(q = ''): return eval(raw_input(q)) sclapp-0.5.3/sclapp/termcontrol.py0000644000175000017500000001107411041121232015235 0ustar fabfab# Copyright (c) 2006 Edward Loper. # Copyright (c) 2007 Forest Bond. # While this file is distributed with the sclapp software package, it is made # available under the terms of a different license. The source code contained # in this module is a derivative of a recipe downloaded from the Active State # Python Cookbook, the contents of which is published under the Python License. # The original copyright has been maintained in this text; any additional # copyrights are also included. # The full text of the Python License can be obtained here: # http://www.python.org/license # The original recipe is available here: # http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/475116 # The relevant meta-information from that recipe follows: # Title: Using terminfo for portable color output & cursor control # Submitter: Edward Loper # Last Updated: 2006/03/27 # Version no: 1.2 # Category: Text # # Description: # # The curses module defines several functions (based on terminfo) that can be # used to perform lightweight cursor control & output formatting (color, bold, # etc). These can be used without invoking curses mode (curses.initwin) or using # any of the more heavy-weight curses functionality. This recipe defines a # TerminalController class, which can make portable output formatting very # simple. Formatting modes that are not supported by the terminal are simply # omitted. '''\ The curses module defines several functions (based on terminfo) that can be used to perform lightweight cursor control & output formatting (color, bold, etc). These can be used without invoking curses mode (curses.initwin) or using any of the more heavy-weight curses functionality. This recipe defines a TerminalController class, which can make portable output formatting very simple. Formatting modes that are not supported by the terminal are simply omitted. ''' caps = { } def empty(*args, **kwargs): return '' def noop(*args, **kwargs): pass def _tigetstr_curses(capname): # String capabilities can include "delays" of the form "$<2>". # For any modern terminal, we should be able to just ignore # these, so strip them out. import curses, re cap = curses.tigetstr(capname) or '' return re.sub(r'\$<\d+>[/*]?', '', cap) def _tigetnum_curses(capname): import curses return curses.tigetnum(capname) def _tigetnum_nocaps(capname): return None def _setupterm_nocaps(): return def _setupterm_curses(): try: import curses except ImportError: return return curses.setupterm() def initializeTermControl(): import sys _STRING_CAPABILITIES = { 'BOL': 'cr', 'UP': 'cuu1', 'DOWN': 'cud1', 'LEFT': 'cub1', 'RIGHT': 'cuf1', 'CLEAR_SCREEN': 'clear', 'CLEAR_EOL': 'el', 'CLEAR_BOL': 'el1', 'CLEAR_EOS': 'ed', 'BOLD': 'bold', 'BLINK': 'blink', 'DIM': 'dim', 'REVERSE': 'rev', 'UNDERLINE': 'smul', 'NORMAL': 'sgr0', 'HIDE_CURSOR': 'cinvis', 'SHOW_CURSOR': 'cnorm', } _COLORS = [ 'BLACK', 'BLUE', 'GREEN', 'CYAN', 'RED', 'MAGENTA', 'YELLOW', 'WHITE'] _ANSICOLORS = [ 'BLACK', 'RED', 'GREEN', 'YELLOW', 'BLUE', 'MAGENTA', 'CYAN', 'WHITE'] __setupterm = _setupterm_curses __tigetstr = _tigetstr_curses __tigetnum = _tigetnum_curses # Curses isn't available on all platforms try: import curses except ImportError: __setupterm, __tigetstr, __tigetnum = noop, empty, noop # If the stream isn't a tty, then assume it has no capabilities. if not sys.stdout.isatty(): __setupterm, __tigetstr, __tigetnum = noop, empty, noop __setupterm() # Look up numeric capabilities. caps['COLS'] = __tigetnum('cols') caps['LINES'] = __tigetnum('lines') # Look up string capabilities. for attrib, capname in _STRING_CAPABILITIES.items(): caps[attrib] = __tigetstr(capname) # Colors set_fg = __tigetstr('setf') if set_fg: for i, color in zip(range(len(_COLORS)), _COLORS): caps[color] = curses.tparm(set_fg, i) set_fg_ansi = __tigetstr('setaf') if set_fg_ansi: for i, color in zip(range(len(_ANSICOLORS)), _ANSICOLORS): caps[color] = curses.tparm(set_fg_ansi, i) set_bg = __tigetstr('setb') if set_bg: for i, color in zip(range(len(_COLORS)), _COLORS): caps['BG_%s' % color] = curses.tparm(set_bg, i) set_bg_ansi = __tigetstr('setab') if set_bg_ansi: for i, color in zip(range(len(_ANSICOLORS)), _ANSICOLORS): caps['BG_%s' % color] = curses.tparm(set_bg_ansi, i) sclapp-0.5.3/sclapp/util.py0000644000175000017500000000466211041121232013647 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import locale from UserDict import DictMixin def safe_encode(s): s = safe_decode(s) return s.encode(locale.getpreferredencoding(), 'replace') def safe_decode(s): if type(s) is unicode: return s return unicode(str(s), locale.getpreferredencoding(), 'replace') def importName(name): '''Imports a Python module referred to by name, and returns that module. Raises an ImportError if the import fails. ''' mod = __import__(name) components = name.split('.') for comp in components[1:]: try: mod = getattr(mod, comp) except AttributeError: raise ImportError, '%s has no attribute %s' % (mod, comp) return mod class DelegateWrapper(object): wrapped_obj = None def __getattr__(self, name): # Note that this only gets called if Python failed to find an attribute # using the normal mechanisms. Thus, only those attributes not defined # by the original class definition are gotten this way. return getattr(self.wrapped_obj, name) def __setattr__(self, name, value): # This prevents setting attributes that aren't present in the class # definition: for cls in [ self.__class__ ] + list(self.__class__.__bases__): if hasattr(cls, name): object.__setattr__(self, name, value) return setattr(self.wrapped_obj, name, value) class CallbackMapping(object, DictMixin): def __init__(self, keys, getitem, setitem = None, delitem = None): if setitem is None: def setitem(name, value): raise NotImplementedError if delitem is None: def delitem(name): raise NotImplementedError self._keys = keys self._getitem__ = getitem self._setitem__ = setitem self._delitem__ = delitem def keys(self): return self._keys() def __getitem__(self, name): return self._getitem__(name) def __setitem__(self, name, value): self._setitem__(name, value) def __delitem__(self, name): self._delitem__(name) sclapp-0.5.3/sclapp.txt0000644000175000017500000007161011041121232013056 0ustar fabfab================================================================================ sclapp ================================================================================ -------------------------------------------------------------------------------- A framework for python command-line applications. -------------------------------------------------------------------------------- :Author: Forest Bond :Copyright: © 2005-2007 Forest Bond :License: GPL version 2 (see the COPYING file) Overview ======== sclapp is a Python module that makes it easier to write well-behaved command-line applications. Good command-line applications respond in a consistent manner to common situations that occur during normal use in a shell. Programs that do not behave consistently with other command-line applications are not intuitive for command-line users. This makes them unpleasant to use. sclapp helps command-line programs deal with the following issues: * Signal handling * Terminal character encodings * Standard output failures (broken pipes) * Common command-line options (like --help and --version) Using sclapp to implement functionality most command-line programs should implement reduces boiler-plate code and increases consistency across applications. In addition to these standard features, sclapp also provides other functionality that developers of command-line programs may find helpful. Many of these features we added as I needed them for my programs, but are general enough that others may find use for them as well. A Note On The ``locale`` Module =============================== The standard Python ``locale`` module always returns "mac-roman" on Darwin systems, ignoring the LC_* and LANG environment variables. sclapp takes special precautions against this by wrapping the import with some code that works around this quirk. To take advantage of this, callers should not import the ``locale`` module directly, but rather import it directly from sclapp:: from sclapp import locale Wrapping The Main Function ========================== Wrapping the main function with some additional functionality is, to a limited extent, the raison d'être of the sclapp module. The goals of these features are the following: * Improved signal handling on POSIX systems * More graceful handling of stdout and stderr failure conditions * Reduced boiler-plate code and increased consistency for: - Standard help (--help/-h) and (--version/-v) options - User-friendly handling of uncaught exceptions - Reporting critical failures to the user - Daemonization - Stdio character decoding and encoding - Passing of sys.argv into the main function At its simplest, utilizing sclapp's main-wrapping features is as easy as adding a decorator to our main function:: >>> import sys, sclapp >>> @sclapp.main_function ... def main(argv): ... do_something() >>> if __name__ == '__main__': ... sys.exit(main()) While this example utilizes the main_function decorator, the mainWrapper function could have been just as easily used:: >>> import sys, sclapp >>> def main(argv): ... do_something() >>> main = sclapp.mainWrapper(main) >>> if __name__ == '__main__': ... sys.exit(main()) The differences between the main_function decorator and the mainWrapper function are covered in detail below. While duplicate examples will not be provided throughout the entirety of this document, be aware that both methods can be used to achieve the same end. The distinction between the two is largely syntactic. The following sub-sections discuss the benefits provided by sclapp's main function-wrapping features. Improved Signal Handling ------------------------ Rationale ~~~~~~~~~ Python's default signal handling behavior is less than satisfactory for most command-line applications. The following simple example illustrates this:: #!/usr/bin/env python '''yes.py: a yes(1) pseudo-clone in Python''' import sys def main(argv): if len(argv) > 1: string = ' '.join(argv[1:]) else: string = 'y' while True: print string if __name__ == '__main__': main(sys.argv) Let's try our yes clone out. If we run it, we'll see an endless stream of y's fly by on the screen. To stop the program, our instinct would be to press Control-C to send it an interrupt:: $ python yes.py y y y [...] y y y Traceback (most recent call last): File "yes.py", line 16, in main(sys.argv) File "yes.py", line 13, in main print string KeyboardInterrupt Python converts SIGINT to a KeyboardInterrupt (a perfectly reasonable thing to do, given Python's cross-platform nature), which causes a traceback to be printed when we press Control-C. The program does quit like we wanted it to, but this error message is surely not going to go over well with users. In order to write a good yes clone, we'll need to handle KeyboardInterrupt's. How about another experiment:: $ python yes.py | head -n5 y y y y y Traceback (most recent call last): File "yes.py", line 16, in main(sys.argv) File "yes.py", line 13, in main print string IOError: [Errno 32] Broken pipe We get another traceback, but with a different exception. Here, Python is converting SIGPIPE into an IOError. Again, this is probably reasonable behavior for many cross-platform programs, but programs that are designed for use at the command-line will need to handle this better. Pipes are a key operation with UNIX shells. Other situations can be tested, too. Python responds to SIGHUP by immediately exiting, with no exception raised, and printing "Hangup" to standard error. With our yes clone that's not too far from what we want, but what if our program required some cleanup action to be performed prior to exiting? With no exception raised, our cleanup code would never have the opportunity to run. The problem here is that we really do want (or need) to handle signals explicitly in order to avoid these issues. There are a few reasons many Python command-line programs don't actually do this, though: * It's easy to write bad signal-handling code that is either not responsive enough, incorrect, or unreliable. * Writing signal handlers for every Python program is a lot of work, and would result in a lot of duplicate code. * It takes extra attention to handle signals explicitly for systems that support them, without breaking cross-platform compatibility. sclapp makes it possible to handle signals correctly with minimal code changes, and eliminates the duplication of code that would be the result of writing signal handlers for every program. Handling Signals The sclapp Way ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A more behaviorally correct yes clone could be written as follows:: #!/usr/bin/env python '''yes.py: a yes(1) pseudo-clone in Python''' import sys, sclapp @sclapp.main_function def main(argv): if len(argv) > 1: string = ' '.join(argv[1:]) else: string = 'y' while True: print string if __name__ == '__main__': sys.exit(main()) Using sclapp's default options, SIGINT, SIGPIPE, and SIGHUP trigger immediate exit (with no exception raised). This behavior is perfect for our yes clone. For applications that require more sophisticated signal handling, sclapp's signal-handling strategy is to convert signals to exceptions. While signals can be difficult to deal with appropriately, exceptions, are trivially handled in Python, which has language constructs to make them easy to work with. If we needed to perform some cleanup prior to exit, the following Pythonic idiom would be more appropriate:: >>> import sys, sclapp >>> @sclapp.main_function( ... exit_signals = ('SIGPIPE', 'SIGINT', 'SIGHUP', 'SIGTERM') ... ) ... def main(argv): ... try: ... do_something() ... finally: ... cleanup() Since signals are mapped to exceptions, we can actually use a try...finally block to execute cleanup actions. This, of course, is the most appropriate way to perform cleanup actions in Python. Note that signals can be specified by name as strings or by number as integers. If specified by name (like in the example above), any of the specified signals which are nonexistent (due to lack of support by the local system) will be silently ignored. However, unsupported signals specified by number will likely generate exceptions from calls to signal.signal. See the documentation for the signal module in the standard library for more information. More Advanced Signal Handling Configurations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Of course, we may need to handle some signals differently than others, depending on the kind of program that we are writing. In general, sclapp considers each signal to be in one of the following four sets: * ``notify_signals`` * ``exit_signals`` * ``default_signals`` * ``ignore_signals`` sclapp expects that callers will specify which signals are of each type at initialization time:: >>> import sys, signal, sclapp >>> @sclapp.main_function( ... notify_signals = (signal.SIGHUP, signal.SIGUSR1), ... exit_signals = (signal.SIGINT, signal.SIGTERM, signal.SIGPIPE), ... default_signals = ( ), ... ignore_signals = (signal.SIGUSR2, signal.SIGALRM), ... ) ... def main(argv): ... try: ... do_something() ... except SignalError, e: ... if e.signum == signal.SIGHUP: ... handle_sighup() ... elif e.signum == signal.SIGUSR1: ... handle_sigusr1() ... else: ... raise These signals are handled as follows: ``notify_signals`` SignalError is raised asynchronously. The program should catch this exception and do something useful with it. ``exit_signals`` ExitSignalError is raised asynchronously, and the program should exit. Preferably, this exception is handled only through the use of a try...finally block (or equivalent). ``default_signals`` The signal is mapped to signal.SIG_DFL. Program behavior is system-specific. Many of the more command signals on many of the more common systems cause immediate program termination, with no exception raised. ``ignore_signals`` The signal is mapped to signal.SIG_IGN. The signal is completely ignored. Note: I need to be able to ignore signals through a critical section, without missing them altogether. sclapp defines two exceptions that may be raised when signals are caught: * SignalError * ExitSignalError Note that ExitSignalError does not inherit from SignalError, so it is appropriate to use a try...except block to handle SignalError exceptions. If a signal number is specified in more than one of the four categories, an AssertionError will be raised. Protected Output ---------------- sclapp's protected output functionality was created with the purpose of circumventing improper program termination due to massive failure of standard I/O. For instance, suppose your program was run like this: $ python myprog.py 2>&1 | head -n2 The problem that occurs in this sort of scenario is simple, but can be destructive. After the first two lines of output, SIGPIPE is sent to the program. Presumably, this triggers the program to exit, but in the process of doing so, many programs will commonly print some messages to stderr to indicate to the user actions being taken and the status of those actions. If enough output is generated, the program may be terminated without an exception being raised. This would be particularly bad if the program depends on an exception initiating some cleanup action. The scheme used by sclapp to deal with this is to disable output from stderr or stdout if it is triggering EPIPE IOError's. If SIGPIPE is mapped as an exit signal, sclapp wait's until SIGPIPE has been caught before disabling the file. Help and Version Options ------------------------ sclapp can automatically handle help and version command-line options for callers. The option literals are not configurable; -h/--help and -v/--version are intercepted and handled appropriately. To take advantage of this functionality, the ``doc`` and ``version`` keyword arguments to main_function/mainWrapper must be specified. Programs needing more sophisticated command-line option handling should probably use the optparse module from the standard library. User-Friendly Handling of Uncaught Exceptions --------------------------------------------- Uncaught Exceptions are bugs, and sclapp handles them by printing a message to stderr indicating that. The exact text of that message can be customized by passing a template string as argument bug_message to main_function or mainWrapper. If this is left unspecified, sclapp uses a default message (let an exception fly to get a peek at that). See Customizing Messages, below, for more information on template strings. Error Reporting (CriticalError, UsageError) ------------------------------------------- Command-line programs have a few different mechanisms by which they notify the user of failure conditions. In many cases, a message indicating the nature of the error that occured should be printed to sys.stderr, and the program should exit with a non-zero return code. Since this functionality is a common requirement of command-line programs, sclapp provides a few exceptions to assist: ``CriticalError`` Indicates that an irrecoverable error that is not a usage error has occured, and the program must terminate. ``UsageError`` Indicates that the user's specification of runtime parameters was erroneous. These exceptions are caught and handled by the main-wrapping code, so callers should raise them to indicate errors. Note that UsageError inherits from CriticalError. CriticalError exceptions accept two optional positional arguments: an exit code, and an error message. If the message is omitted, none will be printed at exit. If the exit code is omitted, the program will exit with zero status. UsageError exceptions accept a signle optional positional argument: an error message. If this error message is not specified, the program will exit with zero status and simply print the usage information for the program, if possible. Otherwise, the message will be printed, the usage information for the program will be printed, and the program will exit with the specified exit status. Note that sclapp's main-wrapping code does not actually call the sys.exit function. Instead, the main function's return value will be the appropriate exit status, and the caller should call sys.exit itself. Daemonization ------------- Daemonization (on UNIX-like systems) is a technique that causes programs to be run in the background, completely detached from a terminal. It is generally accepted that the following four steps must be taken to properly daemonize: 1. The current working directory set to the "/" directory. 2. The current file creation mode mask set to 0. 3. Close all open files (1024). 4. Redirect standard I/O streams to "/dev/null". For further discussion of this, see http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731. To this end, the keyword argument ``daemonize`` may be given to the main_function decorator (or the mainWrapper function) and sclapp will daemonize the program before passing control to the caller's main function. Transparent Encoding and Decoding of Standard IO ------------------------------------------------ By default, sclapp will provide conversion to and from the preferred character encoding (as determined by locale.getpreferredencoding) for sys.stdin, sys.stdout, and sys.stderr. That means that unicode objects may be printed directly to sys.stdout and sys.stderr, and the output will be encoded appropriately. Characters read from sys.stdin will be decoded to unicode strings also. There may be some applications for which this behavior would be undesirable. To disable it, pass False for the one or more of the following keyword arguments to sclapp's main_function or mainWrapper: * decode_stdin * encode_stdout * encode_stderr In particular, automatic decoding of sys.stdin causes problems for the built-in functions input and raw_input. Thus, when this decoding feature is engaged, the alternative functions provided by sclapp should be used. Command-Line Argument Decoding ------------------------------ sclapp will automatically decode the command-line arguments to Unicode strings. If the ``decode_argv`` argument to ``main_function`` and ``mainWrapper`` is True, the members of the ``argv`` argument passed to your main function will contain Unicode strings instead of byte strings. This feature is on by default. Customizing Messages -------------------- sclapp uses string templates to format messages. Several arguments to the ``mainWrapper`` function represent customizable messages that the user may see on a variety of occasions: ``version_message`` Used for handling the --version command-line option. ``doc`` Printed to the screen when handling the --help command-line option. ``bug_message`` Printed to the screen when an unhandled exception is caught. Each of these messages can be customized with the following template substitutions: ``name`` The name of the program. ``author`` The author of the program. Suggested format includes the author's e-mail address, like "Forest Bond ". ``version`` A string representing the version of the program, like "0.6.3". ``traceback`` Traceback for the uncaught exception. Only useful for ``bug_message``. So, to customize these messages, use template-style string substitution. For instance:: @sclapp.main_function( version = '0.2.5', author = 'Forest Bond ', doc = '''\ Ouch! This looks like a bug! ${traceback} Program version: ${version} Please send the full text of this message in an e-mail to ${author}. ''' ) def main(argv): return 0 See the documentation for template strings (string.Template) in the standard library for more information on syntax. Modules ======= sclapp.daemonize ---------------- This module contains a function of the same name that will cause the caller's process to become a daemon. See the section Daemonization, above. sclapp.pipes ------------ I implemented some strange functions that make it possible to run Python functions in sub-processes and connect their standard I/O streams together with pipes. At the very least, they are useful for testing behavior with pipes. The functions: :: pipeFns(fns, argses = None, kwargses = None, pipe_out = True, pipe_err = False, stdin = None, stdout = None, stderr = None) The functions in lists ``fns`` are piped in to each other in order from first to last. The positional and keyword arguments for each function is specified as arguments ``argses`` and ``kwargses``, respectively. ``pipe_out`` and ``pipe_err`` are booleans that determine the source of the input stream for the following function. Arguments ``stdin``, ``stdout``, and ``stderr`` may be specified as for ``redirection.redirectFds`` in order to change the default standard I/O streams for the sub-processes in the pipe. sclapp.processes ---------------- This module contains a few classes for managing background processes. It is often useful to run a program or function in the background. The classes: BackgroundFunction( self, function, args, kwargs, stdin = None, stdout = None, stderr = None) This class can be used to manage a Python function running in a sub-process. ``function`` is the Python function to call; ``args`` and ``kwargs`` are the arguments and keyword arguments (respectively) to pass to the function. BackgroundCommand( self, command, args, stdin = None, stdout = None, stderr = None) This class can be used to manage an external program running in a sub-process. The arguments ``command`` and ``args`` should specify the command to run and the arguments to pass to it, as for ``os.execvp``. For both classes, the arguments ``stdin``, ``stdout``, and ``stderr`` are as for ``redirection.redirectFds``, and the sub-process's standard I/O streams will be redirected as indicated prior to launching the caller's function or command. sclapp.redirection ------------------ Implements a single function: ``redirectFds(stdin = None, stdout = None, stderr = None)`` The standard I/O streams will be immediately redirected as specified. Each argument may be one of the following: * None, indicating no redirection for that stream. * A file-like object supporting method ``fileno``. * An integer file descriptor. * A filename. sclapp.services --------------- startService(pid_file_name, fn, args = None, kwargs = None) stopService(pid_file_name) sclapp.shell ------------ This module implements some classes for interacting with ongoing shell sessions. Warning: my testing with the version of bash distributed with Mac OSX 10.4 indicates that the readline support causes problems due to interference with disabling the tty echo function. If you are dealing with such a version of bash, you should pass the ``--noediting`` command-line option to bash in order to disable readline, and the ``--posix`` command-line option for more standards-compliant behavior. Starting A Session ~~~~~~~~~~~~~~~~~~ A shell session is begun by instantiating the approprate class. Generally, you will use the ``Shell`` class. The class initializer has no required arguments, however, the following optional keyword arguments may be specified: shell (default: '/bin/sh') Path to the shell executable; must be Bourne compatible. prompt (default: randomly generated prompt) The desired shell prompt. Be aware that the prompt is the only indication that a command has finished executing, so it is normally set to a reasonably long random string of alpha-numeric characters. See Searching For The Prompt, below. failure_exceptions (default: True) Boolean specifying whether or not a CommandFailed exception should be raised when a command exits with a non-zero return code. signal_exceptions (default: True) Boolean specifying whether or not a CommandSignalled exception should be raised when a command exits due to a signal trace (default: False) If True, commands are printed to stdout prior to being executed. This is for debugging. timeout (default: 30) The initial timeout used by the underling Pexpect object. Note that other timeout values may be used by various ``Shell`` methods as necessary, but the timeout will be restored after those temporary changes. delaybeforesend (default: 100; defined by pexpect module) The number of milliseconds to delay before sending input to the shell process. See the pexpect documentation for more information. Any additional keyword arguments are passed on to the underlying Pexpect object's initializer. The ``SudoShell`` class provides the same functionality as the ``Shell`` class, but the shell session is started using the sudo command. Thus, the shell has root privileges. The ``SudoShell`` initializer takes an additional optional keyword argument, ``password``. If this argument is not specified and sudo requests a password, or if an incorrect password is specified, a ValueError is raised. Executing Commands ~~~~~~~~~~~~~~~~~~ There are a few different ways that commands can be executed, depending upon what kind of behavior is desired. The following methods can be used: ``close()`` Close the tty that the shell is attached to. This should always be called to clean up after the Shell instance. ``execute(cmd, *values)`` Execute ``cmd``, using ``sclapp.shinterp.interpolate`` to interpolate ``values`` into ``cmd``. The return value is the tuple ``(status, output)``, indicating the exit status and output of the command. ``follow(cmd, *values, **kwargs)`` ``cmd`` is executed and ``values`` interpolated as with ``execute``, but ``follow`` is actually a generator. Callers should iterate over the return value to handle each output character individually. For instance:: output = '' for ch in follow('echo foo'): output = output + ch print output The above code would print the string 'foo\n' to the screen. ``follow`` and the following wrapper methods accept a keyword argument ``follow_input``. If True, the executed command is included in the resulting character stream. This argument defaults to False. ``followCallback(cmd, *values, **kwargs)`` Like ``follow``, but rather than yielding each output character to the caller, ``followCallback`` requires a keyword argument, ``callback``, which should be used to pass a callback function that will be called once for each output character. The callback function should accept a single positional argument, the character being handled. Returns the exit status of the command. ``followWrite(cmd, *values, **kwargs)`` Calls ``followCallback`` with a simple callback function that writes each output character to a file-like object. This object can be specified using optional keyword argument ``outfile``, which defaults to ``sys.stdout``. Returns the exit status of the command. ``followWriteReturn(cmd, *values, **kwargs)`` Like ``followWrite``, but in addition to writing each output character to ``outfile``, the command output is also captured and returned as part of a two-tuple like that returned by ``execute``. ``interact(fitted = False)`` Cause the shell to be connected to stdin/stdout/stderr so that the user can use the shell directly. If keyword argument ``fitted`` is set to True, the window size for the underlying tty object is kept in sync with the window size of stdout. It has been observed that commands including lines longer than 4095 characters cause problems with some shells on some systems. The methods above will refuse to execute such a command unless the optional keyword argument ``force`` is passed with a boolean True value. Otherwise, if such a command is passed for execution, a ValueError will be raised. sclapp.shinterp --------------- This module provides some simple functions for performing string interpolation with shell quoting of parameters. For instance: >>> from sclapp import shinterp >>> x, y = 'foo', 'bar' >>> print shinterp.interpolate('cat ? ?', x, y) cat 'foo' 'bar' However, this function will also correctly handle double and single quotes: >>> from sclapp import shinterp >>> x, y = 'fo"o', "ba'r" >>> print shinterp.interpolate('cat ? ?', x, y) cat 'fo"o' 'ba'\''r' To insert a literal question mark, use two of them: >>> from sclapp import shinterp >>> x = 'foo' >>> print shinterp.interpolate('cat ? ??', x) cat 'foo' ? sclapp.stdio_encoding --------------------- This module implements support for transparent character encoding for standard I/O. In addition it contains a few utility functions related to this implementation. Transparent decoding of standard input has been found to conflict with the built-in functions ``raw_input`` and ``input``. As a result, alternatives have been implemented that do not cause problems. These functions can be imported from this module, and are have the same names as the functions they replace. sclapp.termcontrol ------------------ Based on a ASPN Python Cookbook recipe written by Edward Loper, this module wraps some basic functionality provided by the curses module to provide easy access to some simple terminal features. To use the module, the initialization funcdtion ``initializeTermControl`` must first be called. Terminal capabilities are utilized by writing control strings to standard output. These control strings are accessible to callers via the dict ``sclapp.termcontrol.caps`` that is populated during initialization. If a given terminal capability is not supported, the corresponding value in the ``caps`` dict will be the empty string. Thus, most applications will degrade gracefully when dealing with a terminal lacking capabilities. The following keys are used to access terminal capabilities in the ``caps`` dict: BOL Move the cursor to the beginning of the line. UP Move the cursor up one line. DOWN Move the cursor down one line. LEFT Move the cursor left one column. RIGHT Move the cursor right one column. CLEAR_SCREEN Clear the screen. CLEAR_EOL Clear to the end of the current line. CLEAR_BOL Clear to the beginning of the current line. CLEAR_EOS Clear to the end of the screen. BOLD Use bold text. BLINK Use blinking text. DIM Use dim text. REVERSE Use reverse-color text. UNDERLINE Use underlined text. NORMAL Use normal text (cancels previously set text color and style). HIDE_CURSOR Hide the cursor. SHOW_CURSOR Show the cursor. The following dict keys correspond with various text colors. For each color, the same key can be used with the prefix "BG\_" to affect the background color instead of the foreground color. * BLACK * BLUE * GREEN * CYAN * RED * MAGENTA * YELLOW * WHITE sclapp.util ----------- A few miscellaneous functions: safe_encode(s) Encodes the Unicode string s using the encoding returned by locale.getprefferedencoding() without ever raising a UnicodeEncodeError. safe_decode(s) Decodes the byte string s to unicode using the encoding returned by locale.getpreferredencoding() without ever raising a UnicodeDecodeError. sclapp-0.5.3/setup.py0000755000175000017500000001233311041121232012545 0ustar fabfab#!/usr/bin/env python # Author: Forest Bond # This file is in the public domain. import os, sys, commands, glob from distutils.command.build import build as _build from distutils.command.clean import clean as _clean from distutils.command.build_py import build_py from distutils.core import setup, Command from distutils.spawn import spawn from distutils import log from distutils.dir_util import remove_tree from distutils.dist import Distribution project_dir = os.path.dirname(__file__) sys.path.insert(0, project_dir) from tests.common import TEST_DIR ################################################################################ class test(Command): description = 'run tests' user_options = [('tests=', None, 'names of tests to run')] def initialize_options(self): self.tests = None def finalize_options(self): if self.tests is not None: self.tests = self.tests.split(',') def run(self): from tests import load, main load() main(self.tests) ################################################################################ class clean(_clean): temporary_files = [] nontemporary_files = [] temporary_dirs = [] nontemporary_dirs = [] def clean_file(self, filename): if not os.path.exists(filename): log.info("'%s' does not exist -- can't clean it", filename) return log.info("removing '%s'" % filename) if not self.dry_run: try: os.unlink(filename) except (IOError, OSError): log.warn("failed to remove '%s'" % filename) def clean_dir(self, dirname): if not os.path.exists(dirname): log.info("'%s' does not exist -- can't clean it", dirname) return log.info("removing '%s' (and everything under it)" % dirname) if not self.dry_run: try: remove_tree(dirname) except (IOError, OSError): log.warn("failed to remove '%s'" % dirname) def run(self): remove_files = list(self.temporary_files) if self.all: remove_files = remove_files + self.nontemporary_files for filename in remove_files: if callable(filename): filename = filename(self.distribution) self.clean_file(filename) remove_dirs = list(self.temporary_dirs) if self.all: remove_dirs = remove_dirs + self.nontemporary_dirs for dirname in remove_dirs: if callable(dirname): dirname = dirname(self.distribution) self.clean_dir(dirname) _clean.run(self) ################################################################################ class build_version_file(build_py): def initialize_options(self): build_py.initialize_options(self) self.version = None self.version_file = None def finalize_options(self): build_py.finalize_options(self) self.packages = self.distribution.packages self.py_modules = [self.distribution.version_module] self.version = self.distribution.get_version() self.version_file = self.distribution.version_file def check_module(self, *args, **kwargs): pass def build_modules(self, *args, **kwargs): log.info("creating version file '%s'" % self.version_file) if not self.dry_run: f = open(self.version_file, 'w') f.write('version = %s' % repr(self.version)) f.close() build_py.build_modules(self, *args, **kwargs) clean.temporary_files.append(lambda distribution: distribution.version_file) Distribution.version_module = None Distribution.release_file = None def get_bzr_version(): status, output = commands.getstatusoutput('bzr revno') return 'bzr%s' % output.strip() def get_version(release_file): try: f = open(release_file, 'r') try: version = f.read().strip() finally: f.close() except (IOError, OSError): version = get_bzr_version() return version def get_version_file(version_module): return '%s.py' % os.path.join(*(version_module.split('.'))) def wrap_init(fn): def __init__(self, *args, **kwargs): fn(self, *args, **kwargs) self.version_file = get_version_file(self.version_module) self.metadata.version = get_version(self.release_file) return __init__ Distribution.__init__ = wrap_init(Distribution.__init__) ################################################################################ class build(_build): sub_commands = _build.sub_commands + [ ('build_version_file', (lambda self: True)), ] ################################################################################ setup( cmdclass = { 'test': test, 'build': build, 'build_version_file': build_version_file, 'clean': clean, }, name = 'sclapp', version_module = 'sclapp.version', packages = ['sclapp'], release_file = 'release', author = 'Forest Bond', author_email = 'forest@alittletooquiet.net', url = 'http://www.alittletooquiet.net/software/sclapp/', license = 'GPLV2', description = 'A framework for python command-line applications.', ) sclapp-0.5.3/tests/0000755000175000017500000000000011041121232012170 5ustar fabfabsclapp-0.5.3/tests/__init__.py0000644000175000017500000000101411041121232014275 0ustar fabfabimport glob, os def get_test_modules(): for name in glob.glob(os.path.join(os.path.dirname(__file__), '*.py')): if name == '__init__': continue module = os.path.basename(name)[:-3] yield module def load(): for module in get_test_modules(): __import__('tests', {}, {}, [module]) def run(test_names = None): from tests.manager import manager manager.run(test_names) def main(test_names = None): from tests.manager import manager manager.main(test_names) sclapp-0.5.3/tests/_shell_yes_bullet.py0000644000175000017500000000035111041121232016236 0ustar fabfabfrom sclapp.shell import Shell from sclapp import stdio_encoding def main(): sh = Shell() stdio_encoding.enableStdioEncoding() sh.followWrite(u'yes \u2022 2>/dev/null | head -n10') if __name__ == '__main__': main() sclapp-0.5.3/tests/background_command.py0000644000175000017500000000264411041121232016365 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys, signal from unittest import main import sclapp from common import ( SclappTestCase, assertLogFileContains, ) from manager import manager class BackgroundCommandTestCase(SclappTestCase): @staticmethod def test_basic_operation(): from sclapp import processes as s_processes from sclapp import debug_logging p = s_processes.BackgroundCommand( 'echo', ['echo', 'testing'], stdout = debug_logging.DEBUG_LOGFILE ) p.run() p.wait() assertLogFileContains('testing') assert p.getExitStatus() == 0 @staticmethod def test_death_by_signal(): import time from sclapp import processes as s_processes from sclapp import debug_logging p = s_processes.BackgroundCommand( 'yes', ['yes', 'testing'], stdout = debug_logging.DEBUG_LOGFILE) p.run() time.sleep(1) p.kill() p.wait() assertLogFileContains('testing') assert p.getExitSignal() == 2 manager.add_test_case_class(BackgroundCommandTestCase) sclapp-0.5.3/tests/bug_handling.py0000644000175000017500000000234011041121232015162 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. __author__ = 'Forest Bond' import os, sys, signal from unittest import main import sclapp from common import ( SclappTestCase, redirectToLogFile, waitForPid, assertLogFileContains, ) from manager import manager @sclapp.main_function(author = __author__, exit_signals = [ signal.SIGINT ]) def _main(argv): import time while True: print 'testing bug handling' time.sleep(1) this_should_fail() class BugHandlingTestCase(SclappTestCase): @staticmethod def test_basic_handling(): sclapp.setErrorOutputLevel(sclapp.WARNING) pid = os.fork() if not pid: redirectToLogFile() _main() os._exit(0) waitForPid(pid) assertLogFileContains('Traceback') assertLogFileContains('file a bug report') assertLogFileContains(__author__) manager.add_test_case_class(BugHandlingTestCase) sclapp-0.5.3/tests/common.py0000644000175000017500000000711611041121232014037 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import sys, os, re from unittest import TestCase import sclapp from sclapp import debug_logging def execHead(num_lines = 5): pid = os.fork() if not pid: os.execvp('head', [ 'head', '-n%u' % num_lines ]) return pid CAUGHT_SIGNALS_REGEX = r'pid ([0-9]+) caught signals: ([0-9, ]*)' caught_signals_regex_compiled = re.compile(CAUGHT_SIGNALS_REGEX) def dumpLogFile(): print 'Logfile contents:' print 80 * '-' sys.stdout.write(debug_logging.readLogFile()) print 80 * '-' def getLoggedSignals(pid): contents = debug_logging.readLogFile() for line in contents.split('\n'): match = caught_signals_regex_compiled.match(line) if match is not None: gs = match.groups() assert(len(gs) == 2) logged_pid = int(gs[0]) if pid == logged_pid: signums = [ int(x) for x in [ x.strip() for x in gs[1].split(',') ] if x ] return signums def verifySignalCaught(signum, pid): signums = getLoggedSignals(pid) return ((signums is not None) and (signum in signums)) def assertSignalCaught(signum, pid): assert verifySignalCaught(signum, pid) def logSignals(): debug_logging.logMessage('pid %u caught signals: %s' % \ (os.getpid(), ', '.join([str(x) for x in sclapp.getCaughtSignals()]))) def waitForPid(pid): return os.waitpid(pid, 0) def removeLogFile(): try: return debug_logging.removeLogFile() except (OSError, IOError): pass def redirectToLogFile(): from sclapp import processes as s_processes return s_processes.redirectFds( stdout = debug_logging.DEBUG_LOGFILE, stderr = debug_logging.DEBUG_LOGFILE ) def assertLogFileContains(needle): haystack = debug_logging.readLogFile() assert (haystack.find(needle) > -1) def assertLogFileDoesNotContain(needle): haystack = debug_logging.readLogFile() assert (haystack.find(needle) < 0) def grepCount(haystack, needle): count = 0 i = -1 while True: i = haystack.find(needle, i + 1) if i == -1: break count = count + 1 return count def assertLogFileContainsExactly(needle, num): haystack = debug_logging.readLogFile() count = grepCount(haystack, needle) assert (count == num), ( 'Expected exactly %u, found %u. Logfile:\n%s' % ( num, count, debug_logging.readLogFile() )) def assertLogFileContainsAtLeast(needle, min): haystack = debug_logging.readLogFile() count = grepCount(haystack, needle) assert (count >= min), ( 'Expected at least %u, found %u. Logfile:\n%s' % ( min, count, debug_logging.readLogFile() )) def assertLogFileContainsAtMost(needle, max): haystack = debug_logging.readLogFile() count = grepCount(haystack, needle) assert (count <= max), ( 'Expected at most %u, found %u. Logfile:\n%s' % ( max, count, debug_logging.readLogFile() )) class SclappTestCase(TestCase): def setUp(self): removeLogFile() return super(SclappTestCase, self).setUp() def tearDown(self): removeLogFile() return super(SclappTestCase, self).tearDown() TEST_DIR = os.path.abspath(os.path.dirname(__file__)) PROJECT_DIR = os.path.dirname(TEST_DIR) sclapp-0.5.3/tests/custom_handlers.py0000644000175000017500000000205511041121232015736 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys, signal, time from unittest import main from manager import manager __doc__ = ''' >>> import sclapp >>> def myhandler(signum, frame): ... print 'myhandler' >>> sclapp.enableSignalHandling( ... notify_signals = (signal.SIGINT,), ... custom_handlers = {signal.SIGHUP: myhandler}, ... announce_signals = False ... ) >>> def f(): ... os.kill(os.getpid(), signal.SIGINT) ... time.sleep(2) >>> f() Traceback (most recent call last): ... SignalError: caught signal 2 >>> def f(): ... os.kill(os.getpid(), signal.SIGHUP) ... time.sleep(2) >>> f() myhandler >>> sclapp.disableSignalHandling() ''' manager.add_doc_test_cases_from_string( __doc__, globs = globals(), name = __name__, ) sclapp-0.5.3/tests/documentation.py0000644000175000017500000000034211041121232015412 0ustar fabfabimport os from unittest import main from common import PROJECT_DIR from manager import manager doc_filename = os.path.join(PROJECT_DIR, 'sclapp.txt') manager.add_doc_test_cases_from_text_file(doc_filename, name = __name__) sclapp-0.5.3/tests/main_function_decorator.py0000644000175000017500000000235411041121232017441 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys from unittest import main from common import ( SclappTestCase, waitForPid, assertLogFileContains, assertLogFileDoesNotContain, removeLogFile, ) from manager import manager import sclapp from sclapp import debug_logging @sclapp.main_function def _main(argv): print 'testing' return 0 class MainFunctionDecoratorTestCase(SclappTestCase): @staticmethod def test_noargs(): from sclapp import pipes as s_p pid = os.fork() if not pid: s_p.redirectFds( stdin = '/dev/null', stdout = debug_logging.DEBUG_LOGFILE, stderr = debug_logging.DEBUG_LOGFILE ) _main() os._exit(0) waitForPid(pid) assertLogFileContains('testing') assertLogFileDoesNotContain('Traceback') removeLogFile() manager.add_test_case_class(MainFunctionDecoratorTestCase) sclapp-0.5.3/tests/manager.py0000644000175000017500000000745711041121232014171 0ustar fabfabimport sys from doctest import DocTestCase, DocTestFinder, DocTestParser from unittest import TestCase, TestSuite, TextTestRunner, TestLoader class ListTestLoader(TestLoader): suiteClass = list class TestManager(object): tests = None loader = None def __init__(self): self.tests = [] self.loader = ListTestLoader() def main(self, test_names = None): result = self.run(test_names) if not result.wasSuccessful(): sys.exit(1) sys.exit(0) def run(self, test_names = None): suite = TestSuite() runner = TextTestRunner(verbosity = 2) for test in self.tests: if self.should_run_test(test, test_names): suite.addTest(test) return runner.run(suite) def should_run_test(self, test, test_names): if test_names is None: return True for test_name in test_names: test_name_parts = test_name.split('.') relevant_id_parts = test.id().split('.')[:len(test_name_parts)] if test_name_parts == relevant_id_parts: return True return False def add_test_suite(self, test_suite): self.tests.extend(self.flatten_test_suite(test_suite)) def flatten_test_suite(self, test_suite): tests = [] if isinstance(test_suite, TestSuite): for test in list(test_suite): tests.extend(self.flatten_test_suite(test)) else: tests.append(test_suite) return tests def add_test_case_class(self, test_case_class): self.tests.extend( self.loader.loadTestsFromTestCase(test_case_class)) def make_doc_test_case(self, test): def __init__(self, *args, **kwargs): DocTestCase.__init__(self, test) return type( '%s_TestCase' % test.name.split('.')[-1], (DocTestCase,), {'__init__': __init__}, ) def get_doc_test_cases_from_string( self, string, name = '', filename = '', globs = None, ): if globs is None: globs = {} # Make sure __name__ == '__main__' checks fail: globs = dict(globs) globs['__name__'] = None parser = DocTestParser() test = parser.get_doctest( string, globs = globs, name = name, filename = filename, lineno = 0, ) test_case = self.make_doc_test_case(test) return [test_case] def add_doc_test_cases_from_string(self, *args, **kwargs): for test_case in self.get_doc_test_cases_from_string(*args, **kwargs): self.add_test_case_class(test_case) def get_doc_test_cases_from_module(self, name): from sclapp.util import importName mod = importName(name) finder = DocTestFinder() tests = finder.find(mod) doc_test_cases = [] for test in tests: doc_test_cases.append(self.make_doc_test_case(test)) return doc_test_cases def add_doc_test_cases_from_module(self, dst_name, src_name = None): if src_name is None: src_name = dst_name for test_case in self.get_doc_test_cases_from_module(src_name): test_case.__module__ = dst_name self.add_test_case_class(test_case) def get_doc_test_cases_from_text_file(self, filename, *args, **kwargs): f = open(filename, 'r') try: data = f.read() finally: f.close() return self.get_doc_test_cases_from_string(data, *args, **kwargs) def add_doc_test_cases_from_text_file(self, *args, **kwargs): for test_case in self.get_doc_test_cases_from_text_file( *args, **kwargs): self.add_test_case_class(test_case) manager = TestManager() sclapp-0.5.3/tests/output_protection.py0000644000175000017500000001372011041121232016353 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys, signal, errno from unittest import main from common import ( logSignals, execHead, waitForPid, assertLogFileContainsExactly, assertSignalCaught, assertLogFileContains, SclappTestCase, ) from manager import manager import sclapp def _main(argv): import time sclapp.debug() try: for i in range(30): try: print 'foo' sys.stdout.flush() except sclapp.CriticalError: pass except IOError, e: if e.errno != errno.EPIPE: raise break try: print >>sys.stderr, 'bar' sys.stderr.flush() except sclapp.CriticalError: pass except IOError, e: if e.errno != errno.EPIPE: raise break time.sleep(0.2) finally: time.sleep(2) logSignals() return 0 # Use announce_signals = False to prevent signal announcement (on stderr) from # disturbing our stdio testing. main_with_signal_handling_with_output_protection = sclapp.mainWrapper( _main, exit_signals = [signal.SIGPIPE], protect_output = True, announce_signals = False, bug_message = None, ) main_with_signal_handling_without_output_protection = sclapp.mainWrapper( _main, exit_signals = [signal.SIGPIPE], protect_output = False, announce_signals = False, bug_message = None, ) main_without_signal_handling_with_output_protection = sclapp.mainWrapper( _main, handle_signals = False, protect_output = True, bug_message = None, ) main_without_signal_handling_without_output_protection = sclapp.mainWrapper( _main, handle_signals = False, protect_output = False, bug_message = None, ) def _stdio_fails(main_fn, pipe_out = True, pipe_err = False): from sclapp.debug_logging import DEBUG_LOGFILE from sclapp import pipes as s_p fns = [ main_fn, execHead ] argses = [ None, [ 2 ] ] kwargses = [ None, None ] pid = s_p.pipeFns( fns, argses, kwargses, pipe_out = pipe_out, pipe_err = pipe_err, stdin = '/dev/null', stdout = DEBUG_LOGFILE, stderr = DEBUG_LOGFILE ) waitForPid(pid) return pid def _stdout_fails(main_fn): return _stdio_fails(main_fn) def _stdout_stderr_fail(main_fn): return _stdio_fails(main_fn, pipe_err = True) def _make_test_fn(name, failure_fn, main_fn, foos, bars, signals, exceptions = ()): def test_fn(): pid = failure_fn(main_fn) assertLogFileContainsExactly('foo', foos) assertLogFileContainsExactly('bar', bars) for signal in signals: assertSignalCaught(signal, pid) for exception in exceptions: assertLogFileContains(exception.__name__) test_fn.__name__ = name return test_fn class TestOutputProtection(SclappTestCase): test_stdout_fails_with_signal_handling_with_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_fails_with_signal_handling_with_output_protection', _stdout_fails, main_with_signal_handling_with_output_protection, foos = 2, bars = 30, signals = (signal.SIGPIPE,) ))) test_stdout_stderr_fail_with_signal_handling_with_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_stderr_fail_with_signal_handling_with_output_protection', _stdout_stderr_fail, main_with_signal_handling_with_output_protection, foos = 1, bars = 1, signals = (signal.SIGPIPE,) ))) test_stdout_fails_with_signal_handling_without_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_fails_with_signal_handling_without_output_protection', _stdout_fails, main_with_signal_handling_without_output_protection, foos = 0, bars = 0, signals = (), exceptions = (AssertionError,) ))) # Don't expect to hear about the AssertionError since stderr is borked. test_stdout_stderr_fail_with_signal_handling_without_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_stderr_fail_with_signal_handling_without_output_protection', _stdout_stderr_fail, main_with_signal_handling_without_output_protection, foos = 0, bars = 0, signals = () ))) # Note that, depending upon buffering of stdio, we may or may not get an # IOError when writing into a broken pipe with signals disabled. test_stdout_fails_without_signal_handling_with_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_fails_without_signal_handling_with_output_protection', _stdout_fails, main_without_signal_handling_with_output_protection, 2, 30, () ))) test_stdout_stderr_fail_without_signal_handling_with_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_stderr_fail_without_signal_handling_with_output_protection', _stdout_stderr_fail, main_without_signal_handling_with_output_protection, 1, 1, () ))) test_stdout_fails_without_signal_handling_without_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_fails_without_signal_handling_without_output_protection', _stdout_fails, main_without_signal_handling_without_output_protection, 2, 2, () ))) test_stdout_stderr_fail_without_signal_handling_without_output_protection = ( staticmethod(_make_test_fn( 'test_stdout_stderr_fail_without_signal_handling_without_output_protection', _stdout_stderr_fail, main_without_signal_handling_without_output_protection, 1, 1, () ))) manager.add_test_case_class(TestOutputProtection) sclapp-0.5.3/tests/shell.py0000644000175000017500000000613211041121232013653 0ustar fabfab# coding: utf-8 # Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys, signal from unittest import main from manager import manager __doc__ = r''' >>> import os, sclapp, signal >>> from sclapp import shell >>> sh = shell.Shell() >>> sh.execute('false') Traceback (most recent call last): ... CommandFailed: ('false', 1, u'') >>> sh.execute(u'touch ?', u'•') (0, u'') >>> sh.execute('rm ?', u'•') (0, u'') >>> chars = '' >>> for ch in sh.follow('echo "foo"; sleep 2'): ... chars = chars + ch ... if len(chars) >= 3: ... sh.interrupt() Traceback (most recent call last): ... CommandSignalled: ('echo "foo"; sleep 2', 130, None) >>> chars = '' >>> for ch in sh.follow('echo "foo"; sleep 2'): ... chars = chars + ch ... if len(chars) >= 3: ... sh.interrupt() Traceback (most recent call last): ... CommandSignalled: ('echo "foo"; sleep 2', 130, None) Test that commands can be interrupted asynchronously but the shell returns to a known, usable state: >>> sclapp.enableSignalHandling( ... notify_signals = (signal.SIGALRM,), ... announce_signals = False ... ) >>> signal.alarm(1) 0 >>> sh.execute('sleep 5') Traceback (most recent call last): ... SignalError: caught signal 14 >>> sh.execute('echo foo') (0, u'foo\n') >>> signal.alarm(1) 0 >>> for ch in sh.follow('sleep 5'): pass Traceback (most recent call last): ... SignalError: caught signal 14 >>> sh.execute('echo foo') (0, u'foo\n') >>> status = sh.followWrite('yes 2>/dev/null | head -n10') y y y y y y y y y y >>> status 0 >>> from sclapp.pipes import pipeFns >>> status, output = sh.execute('echo $LANG') >>> language, encoding = output.strip().split('.') # I'd like to test sh.followWrite more thoroughly here, but doctest doesn't # seem to deal well with non-ascii characters written directly to stdout. # Consequently, the tests are executed in a separate process: >>> sh.execute('python _shell_yes_bullet.py >tmpfile.txt') (0, u'') >>> f = open('tmpfile.txt', 'r') >>> unicode(f.read(), encoding) u'\u2022\n\u2022\n\u2022\n\u2022\n\u2022\n\u2022\n\u2022\n\u2022\n\u2022\n\u2022\n' >>> f.close() >>> os.unlink('tmpfile.txt') >>> chars = [ ] >>> signal.alarm(3) 0 >>> for ch in sh.follow('echo "foo"; sleep 5'): ... chars.append(ch) Traceback (most recent call last): ... SignalError: caught signal 14 >>> ''.join(chars) u'foo\n' >>> sclapp.disableSignalHandling() Commands can be executed in the background: >>> sh.execute('yes >/dev/null & pid=$!') (0, u'') >>> status, output = sh.execute('ps $pid') >>> output.strip().endswith('yes') True >>> sh.execute('kill $pid; wait $pid; sleep 1') (0, u'') >>> sh.execute('ps $pid') Traceback (most recent call last): ... CommandFailed: ('ps $pid', 1, u' PID TTY STAT TIME COMMAND\n') ''' manager.add_doc_test_cases_from_string( __name__, globs = globals(), name = __name__, ) sclapp-0.5.3/tests/shinterp.py0000644000175000017500000000071711041121232014403 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. from unittest import main from manager import manager manager.add_doc_test_cases_from_module(__name__, 'sclapp.shinterp') sclapp-0.5.3/tests/signals.py0000644000175000017500000000475511041121232014215 0ustar fabfab# Copyright (c) 2005-2007 Forest Bond. # This file is part of the sclapp software package. # # sclapp is free software; you can redistribute it and/or modify it under the # terms of the GNU General Public License version 2 as published by the Free # Software Foundation. # # A copy of the license has been included in the COPYING file. import os, sys, signal from unittest import main import sclapp from common import ( logSignals, SclappTestCase, waitForPid, assertSignalCaught, execHead, ) from manager import manager @sclapp.main_function(notify_signals = [signal.SIGHUP, signal.SIGPIPE]) def _main(argv): import time sclapp.debug() try: for i in range(20): try: print 'foo' print >>sys.stderr, 'bar' os.kill(os.getpid(), signal.SIGHUP) except sclapp.SignalError, e: if e.signum == signal.SIGHUP: pass if e.signum == signal.SIGPIPE: pass time.sleep(0.5) finally: logSignals() return 0 class SignalsTestCase(SclappTestCase): @staticmethod def test_notify_sighup(): from sclapp.debug_logging import DEBUG_LOGFILE from sclapp import pipes as s_p pid = os.fork() if not pid: s_p.redirectFds( stdin = '/dev/null', stdout = DEBUG_LOGFILE, stderr = DEBUG_LOGFILE) _main() os._exit(0) waitForPid(pid) assertSignalCaught(signal.SIGHUP, pid) @staticmethod def test_notify_sighup_sigpipe(): from sclapp.debug_logging import DEBUG_LOGFILE from sclapp import pipes as s_p fns = [ _main, execHead ] argses = [ None, [ 2 ] ] kwargses = [ None, None ] pid = s_p.pipeFns(fns, argses, kwargses, stdin = '/dev/null', stdout = DEBUG_LOGFILE, stderr = DEBUG_LOGFILE) waitForPid(pid) assertSignalCaught(signal.SIGHUP, pid) assertSignalCaught(signal.SIGPIPE, pid) @staticmethod def test_string_signal_mapping_specification(): import time sclapp.enableSignalHandling( notify_signals = ['SIGALRM'], announce_signals = False, ) try: signal.alarm(1) time.sleep(3) except sclapp.SignalError: pass else: self.fail('Expected to receive SIGALRM.') sclapp.disableSignalHandling() manager.add_test_case_class(SignalsTestCase)