pax_global_header00006660000000000000000000000064122070135220014504gustar00rootroot0000000000000052 comment=0022f7d8977dbf56137df0db977ebe627501522d pyqi-0.2.0/000077500000000000000000000000001220701352200124655ustar00rootroot00000000000000pyqi-0.2.0/COPYING.txt000066400000000000000000000040271220701352200143410ustar00rootroot00000000000000============================= The pyqi licensing terms ============================= pyqi is licensed under the terms of the Modified BSD License (also known as New or Revised BSD), as follows: Copyright (c) 2013, BiPy Development Team All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name BiPy nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE BIPY DEVELOPMENT TEAM BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #-----------------------------------------------------------------------------pyqi-0.2.0/ChangeLog.md000066400000000000000000000001101220701352200146260ustar00rootroot00000000000000pyqi 0.2.0 - (27 Aug 2013) ========================== * Initial release pyqi-0.2.0/README.md000066400000000000000000000023541220701352200137500ustar00rootroot00000000000000pyqi: expose your interface =========================== [![Build Status](http://ci.qiime.org/job/pyqi/badge/icon)](http://ci.qiime.org/job/pyqi/) pyqi (canonically pronounced *pie chee*) is designed to support wrapping general commands in multiple types of interfaces, including at the command line, HTML, and API levels. We're currently in the early stages of development, and there is a lot to be done. We're very interested in having beta users, and we fully embrace collaborative development, so if you're interested in using or developing pyqi, you should get in touch. For now, you can direct questions to gregcaporaso@gmail.com. Development is primarily occurring in the [Caporaso](www.caporaso.us) and [Knight](https://knightlab.colorado.edu/) labs (at Northern Arizona University and University of Colorado, respectively), but the goal is for this to be a very open development effort. We accept code submissions as [pull requests](https://help.github.com/articles/using-pull-requests). pyqi derives from code that was originally developed to support [QIIME](www.qiime.org)'s command line interface, but our interface needs for QIIME and other bioinformatics packages have expanded, and it now makes more sense that this be a stand-alone package. pyqi-0.2.0/doc/000077500000000000000000000000001220701352200132325ustar00rootroot00000000000000pyqi-0.2.0/doc/Makefile000066400000000000000000000126641220701352200147030ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/pyqi.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/pyqi.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/pyqi" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/pyqi" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." pyqi-0.2.0/doc/conf.py000066400000000000000000000172321220701352200145360ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # pyqi documentation build configuration file, created by # sphinx-quickstart on Mon Jun 17 14:01:20 2013. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'pyqi' copyright = u'2013, The BiPy Development Team' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '0.2' # The full version, including alpha/beta/rc tags. release = '0.2.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'haiku' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". html_title = "pyqi: expose your interface" # A shorter title for the navigation bar. Default is the same as html_title. html_short_title = "pyqi: expose your interface" # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. html_use_index = False # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. html_show_sourcelink = False # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'pyqidoc' # -- Options for LaTeX output -------------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'pyqi.tex', u'pyqi Documentation', u'The BiPy Development Team', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'pyqi', u'pyqi Documentation', [u'The BiPy Development Team'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------------ # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'pyqi', u'pyqi Documentation', u'The BiPy Development Team', 'pyqi', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' pyqi-0.2.0/doc/guidelines/000077500000000000000000000000001220701352200153625ustar00rootroot00000000000000pyqi-0.2.0/doc/guidelines/index.rst000066400000000000000000000001461220701352200172240ustar00rootroot00000000000000.. _guidelines-index: pyqi coding guidelines ====================== .. toctree:: :glob: * pyqi-0.2.0/doc/guidelines/optparse.rst000066400000000000000000000100541220701352200177510ustar00rootroot00000000000000.. _optparse-guidelines: Guidelines for creating pyqi-based optparse interfaces ========================================================== This document covers some general suggestions for ``OptparseInterfaces`` with pyqi. These ideas have evolved from our experiences with `QIIME `_. Ideally this document will continue to evolve with time, so if you have suggestions for things that should be included please consider submitting them as a `pull request `_. **Design convenient command line interfaces.** The goal of your interface is to make things easy for the user (who is often you). This section covers some guidelines for how to do that. **Have people who are better programmers than you interact with your command line interface and give you feedback on it.** If your script is difficult to work with, or has requirements that are not intuitive for users who frequently work with command line applications, people won't use your code. **If there are tasks that are automatable, automate them.** For example, if you can make a good guess at what an output file should be named from an input file and a parameter choice, do that and use it as the default output path (but allow the user to overwrite it with a command line option). **Define sensible default values for your command line options.** If most of the time that a script is used it will require a parameter to be set to a certain value, make that value the default to simplify the interface. **Have the user specify named options rather than positional arguments.** The latter are more difficult to work with as users need to remember the order that they need to be passed. pyqi scripts do not allow positional arguments. Note that the idea of avoiding positional arguments contradicts `what Python's optparse docs say `_. We disagree with their comment that all required options should be passed as positional arguments. **Avoid making assumptions about how a script will be run.** Perhaps most importantly, don't assume that the script will be run from the same directory that the script lives in. Users often want to copy executables into a centralized directory on their system (e.g., ``/usr/local/bin``). Facilitate that by not requiring that the script is run from a specific location. If you rely on data files, you have other options such as having users set an environment variable that defines where data files live on the system. Test your script from multiple locations on the file system! **Test calling your script in invalid ways to ensure that it provides informative error messages.** Python's traceback errors are generally not very informative for users, so you should test calling your scripts incorrectly to detect cases when the script might result in uninformative error messages. You should then catch these cases and provide useful error messages. **Define where output will be stored based on how many output files are created.** If a single file is created by your script, give the user control over the output filepath on the command line (usually with a parameter named ``-o`` or ``--output-fp``). If multiple files are created by your script, allow the user to define a directory name, and store all of the output files in that directory with names that you pre-define within your script. Define these options using the option types ``new_filepath`` and ``new_dirpath``, respectively. These will raise errors if the file or directory already exists, which is generally good as it avoids overwriting results that may have taken a long time to generate. We've found that this is the most convenient way for users to define where the output of scripts should live, but there are likely special cases where you'd want to deviate from this general guideline. Do *always* give the user control over the name of either the file or the directory where output will be created. You don't want them to have to move a file to a different location if they want to run your script multiple times. pyqi-0.2.0/doc/index.rst000066400000000000000000000074341220701352200151030ustar00rootroot00000000000000Introduction ============ What is pyqi? ------------- pyqi (canonically pronounced *pie chee*) is a Python framework designed to support wrapping general *commands* in multiple types of *interfaces*, including at the command line, HTML, and API levels. pyqi's only requirement is a working Python 2.7 installation. Why should I care? ------------------ pyqi allows you to write your command once, and easily make it accessible to different types of users through different types of interfaces. In the context of pyqi, **a command is a class that takes some inputs, performs some function, and produces some outputs**. An interface is a light wrapper around that command that makes it accessible to users. After defining and testing your command, you can **configure different types of interfaces**. This enables, for example, basic users to access your command through an **HTML interface** running on a local server, power or cluster users to access your command through a **command line interface**, and developers to access your command through an **application programmer interface (API)**. Because pyqi's interfaces are light wrappers around your underlying command, **users of each of these interfaces will be guaranteed to be accessing the same underlying functionality**. pyqi is currently in the early stages of development, and there is a lot to be done. We're very interested in having beta users, and we fully embrace collaborative development, so if you're interested in using or developing pyqi, you should get in touch. How do I start using pyqi? -------------------------- First, install pyqi (it's easy) by following our :ref:`install instructions `. Then, you can start working through our tutorials, which are designed to help you evaluate the utility of pyqi, and then integrate pyqi into your project. The :ref:`Getting Started ` series of tutorials progress linearly through :ref:`how to stub and build new commands ` and :ref:`how to stub and build new interfaces `. These will give you an idea of what pyqi is capable of and how it works. The :ref:`using-pyqi-in-your-project` series of tutorials will then give you an idea of how you could integrate pyqi into your project. This includes suggestions on :ref:`how to organize your project's repository to look like other repositories that make use of pyqi `, and :ref:`how to define a driver script ` (similar to the ``pyqi`` command which you'll become familiar with in the :ref:`Getting Started ` tutorials) that will give your users access to the commands in your project. As the pyqi project matures, we'll include additional :ref:`documentation for advanced developers `, who are interested in things like defining new interface types (though this is not something that is ever required for most developers). How do I get help with pyqi? ---------------------------- For now, please direct questions to gregcaporaso@gmail.com. Please report bugs and feature requests on the `pyqi issue tracker `_. .. _contributing-to-pyqi: Can I help develop pyqi? ------------------------ Yes! pyqi is open source software, available under the BSD license. All source code is hosted in the `pyqi GitHub repository `_. Development is primarily occurring in the `Caporaso Lab `_ (Northern Arizona University; Argonne National Laboratories) and `Knight Lab `_ (University of Colorado; Howard Hughes Medical Institute), but the goal is for pyqi to be a very open development effort. We accept code submissions as `pull requests `_. pyqi-0.2.0/doc/install/000077500000000000000000000000001220701352200147005ustar00rootroot00000000000000pyqi-0.2.0/doc/install/index.rst000066400000000000000000000046501220701352200165460ustar00rootroot00000000000000.. _install-index: Installing pyqi =============== pyqi has no dependencies outside of Python, so installing is easy. Installation of pyqi -------------------- There are two steps to installing pyqi: * First, decide if you want to work with the release or development version of pyqi. If you work with the release version, you'll have the most recent stable version of pyqi, but may not have access to the latest and greatest features. If you're most interested in having access to the latest features and can tolerate some instability, you should work with the development version of pyqi. If you're unsure about what you want here, you should likely work with the release version. * To use the release version of pyqi, you can download it from `here `_. The latest release is pyqi 0.2.0. After downloading, unzip the file with ``tar -xzf pyqi-0.2.0.tar.gz`` and change to the new ``pyqi-0.2.0`` directory. * To use the latest development version of pyqi you can download it from our `GitHub repository `_ using ``git clone git@github.com:bipy/pyqi.git``. After downloading, change to the new ``pyqi`` directory. * Next, run ``python setup.py install``. That's it! Enabling tab completion of pyqi commands and their command line options ----------------------------------------------------------------------- After installation, you can optionally enable bash completion for pyqi scripts, meaning that when you start typing the name of a command or an option, you can hit the tab key to complete it without typing the full name, if the name is unique. There are two steps in enabling tab completion. First, you'll need to generate the tab completion file, and then you'll need to edit your ``$HOME/.bash_profile`` file. To create the tab completion file for ``pyqi``, run the following commands:: mkdir ~/.bash_completion.d pyqi make-bash-completion --command-config-module pyqi.interfaces.optparse.config --driver-name pyqi -o ~/.bash_completion.d/pyqi Then, add the following lines to your ``$HOME/.bash_profile`` file:: # enable bash completion for pyqi-based scripts for f in ~/.bash_completion.d/*; do source $f; done When you open a new terminal, tab completion should work for the ``pyqi`` commands and their options. You can test this by typing ``pyqi make-c`` and then hitting the tab key (there should be no space after ``pyqi make-c``). pyqi-0.2.0/doc/make.bat000066400000000000000000000117441220701352200146460ustar00rootroot00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=_build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . set I18NSPHINXOPTS=%SPHINXOPTS% . if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\pyqi.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\pyqi.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) :end pyqi-0.2.0/doc/tutorials/000077500000000000000000000000001220701352200152605ustar00rootroot00000000000000pyqi-0.2.0/doc/tutorials/defining_new_commands.rst000066400000000000000000000231621220701352200223330ustar00rootroot00000000000000.. _defining-new-commands: Defining new commands ===================== A pyqi ``Command`` is a class that accepts inputs, does some work, and produces outputs. A ``Command`` is designed to be interface agnostic, so ideally should not be tied to a filesystem (i.e., it shouldn't do I/O or take filepaths) though there are some exceptions. Your ``Command`` class ultimately defines an API for your ``Command`` that can then easily be wrapped in other interface types (for example, a command line interface and/or a web interface) which handle input and output in an interface-specific way. This strategy also facilitates unit testing of your ``Command`` (by separating core functionality, which is essential to test, from interfaces, which can be very difficult to test in an automated fashion), parallel processing with your ``Command``, and constructing workflows that chain multiple ``Commands`` together. In general, your ``Command`` should take structured input (for example, a list of tuples or a numpy array), not a file that needs to be parsed. This document describes how to create your first ``pyqi`` ``Command``. Stubbing a new command ---------------------- After installing pyqi, you can easily stub (i.e., create templates for) new commands using ``pyqi make-command``. You can get usage information by calling:: pyqi make-command -h To create our sequence collection summarizer, we can start by stubbing a ``SequenceCollectionSummarizer`` class:: pyqi make-command -n SequenceCollectionSummarizer -a "Greg Caporaso" -c "Copyright 2013, Greg Caporaso" -e "gregcaporaso@gmail.com" -l BSD --command-version 0.0.1 -o sequence_collection_summarizer.py If you run this command locally, substituting your own name and email address where applicable, you'll have a new file called ``sequence_collection_summarizer.py``, which will look roughly like the following:: #!/usr/bin/env python from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, Greg Caporaso" __credits__ = ["Greg Caporaso"] __license__ = "BSD" __version__ = "0.0.1" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" from pyqi.core.command import Command, Parameter, ParameterCollection class SequenceCollectionSummarizer(Command): BriefDescription = "FILL IN A 1 SENTENCE DESCRIPTION" LongDescription = "GO INTO MORE DETAIL" Parameters = ParameterCollection([ Parameter(Name='foo', DataType=str, Description='some required parameter', Required=True), Parameter(Name='bar', DataType=int, Description='some optional parameter', Required=False, Default=1) ]) def run(self, **kwargs): # EXAMPLE: # return {'result_1': kwargs['foo'] * kwargs['bar'], # 'result_2': "Some output bits"} raise NotImplementedError("You must define this method") CommandConstructor = SequenceCollectionSummarizer Defining a command ------------------ There are several values that you'll need to fill in to define your command based on the stub that is created by ``make-command``. The first, which are the easiest, are ``BriefDescription`` and ``LongDescription``. ``BriefDescription`` should be a one sentence description of your command, and ``LongDescription`` should be a more detailed explanation (usually 2-3 sentences). These are used in auto-generated documentation. Next, you'll need to define the parameters that your new command can take. Each of these parameters will be an instance of the ``pyqi.core.command.Parameter`` class. Our ``SequenceCollectionSummarizer`` command will take one required parameter and one optional parameter. The required parameter will be called ``seqs``, and will be a list (or some other iterable type) of tuples of (sequence identifier, sequence) pairs. For example:: [('sequence1','ACCGTGGACCAA'),('sequence2','TGTGGA'), ...] We'll also need to provide a description of this parameter (used in documentation), its type, and indicate that it is required. The final Parameter definition should look like this:: Parameter(Name='seqs', DataType=list, Description='sequences to be summarized', Required=True) The optional parameter will be called ``suppress_length_summary``, and if passed will indicate that we don't want information on sequence lengths included in our output summary. The ``Parameter`` definition in this case should look like this:: Parameter(Name='suppress_length_summary', DataType=bool, Description='do not generate summary information on the sequence lengths', Required=False, Default=False) The only additional ``Parameter`` that is passed here, relative to our ``seqs`` parameter, is ``Default``. Because this parameter isn't required, it's necessary to give it a default value here. All of the ``Parameters`` should be included in a ``pyqi.core.command.ParameterCollection`` object (as in the stubbed file). .. note:: There are a few restrictions on what ``Name`` can be set to for a ``Parameter``. It must be a `valid python identifier `_ (e.g., it cannot contain ``-`` characters or begin with a number) so the ``Command`` can be called with named options instead of passing a dict. ``Parameter`` names also must be unique for a ``Command``. Next, we'll need to define what our ``Command`` will actually do. This is done in the ``run`` method, and all results are returned in a dictionary. The run method for our ``SequenceCollectionSummarizer`` object would look like the following:: def run(self, **kwargs): """ """ num_seqs = 0 sequence_lengths = [] for seq_id, seq in kwargs['seqs']: num_seqs += 1 sequence_lengths.append(len(seq)) if kwargs['suppress_length_summary']: min_length = None max_length = None else: min_length = min(sequence_lengths) max_length = max(sequence_lengths) return {'num-seqs':num_seqs, 'min-length':min_length, 'max-length':max_length} In practice, if your ``Command`` is more complex than our ``SequenceCollectionSummarizer`` (which it probably is), you can define other methods that are called by ``run``. These should likely be private methods. .. note:: ``kwargs`` is validated prior to ``run`` being called, so that any required ``kwargs`` that are missing will raise an error, and any optional ``kwargs`` that are missing will have their default values filled in. To customize the validation that is performed on ``kwargs`` for your ``Command`` you should override ``_validate_kwargs`` in your ``Command``. A complete example Command -------------------------- The following illustrates a complete python file defining a new pyqi ``Command``:: #!/usr/bin/env python from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, Greg Caporaso" __credits__ = ["Greg Caporaso"] __license__ = "BSD" __version__ = "0.0.1" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" from pyqi.core.command import Command, Parameter, ParameterCollection class SequenceCollectionSummarizer(Command): BriefDescription = "Generate summary statistics on a collection of sequences." LongDescription = "Provide the number of sequences, the minimum sequence length, and the maximum sequence length given a collection of sequences. Sequences should be provided as a list (or other iterable object) of tuples of (sequence id, sequence) pairs." Parameters = ParameterCollection([ Parameter(Name='seqs', DataType=list, Description='sequences to be summarized', Required=True), Parameter(Name='suppress_length_summary', DataType=bool, Description='do not generate summary information on the sequence lengths', Required=False,Default=False) ]) def run(self, **kwargs): """ """ num_seqs = 0 sequence_lengths = [] for seq_id, seq in kwargs['seqs']: num_seqs += 1 sequence_lengths.append(len(seq)) if kwargs['suppress_length_summary']: min_length = None max_length = None else: min_length = min(sequence_lengths) max_length = max(sequence_lengths) return {'num-seqs':num_seqs, 'min-length':min_length, 'max-length':max_length} CommandConstructor = SequenceCollectionSummarizer At this stage you have defined a new command and its API. To access the API in the python terminal, you could do the following:: # Import your new class >>> from sequence_collection_summarizer import SequenceCollectionSummarizer # Instantiate it >>> s = SequenceCollectionSummarizer() # Call the command, passing a list of (seq id, sequence) tuples as input. # Note that because the parameters are provided as kwargs, you need to # pass the parameter with a keyword. >>> r = s(seqs=[('sequence1','ACCGTGGACCAA'),('sequence2','TGTGGA')]) # You can now see the full output of the command by inspecting the # result dictionary. >>> r {'max-length': 12, 'min-length': 6, 'num-seqs': 2} # Alternatively, you can access each value independently, as with any dictionary. >>> print r['num-seqs'] 2 >>> print r['min-length'] 6 >>> print r['max-length'] 12 # You can call this command again with different input. # For example, we can call the command again passing the # suppress_length_summary parameter. >>> r = s(seqs=[('sequence1','ACCGTGGACCAA'),('sequence2','TGTGGA')],suppress_length_summary=True) >>> r {'max-length': None, 'min-length': None, 'num-seqs': 2} pyqi-0.2.0/doc/tutorials/defining_new_interfaces.rst000066400000000000000000000750501220701352200226600ustar00rootroot00000000000000.. _defining-new-interfaces: Defining new interfaces ======================= After defining a new ``Command`` and its API, as covered in :ref:`defining-new-commands`, you're ready to create a first user interface for that command. In this tutorial we'll define a command line interface for the ``SequenceCollectionSummarizer`` command. The main differences that need to be handled when defining a command line interface are that we'll want the user to provide their sequence collection on the command line, and we'll want to write the output to a filepath that the user specifies on the command line. This is different than what happens in ``SequenceCollectionSummarizer``, where the input and output are python objects. This is a very important distinction - since our derived ``Commands`` are meant to be interface-independent, they should not do things like require files as input. In some rare circumstances, it may be required for a ``Command`` to write files as its output (for example, if storing the output in memory is intractable). pyqi currently provides support for building command line interfaces based on python's `optparse `_ module. Your interface will ultimately be an instance of ``pyqi.interfaces.optparse.OptparseInterface``, but the interface class itself is generated dynamically. As a developer, you only define the configuration for the interface via an *interface configuration file* (which is a valid python file) - you won't actually define the interface class itself. If this sounds confusing, just get started - it's easier than it sounds. Stubbing a new command line interface ------------------------------------- pyqi provides a command, ``make-optparse``, that allows developers to easily stub (i.e., create templates for) their optparse interface configuration files. After installing pyqi, you can get usage information by calling:: pyqi make-optparse -h To create your interface, you'll need to pass the ``Command`` as a fully specified python module name, the name of the module where that ``Command`` is defined, ownership information (e.g., author name, copyright, license, etc.) and the path where the new configuration file should be written. For example, to create a stub for an ``OptparseInterface`` for our ``SequenceCollectionSummarizer`` command, you could run the following:: pyqi make-optparse -c sequence_collection_summarizer.SequenceCollectionSummarizer -m sequence_collection_summarizer -a "Greg Caporaso" --copyright "Copyright 2013, Greg Caporaso" -e "gregcaporaso@gmail.com" -l BSD --config-version 0.0.1 -o summarize_sequence_collection.py .. warning:: For the above command to work, the directory containing ``sequence_collection_summarizer.py`` will need to be in your ``$PYTHONPATH``. The resulting file will look something like this:: #!/usr/bin/env python from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, Greg Caporaso" __credits__ = ["Greg Caporaso"] __license__ = "BSD" __version__ = "0.0.1" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" from pyqi.core.interfaces.optparse import (OptparseUsageExample, OptparseOption, OptparseResult) from pyqi.core.command import make_parameter_collection_lookup_f from sequence_collection_summarizer import CommandConstructor # If you need access to input or output handlers provided by pyqi, consider # importing from the following modules: # pyqi.core.interfaces.optparse.input_handler # pyqi.core.interfaces.optparse.output_handler # pyqi.interfaces.optparse.input_handler # pyqi.interfaces.optparse.output_handler # Convenience function for looking up parameters by name. param_lookup = make_parameter_collection_lookup_f(CommandConstructor) # Examples of how the command can be used from the command line using an # optparse interface. usage_examples = [ OptparseUsageExample(ShortDesc="A short single sentence description of the example", LongDesc="A longer, more detailed description", Ex="%prog --foo --bar some_file") ] # inputs map command line arguments and values onto Parameters. It is possible # to define options here that do not exist as parameters, e.g., an output file. inputs = [ # An example option that has a direct relationship with a Parameter. # OptparseOption(Parameter=param_lookup('name_of_a_parameter'), # InputType='existing_filepath', # the optparse type of input # InputAction='store', # the optparse action # InputHandler=None, # Apply a function to the input value to convert it into the type expected by Parameter.DataType # ShortName='n', # a parameter short name, can be None # # Name='foo', # implied by Parameter.Name. Can be overwritten here if desired # # Required=False, # implied by Parameter.Required. Can be promoted by setting True # # Help='help', # implied by Parameter.Description. Can be overwritten here if desired # # Default=None, # implied by Parameter.Default. Can be overwritten here if desired # # DefaultDescription=None, # implied by Parameter.DefaultDescription. Can be overwritten here if desired # convert_to_dashed_name=True), # whether the Name (either implied by Parameter or defined above) should have underscores converted to dashes when displayed to the user # # An example option that does not have an associated Parameter. # OptparseOption(Parameter=None, # InputType='new_filepath', # InputAction='store', # InputHandler=None, # we don't need an InputHandler because this option isn't being converted into a format that a Parameter expects # ShortName='o', # Name='output-fp', # Required=True, # Help='output filepath') OptparseOption(Parameter=param_lookup('seqs'), InputType=, InputAction='store', # default is 'store', change if desired InputHandler=None, # must be defined if desired ShortName=None), # must be defined if desired # Name='seqs', # implied by Parameter # Required=True, # implied by Parameter # Help='sequences to be summarized', # implied by Parameter OptparseOption(Parameter=param_lookup('suppress_length_summary'), InputType=, InputAction='store', # default is 'store', change if desired InputHandler=None, # must be defined if desired ShortName=None), # must be defined if desired # Name='suppress_length_summary', # implied by Parameter # Required=False, # implied by Parameter # Help='do not generate summary information on the sequence lengths', # implied by Parameter # Default=False, # implied by Parameter # DefaultDescription=None, # implied by Parameter ] # outputs map result keys to output options and handlers. It is not necessary # to supply an associated option, but if you do, it must be an option from the # inputs list (above). outputs = [ # An example option that maps to a result key. # OptparseResult(ResultKey='some_result', # OutputHandler=write_string, # a function applied to the value at ResultKey # # # the name of the option (defined in inputs, above), whose # # value will be made available to OutputHandler. This name # # can be either an underscored or dashed version of the # # option name (e.g., 'output_fp' or 'output-fp') # OptionName='output-fp'), # # An example option that does not map to a result key. # OptparseResult(ResultKey='some_other_result', # OutputHandler=print_string) ] There are three lists of values that we'll need to populate here to define the optparse interface for our ``SequenceCollectionSummarizer`` command. These are the ``inputs``, the ``outputs``, and the ``usage_examples``. We'll also need to define an input handler and an output handler to tell the ``OptparseInterface`` how to take input from the command line and turn it into something that ``SequenceCollectionSummarizer`` can use, and to take output from ``SequenceCollectionSummarizer`` and turn it into something a command line user will want. ``make-optparse`` will auto-populate the ``inputs`` based on the ``Parameters``, but some changes will usually be required (detailed below). The following sections describe each of these steps. .. note:: There is a fourth value that is required when defining an optparse interface, which is the version string of the command/interface (e.g., ``0.0.1``). This value has already been filled in for us in the configuration file template (see ``__version__`` at the top of the file). You can specify the version string when creating the configuration file template via ``--config-version``. In the example above, we specified a version string of ``0.0.1``. Defining usage examples ----------------------- The first thing to do when defining the ``OptparseInterface`` for our ``SequenceCollectionSummarizer`` command is define a set of usage examples. While in practice this documentation step may seem like something you'd want to do last, it's really helpful to do first to get you thinking about how you'd like to interact with your command from the command line. Usage examples are defined as instances of the ``pyqi.interface.optparse.UsageExample`` class, and are instantiated with three parameters: ``ShortDescription``, ``LongDescription``, and ``Ex``. ``Ex`` is the usage example itself, ``ShortDescription`` is a one sentence description of what ``Ex`` will do, and ``LongDescription`` elaborates on what ``Ex`` does. Find the ``usage_examples`` list in your new ``summarize_sequence_collection.py`` file, and replace its definition with:: usage_examples = [ OptparseUsageExample(ShortDesc="Summarize the input sequence collection and write the result to file.", LongDesc="Read the file specified by -i, and compute the number of sequences in the file, and the minimum and maximum sequence lengths. Write all of that information to path specified by -o.", Ex="%prog -i seqs.fna -o seqs.summary.txt"), OptparseUsageExample(ShortDesc="Summarize the input sequence collection and write the result to file, excluding information on sequence lengths.", LongDesc="Read the file specified by -i, compute the number of sequences in the file, and write that information to path specified by -o.", Ex="%prog -i seqs.fna -o seqs.summary.txt --suppress-length-summary") ] Here we define two usage examples, each of which gives us an idea about how we want our script to behave: we want it to take an ``i`` parameter (where the user passes their input file name), an ``o`` parameter (where the user passes their output file name), and an optional parameter called ``suppress-length-summary`` which controls some of the script behavior. .. warning:: You shouldn't ever include the name of the script when defining ``UsageExample.Ex``, but instead include the text ``%prog``. This will be automatically replaced with the script name, so if you ever change the name of the script in the future, the change will take effect in all of your usage examples without you having to remember to update them. Defining inputs --------------- Next we'll define the list of ``inputs`` that should be associated with our ``OptparseInterface``. Each of these inputs will be an instance of a ``pyqi.core.interface.optparse.OptparseOption`` object. These will roughly map on to the ``Parameters`` that we defined for ``SequenceCollectionSummarizer``, but there are usually additional interface options relative to command parameters, as we'll see here. For the ``OptparseOptions`` that map onto ``Parameters`` directly, you can look up the corresponding ``Parameter`` in the ``param_lookup`` dictionary (which is created for you by ``make-optparse``), and most of the information in the ``OptparseOption`` will be auto-populated for you. ``make-optparse`` will actually fill in as much information as possible for each ``OptparseOption`` that corresponds to an existing ``Parameter``. In our example, you'll notice that there are two ``OptparseOptions`` that are already defined. There are a few values that may need to be changed here. In almost all cases, you'll need to change the ``InputType``, which is set to the ``Parameter``' ``DataType`` value by default, but should be updated to the ``optparse`` type. You can find discussion of these types in the :ref:`optparse type definitions ` section. Note that the ``InputType`` should be ``None`` for command line flags, as the type describes the value that is passed via that option, and command line flags don't take a value. The other value that often will need to be changed is ``InputHandler``, which tells ``OptparseInterface`` how to transform the ``OptparseOption`` into the corresponding ``Parameter``. In our case, for our ``seqs`` ``OptparseOption``, that involves converting a filepath into a list of tuples of (sequence id, sequence) pairs. First let's define the ``OptparseOptions``, and then we'll define a new ``InputHandler``. The ``OptparseOptions`` corresponding to the existing ``Parameters`` should look like this:: inputs = [ OptparseOption(Parameter=param_lookup('seqs'), InputType='existing_filepath', InputAction='store', InputHandler=parse_fasta, ShortName='i'), OptparseOption(Parameter=param_lookup('suppress_length_summary'), InputType=None, InputAction='store_true', InputHandler=None, ShortName=None), ] These definitions are exactly as generated by ``make-optparse``, except that many of the comments have been removed, and we've modified the ``InputTypes`` and the ``InputHandler`` for our ``seqs`` option. In the :ref:`next section ` we'll define this new ``parse_fasta`` input handler, but first we'll add one more OptparseOption which is specific to our command line interface. The output from our ``SequenceCollectionSummarizer`` is a dictionary, where some of the values are integers and some of the values may be ``None``. Generally a command line user will want to have information printed to stdout or to file. We'll define our interface so that the output is written to file with some basic formatting put in place. To do this, we need to define a new OptparseOption to allow the user to specify the path where output should be written. This ``OptparseOption`` does not map onto one of our existing ``Parameters``, and should be defined as follows:: OptparseOption(Parameter=None, InputType='new_filepath', InputAction='store', ShortName='o', Name='output-fp', Required=True, Help='path where output should be written') Notice the ``Parameter=None`` parameter here: this indicates that this ``OptparseOption`` does not correspond to one of the ``SequenceCollectionSummarizer`` parameters. You should include this ``OptparseOption`` definition in the ``inputs`` list to define the three options for our command line interface. .. _defining-input-handlers: Defining input handlers ----------------------- Input handlers tell the ``OptparseInterface`` class how to take input from the command line and get it into the form that the ``Command`` is expecting. In our case, the user will be providing a filepath on the command line, and our ``SequenceCollectionSummarizer`` expects to receive a list (or other iterable object) of tuples of (sequence id, sequence) pairs. Our input handler is therefore a simple fasta parser, which is a `generator `_ of (sequence id, sequence) tuples. We can define this as follows:: def parse_fasta(fp): """ fp: path to a fasta-formatted file This function is a fasta record generator, yielding (sequence id, sequence) pairs when provided with a valid fasta file. NO ERROR CHECKING IS PERFORMED! """ # Always open files for reading in python using mode 'U' # to correctly handle different types of line breaks f = open(fp,'U') seq_id = None seq = [] for line in f: line = line.strip() if line.startswith('>'): if len(seq) != 0: # we've completed a fasta record yield seq_id, ''.join(seq) seq_id = line[1:] seq = [] else: seq.append(line) yield seq_id, ''.join(seq) f.close() This definition can go in the interface configuration file that we've been working on in this tutorial. Alternatively, if your input handler is generally useful for your project you can centralize it within your project (see :ref:`organizing-your-repository`), or if you think it's generally useful for pyqi users, you should consider submitting it to the pyqi project :ref:`contributing it to pyqi `. Defining outputs ---------------- The last thing we need to do is define which of the outputs generated by ``SequenceCollectionSummarizer`` are things we care about with this interface, and tell our ``OptparseInterface`` how to handle those. We do this by defining the ``outputs`` list of ``pyqi.core.interfaces.optparse.OptparseResult`` objects. In our case, we'll want to write all of the values that are not ``None`` to the filepath specified by the user with ``output-fp``. To do that, we need to handle three possible outputs, so we'll define those outputs and write an output handler. You should start with the stubbed ``outputs`` list to define how you want to handle each of the parameters. We'll do this as follows:: outputs = [ OptparseResult(ResultKey='num-seqs', OutputHandler=append_datum_to_file, OptionName='output-fp'), OptparseResult(ResultKey='min-length', OutputHandler=append_datum_to_file, OptionName='output-fp'), OptparseResult(ResultKey='max-length', OutputHandler=append_datum_to_file, OptionName='output-fp'), ] In this case, each of our ``OptparseResults`` are associated with a single ``OptionName``: ``output-fp``. We do this because each of these should be written to the same file, but in practice each of these could be associated with different ``OptionNames`` (e.g., if each should be written to a different file), or ``OptionName=None``, if (for example) a particular result should be written to stdout or stderr. We'll next define the new output handler, ``append_datum_to_file``, used by each of these ``OptparseResult`` objects. Defining output handlers ------------------------ Each of these ``OptparseResult`` objects uses the same ``OutputHandler``, which we need to define now. This will take the result and write it to the file specified by the user as ``output-fp``. This should look like the following:: def append_datum_to_file(result_key, data, option_value=None): """Append summary information to a file. """ # don't do anything if data is None if data is None: return # If option_value is None when this output handler is called, # the interface developer did something wrong when defining # the OptparseResults. Politely alert the developer that # this output handler isn't associated with an option # (it needs to be associated with an output filepath). if option_value is None: raise IncompetentDeveloperError( "Cannot write output without a filepath.") # open the output file for appending, and write the # summary information to a single tab-separated line with open(option_value, 'a') as f: f.write('%s\t%d\n' % (result_key, data)) Complete OptparseInterface configuration file --------------------------------------------- At this stage we've fully configured our interface. The final interface configuration file should look like this:: #!/usr/bin/env python from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, Greg Caporaso" __credits__ = ["Greg Caporaso"] __license__ = "BSD" __version__ = "0.0.1" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" from pyqi.core.interfaces.optparse import (OptparseUsageExample, OptparseOption, OptparseResult) from pyqi.core.command import make_parameter_collection_lookup_f from sequence_collection_summarizer import CommandConstructor from pyqi.core.exception import IncompetentDeveloperError import os param_lookup = make_parameter_collection_lookup_f(CommandConstructor) def parse_fasta(fp): """ fp: path to a fasta-formatted file This function is a fasta record generator, yielding (sequence id, sequence) pairs when provided with a valid fasta file. NO ERROR CHECKING IS PERFORMED! """ # Always open files for reading in python using mode 'U' # to correctly handle different types of line breaks f = open(fp,'U') seq_id = None seq = [] for line in f: line = line.strip() if line.startswith('>'): if len(seq) != 0: # we've completed a fasta record yield seq_id, ''.join(seq) seq_id = line[1:] seq = [] else: seq.append(line) yield seq_id, ''.join(seq) def append_datum_to_file(result_key, data, option_value=None): """Append summary information to a file. """ # don't do anything if data is None if data is None: return # If option_value is None when this output handler is called, # the interface developer did something wrong when defining # the OptparseResults. Politely alert the developer that # this output handler isn't associated with an option # (it needs to be associated with an output filepath). if option_value is None: raise IncompetentDeveloperError( "Cannot write output without a filepath.") # open the output file for appending, and write the # summary information to a single tab-separated line with open(option_value, 'a') as f: f.write('%s\t%d\n' % (result_key, data)) usage_examples = [ OptparseUsageExample(ShortDesc="Summarize the input sequence collection and write the result to file.", LongDesc="Read the file specified by -i, and compute the number of sequences in the file, and the minimum and maximum sequence lengths. Write all of that information to path specified by -o.", Ex="%prog -i seqs.fna -o seqs.summary.txt"), OptparseUsageExample(ShortDesc="Summarize the input sequence collection and write the result to file, excluding information on sequence lengths.", LongDesc="Read the file specified by -i, compute the number of sequences in the file, and write that information to path specified by -o.", Ex="%prog -i seqs.fna -o seqs.summary.txt --suppress-length-summary") ] inputs = [ OptparseOption(Parameter=param_lookup('seqs'), InputType='existing_filepath', InputAction='store', InputHandler=parse_fasta, ShortName='i'), OptparseOption(Parameter=param_lookup('suppress_length_summary'), InputType=None, InputAction='store_true', InputHandler=None, ShortName=None), OptparseOption(Parameter=None, InputType='new_filepath', InputAction='store', ShortName='o', Name='output-fp', Required=True, Help='path where output should be written') ] outputs = [ OptparseResult(ResultKey='num-seqs', OutputHandler=append_datum_to_file, OptionName='output-fp'), OptparseResult(ResultKey='min-length', OutputHandler=append_datum_to_file, OptionName='output-fp'), OptparseResult(ResultKey='max-length', OutputHandler=append_datum_to_file, OptionName='output-fp'), ] .. _running-our-command: Running our Command via its OptparseInterface --------------------------------------------- To run this, there are a couple of additional things you need to do. First, you need to confirm that the directory where you've written these files is accessible via your ``PYTHONPATH``. For example, if you've been working in ``$HOME/code/pyqi_experiments/``, you should have ``$HOME/code/`` in your ``PYTHONPATH``. You can add that as follows:: export PYTHONPATH=$HOME/code/:$PYTHONPATH Next, so you can import from that directory, it'll need to contain an ``__init__.py`` file. That file can be empty, but it does need to exist. You can do this as follows:: touch $HOME/code/pyqi_experiments/__init__.py Now we're ready to run our ``Command`` via its ``OptparseInterface``. You can do this as follows:: pyqi --command-config-module pyqi_experiments -- summarize-sequence-collection -h This will print the help text associated with the ``summarize_sequence_collection`` ``OptparseInterface`` configuration file that we just created. .. note:: The ``pyqi`` driver that we used above recognizes command names that match an ``OptparseInterface`` configuration file in the ``--command-config-module`` directory, minus the ``.py``. For example, we created a ``summarize_sequence_collection.py`` configuration file in the ``pyqi_experiments`` directory, so the ``pyqi`` driver recognizes the ``summarize_sequence_collection`` command. It also recognizes the dashed version of a command name, such as ``summarize-sequence-collection``. These names both map to the same command. You can test the command by applying it to some sequence collection as follows:: pyqi --command-config-module pyqi_experiments -- summarize-sequence-collection -i seqs.fna -o seqs.summary.txt If ``seqs.fna`` contains the following:: >s1 ACCTTTAACC >s2 CCGG >s3 AAAAAAAAAAAAAAAAAAAAAAAAAAA The resulting ``seqs.summary.txt`` should contain the following lines:: num-seqs 3 min-length 4 max-length 27 Calling your command via the pyqi driver itself, as we're doing here, is a little clunky. Creating a project-specific driver however is very simple (it's a two-line shell script) and is covered in :ref:`defining-your-command-driver`. .. _optparse-types: OptparseOption Types -------------------- pyqi defines several new option types in addition to the optparse's built-in option types. All of the available option types are: +------------------------------+------------------------------------------------------------+ | option type | brief description | +==============================+============================================================+ | string | a string | +------------------------------+------------------------------------------------------------+ | int | an int | +------------------------------+------------------------------------------------------------+ | long | a long | +------------------------------+------------------------------------------------------------+ | float | a float | +------------------------------+------------------------------------------------------------+ | complex | a complex number | +------------------------------+------------------------------------------------------------+ | choice | one value from a list of choices | +------------------------------+------------------------------------------------------------+ | existing_path | path to an existing file or directory | +------------------------------+------------------------------------------------------------+ | new_path | path to a new file or directory | +------------------------------+------------------------------------------------------------+ | existing_filepath | path to an existing file | +------------------------------+------------------------------------------------------------+ | existing_filepaths | path to one or more existing files | +------------------------------+------------------------------------------------------------+ | new_filepath | path to a new file | +------------------------------+------------------------------------------------------------+ | existing_dirpath | path to an existing directory | +------------------------------+------------------------------------------------------------+ | new_dirpath | path to a new directory | +------------------------------+------------------------------------------------------------+ | multiple_choice | one or more values from a list of choices | +------------------------------+------------------------------------------------------------+ | blast_db | a blast database | +------------------------------+------------------------------------------------------------+ Thoughts and guidelines on designing command line interfaces ------------------------------------------------------------ Based on our experiences developing command line interfaces for `QIIME `_, we've compiled some thoughts on best practices, which you can find in :ref:`optparse-guidelines`. pyqi-0.2.0/doc/tutorials/defining_your_command_driver.rst000066400000000000000000000070571220701352200237350ustar00rootroot00000000000000.. _defining-your-command-driver: Defining your command driver ============================ It's possible to run your ``OptparseInterfaces`` using the ``pyqi`` command, as illustrated in :ref:`running-our-command`, but that mechanism is clunky and not how you'd want your users to interact with your software. To handle this more gracefully, you can create a shell script that can be distributed with your package and used as the primary driver for all ``OptparseInterfaces``. Creating the driver shell script -------------------------------- To define a driver command for your project, create a new file named as you'd like your users to access your code. For example, the driver for the ``biom-format`` package is called ``biom``, and the driver for the ``pyqi`` package is called ``pyqi``. In this example our driver name will be ``my-project``. Add the following two lines to that file, replacing ``my-project`` with your driver name:: #!/bin/sh exec pyqi --driver-name my-project --command-config-module my_project.interfaces.optparse.config -- "$@" The value passed with ``--command-config-module`` must be the directory where the ``OptparseInterface`` configuration files can be found. If you followed the suggestions in :ref:`organizing-your-repository` the above should work. The driver script should then be made executable with:: chmod +x my-project You'll next need to ensure that the directory containing this driver file is in your ``PATH`` environment variable. Again, if you followed the recommendations in :ref:`organizing-your-repository` and if your project directory is under ``$HOME/code``, you can do this by running:: export PATH=$HOME/code/my-project/scripts/:$PATH You should now be able to run:: my-project This will print a list of the commands that are available via the driver script, which will be all of the ``Commands`` for which you've defined ``OptparseInterfaces``. If one of these commands is called ``my-command``, you can now run it as follows to get the help text associated with that command:: my-project my-command -h The command names that you pass to the driver (``my-command``, in this example) match the name of the ``OptparseInterface`` config file, minus the ``.py``. The driver also matches the dashed version of a command name, so ``my-command`` and ``my_command`` both map to the same command. Configuring bash completion --------------------------- One very useful feature for your driver script is to enable tab-completion of commands and command line options (meaning that when a user starts typing the name of a command or an option, they can hit the tab key to complete it without typing the full name, if the name is unique). pyqi facilitates this with the ``pyqi make-bash-completion`` command. There are two steps in enabling tab completion. First, you'll need to generate the tab completion file, and then you'll need to edit your ``$HOME/.bash_profile`` file. To create the tab completion file for ``my-project``, run the following commands (again, this is assuming that your ``OptparseInterface`` config files are located as described in :ref:`organizing-your-repository`):: mkdir ~/.bash_completion.d pyqi make-bash-completion --command-config-module my_project.interfaces.optparse.config --driver-name my-project -o ~/.bash_completion.d/my-project Then, add the following lines to your ``$HOME/.bash_profile`` file:: # enable bash completion for pyqi-based scripts for f in ~/.bash_completion.d/*; do source $f; done When you open a new terminal, tab completion should work for the ``my-project`` commands and their options. pyqi-0.2.0/doc/tutorials/index.rst000066400000000000000000000054461220701352200171320ustar00rootroot00000000000000.. _tutorial_index: .. pyqi tutorials .. index:: Tutorials ================== pyqi tutorials ================== Installing pyqi --------------- Before working through the tutorials below you'll need to have a working installation of pyqi (it's really easy to install). See our :ref:`install instructions ` before you get started. .. _getting-started: Getting started: defining new commands and interfaces using pyqi ---------------------------------------------------------------- This section of the documentation covers how to define a new ``Command``, its API, and its command line interface. You should work through these documents in order. As an example, we'll define a new ``Command`` that provides a summary of a collection of biological sequences. (*Biological sequences*, in this context, are DNA sequences. These are `canonically represented `_ as strings of primarily ``A``, ``C``, ``G``, and ``T`` characters, and `fasta format `_ is the most common file format for storing biological sequences on disk.) We'll be able to pass the sequences to the ``Command``, and the result from the ``Command`` will be the number of sequences in the collection, the minimum sequence length, and the maximum sequence length. We'll then wrap that ``Command`` in an ``OptparseInterface``, which will allow users to access it from the command line providing a fasta file as input and having a summary written to file as output. .. toctree:: :maxdepth: 2 defining_new_commands.rst defining_new_interfaces.rst .. _using-pyqi-in-your-project: Using pyqi in your project -------------------------- After you've experimented with defining a toy pyqi ``Command``, its API, and an ``OptparseInterface``, you are ready to start thinking about integrating pyqi into your project. This section of the documentation begins by providing suggestions for organizing your code to match best with the organization of pyqi and projects that use pyqi. We then cover how to define a command line driver (similar to the ``pyqi`` command) that can be used with your optparse interfaces, to customize how your users will interact with your project. .. toctree:: :maxdepth: 2 organizing_your_repository.rst defining_your_command_driver.rst .. _advanced-topics: Advanced topics --------------- As pyqi matures we'll include tutorials covering topics such as how to define new interface types. However, given the early state of development that we're currently in, these will likely change a lot, so we recommend that if you are interested in developing new interface types now, that you get in touch (you can e-mail gregcaporaso@gmail.com for now) to discuss what you'd like to do, and possibly get involved with development of pyqi. pyqi-0.2.0/doc/tutorials/organizing_your_repository.rst000066400000000000000000000150341220701352200235410ustar00rootroot00000000000000.. _organizing-your-repository: Organizing your repository ========================== This document covers suggestions for organizing your repository to align with how pyqi and projects that use pyqi are organized. Following these guidelines is not a requirement, but may simplify using pyqi in your project. Version 1.1.3 of the `biom-format `_ project was the first project to use pyqi for its commands and interfaces, so the structure of that repository is used as an example here. Structure of the biom-format project ------------------------------------ This directory tree (created with the unix ``tree`` command) illustrates the structure of the biom-format repository (some files that are not relevant to this discussion have been omitted to keep this as simple as possible). Annotations have been added following ``##`` to reference specific directories. :: biom-format/ ├── ChangeLog ├── COPYING ├── doc ├── examples ├── images ├── INSTALL ├── python-code │   ├── biom ## Library code directory │   │   ├── biomdb.py │   │   ├── commands ## derived Command classes │   │   │   ├── __init__.py │   │   │   ├── installation_informer.py │   │   │   ├── metadata_adder.py │   │   │   ├── table_subsetter.py │   │   │   ├── table_summarizer.py │   │   │   └── table_validator.py │   │   ├── csmat.py │   │   ├── dbdata.py │   │   ├── exception.py │   │   ├── __init__.py │   │   ├── interfaces ## per-interface-type directories │   │   │   ├── __init__.py │   │   │   └── optparse ## derived OptparseInterface classes │   │   │   ├── config │   │   │   │   ├── add_metadata.py │   │   │   │   ├── __init__.py │   │   │   │   ├── show_install_info.py │   │   │   │   ├── subset_table.py │   │   │   │   ├── summarize_table.py │   │   │   │   └── validate_table.py │   │   │   ├── __init__.py │   │   │   ├── input_handler.py ## general purpose OptparseInterface input handler functions │   │   │   └── output_handler.py ## general purpose OptparseInterface output handler functions │   │   ├── parse.py │   │   ├── sparsedict.py │   │   ├── sparsemat.py │   │   ├── table.py │   │   ├── unit_test.py │   │   └── util.py │   ├── support-code │   └── tests ## Test code directory │   ├── bench │   ├── __init__.py │   ├── test_biomdb.py │   ├── test_commands # unit tests of the derived Command classes │   │   ├── __init__.py │   │   ├── test_installation_informer.py │   │   ├── test_metadata_adder.py │   │   ├── test_table_subsetter.py │   │   ├── test_table_summarizer.py │   │   └── test_table_validator.py │   ├── test_csmat.py │   ├── test_dbdata.py │   ├── test_interfaces # unit tests of input and output handlers │   │   ├── __init__.py │   │   ├── test_optparse │   │   │   ├── __init__.py │   │   │   ├── test_input_handler.py ## tests of OptparseInterface input handler functions │   │   │   └── test_output_handler.py ## tests of OptparseInterface output handler functions │   ├── test_parse.py │   ├── test_sparsedict.py │   ├── test_sparsemat.py │   ├── test_table.py │   ├── test_unit_test.py │   └── test_util.py ├── R-code ├── README.md ├── scripts │   └── biom ## biom command driver ├── setup.py └── support_files Discussion of the biom-format directory structure ------------------------------------------------- Under the ``biom`` *library code directory*, there are two directories that house pyqi-related code. The first is ``commands``, which contains derivations of the ``Command`` class (see :ref:`defining-new-commands` for discussion of these files). All of the ``biom-format`` commands are therefore defined in this directory. The second is ``interfaces``, which contains all of the derivations of the pyqi ``Interface`` class, and which are nested based on interface type. Currently ``biom-format`` only implements ``OptparseInterface`` classes, so there is only an ``optparse`` directory, but by nesting these on an per-interface-type basis we avoid name conflicts if multiple interfaces types were defined. Under the ``biom/interfaces/optparse`` directory, there is a ``config`` directory which contains all of the config files (see :ref:`defining-new-interfaces` for discussion of these files). There are also top-level ``input_handler.py`` and ``output_handler.py`` files. These files contain general purpose input and output handlers that may be used in multiple ``OptparseInterfaces``. Since input and output handlers are interface specific, it makes sense for these files to be contained under the ``biom/interfaces/optparse`` directory. Under the ``tests`` directory there are subdirectories for ``test_commands`` and ``test_interfaces``. The ``test_commands`` directory should contain a file corresponding to each file in the ``biom/commands`` directory, and should provide extensive unit testing of each of your commands. The ``test_interfaces`` directory is more minimal as typically there is not any functionality in the interfaces (the files are just providing configuration details). The exception is the input and output handlers, so there are test files corresponding to the files where those are defined. Note that the nesting of all test files matches the nesting in the library code directory. Finally, under the ``scripts`` directory there is a single executable, ``biom``, which is the ``OptparseInterface`` command driver. This is a simple shell script that allows users to access the ``OptparseInterfaces`` defined in the ``biom-format`` project. Defining this script for your project is covered in :ref:`defining-your-command-driver`. pyqi-0.2.0/pyqi/000077500000000000000000000000001220701352200134475ustar00rootroot00000000000000pyqi-0.2.0/pyqi/__init__.py000066400000000000000000000012471220701352200155640ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Rob Knight", "Greg Caporaso", "Daniel McDonald", "Jai Ram Rideout", "Doug Wendel"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" pyqi-0.2.0/pyqi/commands/000077500000000000000000000000001220701352200152505ustar00rootroot00000000000000pyqi-0.2.0/pyqi/commands/__init__.py000066400000000000000000000012341220701352200173610ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Jai Ram Rideout", "Greg Caporaso", "Doug Wendel"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" pyqi-0.2.0/pyqi/commands/code_header_generator.py000066400000000000000000000055551220701352200221240ustar00rootroot00000000000000#!/usr/bin/env python from __future__ import division #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Jai Ram Rideout" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Jai Ram Rideout", "Daniel McDonald"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Jai Ram Rideout" __email__ = "jai.rideout@gmail.com" from pyqi.core.command import Command, Parameter, ParameterCollection header_format = """#!/usr/bin/env python from __future__ import division __author__ = "%(author)s" __copyright__ = "%(copyright)s" __credits__ = [%(credits)s] __license__ = "%(license)s" __version__ = "%(version)s" __maintainer__ = "%(author)s" __email__ = "%(email)s" """ class CodeHeaderGenerator(Command): BriefDescription = "Generate header code for use in a Python file" LongDescription = ("Generate valid Python code containing header " "information, such as author, email address, " "maintainer, version, etc.. This code can be placed at " "the top of a Python file.") Parameters = ParameterCollection([ Parameter(Name='author', DataType=str, Description='author/maintainer name', Required=True), Parameter(Name='email', DataType=str, Description='maintainer email address', Required=True), Parameter(Name='license', DataType=str, Description='license (e.g., BSD)', Required=True), Parameter(Name='copyright', DataType=str, Description='copyright (e.g., Copyright 2013, The pyqi ' 'project)', Required=True), Parameter(Name='version', DataType=str, Description='version (e.g., 0.1)', Required=True), Parameter(Name='credits', DataType=list, Description='list of other authors', Required=False, Default=None) ]) def run(self, **kwargs): # Build a string formatting dictionary for the file header. head = {} head['author'] = kwargs['author'] head['email'] = kwargs['email'] head['license'] = kwargs['license'] head['copyright'] = kwargs['copyright'] head['version'] = kwargs['version'] # Credits always includes author. credits = [head['author']] if kwargs['credits']: credits.extend(kwargs['credits']) f = lambda x: '"%s"' % x head['credits'] = ', '.join(map(f, credits)) return {'result': (header_format % head).split('\n')} CommandConstructor = CodeHeaderGenerator pyqi-0.2.0/pyqi/commands/make_bash_completion.py000066400000000000000000000062251220701352200217720ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Jai Ram Rideout", "Doug Wendel", "Greg Caporaso"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" import importlib from pyqi.core.command import Command, Parameter, ParameterCollection from pyqi.core.interface import get_command_names, get_command_config def _get_cfg_module(desc): """Load a module""" mod = importlib.import_module(desc) return mod # Based on http://stackoverflow.com/questions/5302650/multi-level-bash-completion script_fmt = """_%(driver)s_complete() { local cur prev COMPREPLY=() cur=${COMP_WORDS[COMP_CWORD]} prev=${COMP_WORDS[COMP_CWORD-1]} if [ $COMP_CWORD -gt 1 ]; then prev=${COMP_WORDS[1]} fi if [ $COMP_CWORD -eq 1 ]; then COMPREPLY=( $(compgen -W "%(command_list)s" -- $cur) ) elif [ $COMP_CWORD -gt 1 ]; then case "$prev" in %(commands)s *) ;; esac fi return 0 } && complete -F _%(driver)s_complete -f %(driver)s """ command_fmt = """ "%(command)s") COMPREPLY=( $(compgen -W "%(options)s" -- $cur) ) ;; """ class BashCompletion(Command): BriefDescription = "Construct a bash completion script" LongDescription = """Construct a bash tab completion script that will search through available commands and options""" Parameters = ParameterCollection([ Parameter(Name='command_config_module', DataType=str, Description="CLI command configuration module", Required=True), Parameter(Name='driver_name', DataType=str, Description="name of the driver script", Required=True) ]) def run(self, **kwargs): driver = kwargs['driver_name'] cfg_mod_path = kwargs['command_config_module'] cfg_mod = _get_cfg_module(cfg_mod_path) command_names = get_command_names(cfg_mod_path) command_list = ' '.join(command_names) commands = [] for cmd in command_names: cmd_cfg, _ = get_command_config(cfg_mod_path, cmd, exit_on_failure=False) if cmd_cfg is not None: command_options = [] command_options.extend( sorted(['--%s' % p.Name for p in cmd_cfg.inputs])) opts = ' '.join(command_options) commands.append(command_fmt % {'command':cmd, 'options':opts}) all_commands = ''.join(commands) return {'result':script_fmt % {'driver':driver, 'commands':all_commands, 'command_list':command_list}} CommandConstructor = BashCompletion pyqi-0.2.0/pyqi/commands/make_command.py000066400000000000000000000064151220701352200202430ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from pyqi.core.command import Command, Parameter, ParameterCollection from pyqi.commands.code_header_generator import CodeHeaderGenerator command_imports = """from pyqi.core.command import Command, Parameter, ParameterCollection""" command_format = """class %s(Command): BriefDescription = "FILL IN A 1 SENTENCE DESCRIPTION" LongDescription = "GO INTO MORE DETAIL" Parameters = ParameterCollection([ Parameter(Name='foo', DataType=str, Description='some required parameter', Required=True), Parameter(Name='bar', DataType=int, Description='some optional parameter', Required=False, Default=1) ]) def run(self, **kwargs): # EXAMPLE: # return {'result_1': kwargs['foo'] * kwargs['bar'], # 'result_2': "Some output bits"} raise NotImplementedError("You must define this method") CommandConstructor = %s""" test_format = """from unittest import TestCase, main from FILL IN MODULE PATH import %(name)s class %(name)sTests(TestCase): def setUp(self): self.cmd_obj = %(name)s() def test_run(self): self.fail() if __name__ == '__main__': main()""" class MakeCommand(CodeHeaderGenerator): BriefDescription = "Construct a stubbed out Command object" LongDescription = """This command is intended to construct the basics of a Command object so that a developer can dive straight into the implementation of the command""" Parameters = ParameterCollection( CodeHeaderGenerator.Parameters.Parameters + [ Parameter(Name='name', DataType=str, Description='the name of the Command', Required=True), Parameter(Name='test_code', DataType=bool, Description='create stubbed out unit test code', Required=False, Default=False) ] ) def run(self, **kwargs): code_header_lines = super(MakeCommand, self).run( author=kwargs['author'], email=kwargs['email'], license=kwargs['license'], copyright=kwargs['copyright'], version=kwargs['version'], credits=kwargs['credits'])['result'] result_lines = code_header_lines if kwargs['test_code']: result_lines.extend( (test_format % {'name': kwargs['name']}).split('\n')) else: result_lines.extend(command_imports.split('\n')) result_lines.append('') result_lines.extend((command_format % ( kwargs['name'], kwargs['name'])).split('\n')) return {'result': result_lines} CommandConstructor = MakeCommand pyqi-0.2.0/pyqi/commands/make_optparse.py000066400000000000000000000172651220701352200204670ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division from operator import attrgetter from pyqi.core.command import Command, Parameter, ParameterCollection from pyqi.commands.code_header_generator import CodeHeaderGenerator __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Jai Ram Rideout", "Greg Caporaso", "Doug Wendel"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" header_format = """from pyqi.core.interfaces.optparse import (OptparseUsageExample, OptparseOption, OptparseResult) from pyqi.core.command import make_parameter_collection_lookup_f from %(command_module)s import CommandConstructor # If you need access to input or output handlers provided by pyqi, consider # importing from the following modules: # pyqi.core.interfaces.optparse.input_handler # pyqi.core.interfaces.optparse.output_handler # pyqi.interfaces.optparse.input_handler # pyqi.interfaces.optparse.output_handler # Convenience function for looking up parameters by name. param_lookup = make_parameter_collection_lookup_f(CommandConstructor) # Examples of how the command can be used from the command line using an # optparse interface. usage_examples = [ OptparseUsageExample(ShortDesc="A short single sentence description of the example", LongDesc="A longer, more detailed description", Ex="%%prog --foo --bar some_file") ] # inputs map command line arguments and values onto Parameters. It is possible # to define options here that do not exist as parameters, e.g., an output file. inputs = [ # An example option that has a direct relationship with a Parameter. # OptparseOption(Parameter=param_lookup('name_of_a_parameter'), # InputType='existing_filepath', # the optparse type of input # InputAction='store', # the optparse action # InputHandler=None, # Apply a function to the input value to convert it into the type expected by Parameter.DataType # ShortName='n', # a parameter short name, can be None # # Name='foo', # implied by Parameter.Name. Can be overwritten here if desired # # Required=False, # implied by Parameter.Required. Can be promoted by setting True # # Help='help', # implied by Parameter.Description. Can be overwritten here if desired # # Default=None, # implied by Parameter.Default. Can be overwritten here if desired # # DefaultDescription=None, # implied by Parameter.DefaultDescription. Can be overwritten here if desired # convert_to_dashed_name=True), # whether the Name (either implied by Parameter or defined above) should have underscores converted to dashes when displayed to the user # # An example option that does not have an associated Parameter. # OptparseOption(Parameter=None, # InputType='new_filepath', # InputAction='store', # InputHandler=None, # we don't need an InputHandler because this option isn't being converted into a format that a Parameter expects # ShortName='o', # Name='output-fp', # Required=True, # Help='output filepath') %(input_fmt)s ] # outputs map result keys to output options and handlers. It is not necessary # to supply an associated option, but if you do, it must be an option from the # inputs list (above). outputs = [ # An example option that maps to a result key. # OptparseResult(ResultKey='some_result', # OutputHandler=write_string, # a function applied to the value at ResultKey # # # the name of the option (defined in inputs, above), whose # # value will be made available to OutputHandler. This name # # can be either an underscored or dashed version of the # # option name (e.g., 'output_fp' or 'output-fp') # OptionName='output-fp'), # # An example option that does not map to a result key. # OptparseResult(ResultKey='some_other_result', # OutputHandler=print_string) ]""" # Fill out by Parameter, and comment out some of the most common stuff. input_format = """ OptparseOption(Parameter=param_lookup('%(name)s'), InputType=%(datatype)s, InputAction='%(action)s', # default is 'store', change if desired InputHandler=None, # must be defined if desired ShortName=None), # must be defined if desired # Name='%(name)s', # implied by Parameter # Required=%(required)s, # implied by Parameter # Help='%(help)s', # implied by Parameter %(default_block)s """ default_block_format = """# Default=%(default)s, # implied by Parameter # DefaultDescription=%(default_description)s, # implied by Parameter """ class MakeOptparse(CodeHeaderGenerator): BriefDescription = "Consume a Command, stub out an optparse configuration" LongDescription = """Construct and stub out the basic optparse configuration for a given Command. This template provides comments and examples of what to fill in.""" Parameters = ParameterCollection( CodeHeaderGenerator.Parameters.Parameters + [ Parameter(Name='command', DataType=Command, Description='an existing Command', Required=True), Parameter(Name='command_module', DataType=str, Description='the Command source module', Required=True) ] ) def run(self, **kwargs): code_header_lines = super(MakeOptparse, self).run( author=kwargs['author'], email=kwargs['email'], license=kwargs['license'], copyright=kwargs['copyright'], version=kwargs['version'], credits=kwargs['credits'])['result'] result_lines = code_header_lines # construct inputs based off of parameters param_formatted = [] for param in sorted(kwargs['command'].Parameters.values(), key=attrgetter('Name')): if param.Required: default_block = '' else: default_fmt = { 'default': repr(param.Default), 'default_description': repr(param.DefaultDescription) } default_block = default_block_format % default_fmt if param.DataType is bool: action = 'store_true' data_type = None else: action = 'store' data_type = param.DataType fmt = {'name':param.Name, 'datatype':data_type, 'action':action, 'required':str(param.Required), 'help':param.Description, 'default_block':default_block} param_formatted.append(input_format % fmt) param_formatted = ''.join(param_formatted) header_fmt = {'command_module':kwargs['command_module'], 'input_fmt': param_formatted} result_lines.extend((header_format % header_fmt).split('\n')) return {'result': result_lines} CommandConstructor = MakeOptparse pyqi-0.2.0/pyqi/core/000077500000000000000000000000001220701352200143775ustar00rootroot00000000000000pyqi-0.2.0/pyqi/core/__init__.py000066400000000000000000000012311220701352200165050ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" pyqi-0.2.0/pyqi/core/command.py000066400000000000000000000162141220701352200163730ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" import re from pyqi.core.log import NullLogger from pyqi.core.exception import (IncompetentDeveloperError, InvalidReturnTypeError, UnknownParameterError, MissingParameterError) class Parameter(object): """A ``Command`` variable A ``Command`` variable is interface agnostic and is analogous to a function argument. """ def __init__(self, Name, DataType, Description, Required=False, Default=None, DefaultDescription=None, ValidateValue=None): """ ``Name`` should be a valid Python name so that users can supply either a dictionary as input or named arguments. ``DataType`` specifies the type that the input must be. The input should be an instance of type ``DataType``. ``ValidateValue`` can be set as a function that will validate the value associated with a ``Parameter``. """ if not self._is_valid_name(Name): raise IncompetentDeveloperError("Parameter '%s' is not a valid " "Python variable name. Parameter " "names must be alphanumeric and " "start with a letter or " "underscore." % Name) if Required and Default is not None: raise IncompetentDeveloperError("Found required parameter '%s' " "with default value '%s'. Required parameters cannot have " "default values." % (Name, Default)) self.Name = Name self.DataType = DataType self.Description = Description self.Required = Required self.Default = Default self.DefaultDescription = DefaultDescription self.ValidateValue = ValidateValue def _is_valid_name(self, name): return name == self._pythonize(name) def _pythonize(self, name): """Taken from http://stackoverflow.com/a/3303361""" # Remove invalid characters. name = re.sub('[^0-9a-zA-Z_]', '', name) # Remove leading characters until we find a letter or underscore. name = re.sub('^[^a-zA-Z_]+', '', name) return name class ParameterCollection(dict): """A collection of parameters with dict like lookup""" def __init__(self, Parameters): self.Parameters = Parameters for p in self.Parameters: if p.Name in self: raise IncompetentDeveloperError("Found duplicate Parameter " "name '%s'. Parameter names " "must be unique." % p.Name) else: super(ParameterCollection, self).__setitem__(p.Name, p) def __getitem__(self, key): try: return super(ParameterCollection, self).__getitem__(key) except KeyError: raise UnknownParameterError("Parameter not found: %s" % key) def __setitem__(self, key, val): raise TypeError("ParameterCollections are immutable") __delattr__ = __setitem__ class Command(object): """Base class for ``Command`` A ``Command`` is interface agnostic, knows how to run itself and knows about the arguments that it can take (via ``Parameters``). """ BriefDescription = "" # 1 sentence description LongDescription = """""" # longer, more detailed description Parameters = ParameterCollection([]) def __init__(self, **kwargs): """ """ self._logger = NullLogger() def __call__(self, **kwargs): """Safely execute a ``Command``""" self_str = str(self.__class__) self._logger.info('Starting command: %s' % self_str) self._validate_kwargs(kwargs) self._set_defaults(kwargs) try: result = self.run(**kwargs) except Exception, e: self._logger.fatal('Error executing command: %s' % self_str) raise e else: self._logger.info('Completed command: %s' % self_str) # verify the result type if not isinstance(result, dict): self._logger.fatal('Unsupported result return type for command: ' '%s' % self_str) raise InvalidReturnTypeError("Unsupported result return type. " "Results must be stored in a " "dictionary.") return result def _validate_kwargs(self, kwargs): """Validate input kwargs prior to executing a ``Command`` This method can be overridden by subclasses. The baseclass defines only a basic validation. """ self_str = str(self.__class__) # check required parameters for p in self.Parameters.values(): if p.Required and p.Name not in kwargs: self._logger.fatal('Missing required parameter %s in %s' % (p.Name, self_str)) raise MissingParameterError("Missing required parameter %s in %s" % (p.Name, self_str)) if p.Name in kwargs and p.ValidateValue: if not p.ValidateValue(kwargs[p.Name]): self._logger.fatal("Parameter %s cannot take value %s in %s" % (p.Name, kwargs[p.Name], self_str)) raise ValueError("Parameter %s cannot take value %s in %s" % (p.Name, kwargs[p.Name], self_str)) # make sure we only have things we expect for opt in kwargs: if opt not in self.Parameters: self._logger.fatal('Unknown parameter %s in %s' % (opt, self_str)) raise UnknownParameterError("Unknown parameter %s in %s" % (opt, self_str)) def _set_defaults(self, kwargs): """Set defaults for optional parameters""" for p in self.Parameters.values(): if not p.Required and p.Name not in kwargs: kwargs[p.Name] = p.Default def run(self, **kwargs): """Exexcute a ``Command`` A ``Command`` must accept **kwargs to run, and must return a ``dict`` as a result. """ raise NotImplementedError("All subclasses must implement run.") def make_parameter_collection_lookup_f(obj): """Return a function for convenient parameter lookup. ``obj`` should be a Command (sub)class or instance. """ def lookup_f(name): return obj.Parameters[name] return lookup_f pyqi-0.2.0/pyqi/core/container.py000066400000000000000000000150151220701352200167350ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" class ContainerError(Exception): pass class CannotReadError(ContainerError): pass class CannotWriteError(ContainerError): pass class Passthrough(object): """Basic pass through container class""" _reserved = set(['_reserved','TypeName', 'Info']) TypeName = "Passthrough" Info = None def __init__(self, *args, **kwargs): if 'Info' in kwargs: super(Passthrough, self).__setattr__('Info', kwargs['Info']) def _load_if_needed(self): raise NotImplementedError("Passthrough cannot issue I/O.") def __getattr__(self, attr): """Pass through to contained class if the attribute is not recognized""" if attr in super(Passthrough, self).__getattribute__('_reserved'): return super(Passthrough, self).__getattribute__(attr) self._load_if_needed() return getattr(self._object, attr) def __setattr__(self, attr, val): """Pass through to contained class if the attribute is not recognized""" if attr in super(Passthrough, self).__getattribute__('_reserved'): return super(Passthrough, self).__setattr__(attr, val) self._load_if_needed() setattr(self._object, attr, val) def __hasattr__(self, attr): """Pass through to contained class if the attribute is not recognized""" if attr in super(Passthrough, self).__getattribute__('_reserved'): return True self._load_if_needed() return hasattr(self._object, attr) class PassthroughIO(Passthrough): _reserved = set(['_reserved', 'TypeName', '_reader', '_writer', '_object', 'InPath', 'OutPath','read', 'write', '_load_if_needed', 'Info']) TypeName = "PassthroughIO" def __init__(self, *args, **kwargs): super(PassthroughIO, self).__init__(*args, **kwargs) if 'Object' in kwargs: super(PassthroughIO, self).__setattr__('_object', kwargs['Object']) else: self._object = None if 'InPath' in kwargs: super(PassthroughIO, self).__setattr__('InPath', kwargs['InPath']) else: self.InPath = None if 'OutPath' in kwargs: super(PassthroughIO, self).__setattr__('OutPath', kwargs['OutPath']) else: self.OutPath = None def _load_if_needed(self): """Load if the object has not already been loaded""" if self._object is None: if self.InPath is not None: self._object = self._reader(self, self.InPath) else: raise CannotReadError("No object and InPath is None.") def read(self): """Attempt to read""" if self._object is None: if self.InPath is None: raise CannotReadError("InPath is None.") self._object = self._reader(self, self.InPath) def write(self): """Attempt to write""" if self._object is None: self._read() if self._object is not None: if self.OutPath is None: raise CannotWriteError("OutPath is None.") self._writer(self, self.OutPath) self._object = None class PassthroughRead(PassthroughIO): def __init__(self, *args, **kwargs): if 'reader' in kwargs: super(PassthroughRead, self).__setattr__('_reader', kwargs['reader']) else: raise ContainerError("A reader is required.") super(PassthroughRead, self).__init__(*args, **kwargs) class PassthroughWrite(PassthroughIO): def __init__(self, *args, **kwargs): if 'writer' in kwargs: super(PassthroughWrite, self).__setattr__('_reader', kwargs['writer']) else: raise ContainerError("A writer is required.") super(PassthroughWrite, self).__init__(*args, **kwargs) class DelayRead(PassthroughRead): """Contain an object and issue IO when an object attribute is requested""" TypeName = "DelayRead" class DelayWrite(PassthroughWrite): """Contain an object and issue IO with the container is no more""" TypeName = "DelayWrite" def __del__(self): self.write() class ImmediateRead(PassthroughRead): """Issue an immediate read on construction""" TypeName = "ImmediateRead" def __init__(self, *args, **kwargs): super(ImmediateRead, self).__init__(*args, **kwargs) self.read() class ImmediateWrite(PassthroughWrite): TypeName = "ImmediateWrite" def __init__(self, *args, **kwargs): super(ImmediateWrite, self).__init__(*args, **kwargs) self.write() def default_write_str(obj, path): f = open(path,'w') f.write(str(obj._object)) f.close() def default_read_str(obj, path): f = open(path) return f.read() def default_write_object(obj, path): f = open(path, 'w') f.write(repr(obj._object)) f.close() def default_read_object(obj, path): f = open(path) return f.read() # eval isn't safe... IOType = {'ImmediateRead':ImmediateRead, 'ImmediateWrite':ImmediateWrite, 'DelayRead':DelayRead, 'DelayWrite':DelayWrite} IOLookup = {str:(default_read_str, default_write_str)} def WithIO(obj, IO_type=None, IO_lookup=None, **kwargs): if IO_type is None: raise ContainerError("IO_type is required.") if IO_type not in IOType: raise ContainerError("Unknown IO_type: %s" % IO_type) if kwargs is None: kwargs = {} if IO_lookup is None: IO_lookup = IOLookup obj_type = obj.__class__ kwargs['Object'] = obj if obj_type in IO_lookup: reader, writer = IO_lookup[obj_type] else: reader, writer = default_read_object, default_write_object kwargs['reader'] = reader kwargs['writer'] = writer return IOType[IO_type](**kwargs) def WithoutIO(obj, **kwargs): return obj pyqi-0.2.0/pyqi/core/exception.py000066400000000000000000000016661220701352200167600ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" class CommandError(Exception): pass class IncompetentDeveloperError(CommandError): pass class MissingParameterError(CommandError): pass class InvalidReturnTypeError(IncompetentDeveloperError): pass class UnknownParameterError(IncompetentDeveloperError): pass pyqi-0.2.0/pyqi/core/factory.py000066400000000000000000000022771220701352200164300ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" def general_factory(command_constructor, usage_examples, inputs, outputs, version, interface=None): """Generalized interface factory""" class IObject(interface): """Dynamic interface object""" CommandConstructor = command_constructor def _get_usage_examples(self): return usage_examples def _get_inputs(self): return inputs def _get_outputs(self): return outputs def _get_version(self): return version return IObject pyqi-0.2.0/pyqi/core/interface.py000066400000000000000000000254061220701352200167200ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" import importlib from sys import exit, stderr from ConfigParser import SafeConfigParser from glob import glob from os.path import basename, dirname, expanduser, join from pyqi.core.exception import IncompetentDeveloperError class Interface(object): CommandConstructor = None def __init__(self, **kwargs): """ """ self.CmdInstance = None if self.CommandConstructor is None: raise IncompetentDeveloperError("Cannot construct an Interface " "without a CommandConstructor.") self.CmdInstance = self.CommandConstructor(**kwargs) self._validate_usage_examples(self._get_usage_examples()) self._validate_inputs(self._get_inputs()) self._validate_outputs(self._get_outputs()) def __call__(self, in_, *args, **kwargs): self._the_in_validator(in_) cmd_input = self._input_handler(in_, *args, **kwargs) return self._output_handler(self.CmdInstance(**cmd_input)) def _validate_usage_examples(self, usage_examples): """Perform validation on a list of ``InterfaceUsageExample`` objects. ``usage_examples`` will be the output of ``self._get_usage_examples()``. Subclasses can override to perform validation that requires a list of all usage examples. Validation that should be performed on a per-usage example basis should instead go into ``InterfaceUsageExample._validate_usage_example``. """ pass def _validate_inputs(self, inputs): """Perform validation on a list of ``InterfaceOption`` objects. ``inputs`` will be the output of ``self._get_inputs()``. Subclasses can override to perform validation that requires a list of all input options. Validation that should be performed on a per-option basis should instead go into ``InterfaceOption._validate_option``. """ param_names = [input_.getParameterName() for input_ in inputs if input_.getParameterName() is not None] if len(param_names) != len(set(param_names)): raise IncompetentDeveloperError("Found more than one " "InterfaceOption mapping to the " "same Parameter.") def _validate_outputs(self, outputs): """Perform validation on a list of ``InterfaceResult`` objects. ``outputs`` will be the output of ``self._get_outputs()``. Subclasses can override to perform validation that requires a list of all interface results. Validation that should be performed on a per-interface result basis should instead go into ``InterfaceResult._validate_result``. """ pass def _the_in_validator(self, in_): """The job securator""" raise NotImplementedError("All subclasses must implement " "_the_in_validator.") def _input_handler(self, in_, *args, **kwargs): raise NotImplementedError("All subclasses must implement " "_input_handler.") def _output_handler(self, results): raise NotImplementedError("All subclasses must implement " "_output_handler.") def _get_usage_examples(self): """Return a list of ``InterfaceUsageExample`` objects These are typically set in a command+interface specific configuration file and passed to ``pyqi.core.general_factory`` """ raise NotImplementedError("Must define _get_usage_examples") def _get_inputs(self): """Return a list of ``InterfaceOption`` objects These are typically set in a command+interface specific configuration file and passed to ``pyqi.core.general_factory`` """ raise NotImplementedError("Must define _get_inputs") def _get_outputs(self): """Return a list of ``InterfaceResult`` objects These are typically set in a command+interface specific configuration file and passed to ``pyqi.core.general_factory`` """ raise NotImplementedError("Must define _get_outputs") def _get_version(self): """Return a version string, e.g., ``'0.1'`` This is typically set in a command+interface specific configuration file and passed to ``pyqi.core.general_factory`` """ raise NotImplementedError("Must define _get_version") class InterfaceOption(object): """Describes an option and what to do with it""" def __init__(self, Parameter=None, InputType=None, InputAction=None, InputHandler=None, ShortName=None, Name=None, Required=False, Help=None, Default=None, DefaultDescription=None, convert_to_dashed_name=True): self.Parameter = Parameter if self.Parameter is None: if Name is None: raise IncompetentDeveloperError("Must specify a Name for the " "InterfaceOption since it " "doesn't have a Parameter.") if Help is None: raise IncompetentDeveloperError("Must specify Help for the " "InterfaceOption since it " "doesn't have a Parameter.") self.Name = Name self.Help = Help self.Required = Required self.Default = Default self.DefaultDescription = DefaultDescription else: # Transfer information from Parameter unless overridden here. self.Name = Parameter.Name if Name is None else Name self.Help = Parameter.Description if Help is None else Help self.Default = Parameter.Default if Default is None else Default self.DefaultDescription = Parameter.DefaultDescription if \ DefaultDescription is None else DefaultDescription # If a parameter is required, the option is always required, but # if a parameter is not required, but the option does require it, # then we make the option required. if not Parameter.Required and Required: self.Required = True else: self.Required = Parameter.Required # This information is never contained in a Parameter. self.InputType = InputType self.InputAction = InputAction self.InputHandler = InputHandler self.ShortName = ShortName if convert_to_dashed_name: self.Name = self.Name.replace('_', '-') if self.Required and self.Default is not None: raise IncompetentDeveloperError("Found required option '%s' " "with default value '%s'. Required options cannot have " "default values." % (self.Name, self.Default)) self._validate_option() def _validate_option(self): """Interface specific validation requirements""" raise NotImplementedError("Must define in the subclass") def getParameterName(self): if self.Parameter is None: return None else: return self.Parameter.Name class InterfaceResult(object): """Describes a result and what to do with it""" def __init__(self, ResultKey, OutputHandler, OptionName=None): self.ResultKey = ResultKey self.OutputHandler = OutputHandler self.OptionName = OptionName self._validate_result() def _validate_result(self): """Validate a result object""" raise NotImplementedError("Must implement in a subclass") class InterfaceUsageExample(object): """Provide structure to a usage example""" def __init__(self, ShortDesc, LongDesc, Ex): self.ShortDesc = ShortDesc self.LongDesc = LongDesc self.Ex = Ex self._validate_usage_example() def _validate_usage_example(self): """Interface specific usage example validation""" raise NotImplementedError("Must define in the subclass") def get_command_names(config_base_name): """Return a list of available command names. Command names are strings and are returned in alphabetical order. ``config_base_name`` must be the python module path to a directory containing config files. """ # Load the interface configuration base. try: config_base_module = importlib.import_module(config_base_name) except ImportError: raise ImportError("Unable to load base config module: %s" % config_base_name) config_base_dir = dirname(config_base_module.__file__) # from http://stackoverflow.com/questions/1057431/loading-all-modules-in-a-folder-in-python command_names = CommandList() for f in glob(join(config_base_dir, '*.py')): command_name = basename(f) if not command_name.startswith('__init__'): command_names.append(command_name[:-3]) command_names.sort() return command_names def get_command_config(command_config_module, cmd, exit_on_failure=True): """Get the configuration for a ``Command``""" cmd_cfg = None error_msg = None python_cmd_name = cmd.replace('-', '_') try: cmd_cfg = importlib.import_module('.'.join([command_config_module, python_cmd_name])) except ImportError, e: error_msg = str(e) if exit_on_failure: stderr.write("Unable to import the command configuration for " "%s:\n" % cmd) stderr.write(error_msg) stderr.write('\n') exit(1) return cmd_cfg, error_msg class CommandList(list): def __init__(self): super(CommandList, self).__init__() def append(self, item): super(CommandList, self).append(self._convert_to_dashed_name(item)) def __contains__(self, item): return super(CommandList, self).__contains__(self._convert_to_dashed_name(item)) def _convert_to_dashed_name(self, name): return name.replace('_', '-') pyqi-0.2.0/pyqi/core/interfaces/000077500000000000000000000000001220701352200165225ustar00rootroot00000000000000pyqi-0.2.0/pyqi/core/interfaces/__init__.py000066400000000000000000000012311220701352200206300ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" pyqi-0.2.0/pyqi/core/interfaces/optparse/000077500000000000000000000000001220701352200203575ustar00rootroot00000000000000pyqi-0.2.0/pyqi/core/interfaces/optparse/__init__.py000066400000000000000000000426011220701352200224730ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Gavin Huttley", "Rob Knight", "Doug Wendel", "Jai Ram Rideout", "Jose Antonio Navas Molina"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" import os import types from copy import copy from glob import glob from os.path import abspath, exists, isdir, isfile, split from optparse import (Option, OptionParser, OptionGroup, OptionValueError, OptionError) from pyqi.core.interface import (Interface, InterfaceOption, InterfaceUsageExample, InterfaceResult) from pyqi.core.factory import general_factory from pyqi.core.exception import IncompetentDeveloperError from pyqi.core.command import Parameter class OptparseResult(InterfaceResult): def _validate_result(self): pass class OptparseOption(InterfaceOption): """An augmented option that expands a Parameter into an Option""" def __init__(self, Parameter=None, InputType=str, InputAction='store', InputHandler=None, ShortName=None, Name=None, Required=False, Help=None, Default=None, DefaultDescription=None, convert_to_dashed_name=True): super(OptparseOption, self).__init__(Parameter=Parameter, InputType=InputType, InputAction=InputAction, InputHandler=InputHandler, ShortName=ShortName, Name=Name, Required=Required, Help=Help, Default=Default, DefaultDescription=DefaultDescription, convert_to_dashed_name=convert_to_dashed_name) def _validate_option(self): # optparse takes care of validating InputType, InputAction, and # ShortName, so we don't need any checks here. pass def __str__(self): if self.ShortName is None: return '--%s' % self.Name else: return '-%s/--%s' % (self.ShortName, self.Name) def getOptparseOption(self): if self.Required: # If the option doesn't already end with [REQUIRED], add it. help_text = self.Help if not help_text.strip().endswith('[REQUIRED]'): help_text += ' [REQUIRED]' if self.ShortName is None: option = PyqiOption('--' + self.Name, type=self.InputType, action=self.InputAction, help=help_text) else: option = PyqiOption('-' + self.ShortName, '--' + self.Name, type=self.InputType, action=self.InputAction, help=help_text) else: if self.DefaultDescription is None: help_text = '%s [default: %%default]' % self.Help else: help_text = '%s [default: %s]' % (self.Help, self.DefaultDescription) if self.ShortName is None: option = PyqiOption('--' + self.Name, type=self.InputType, action=self.InputAction, help=help_text, default=self.Default) else: option = PyqiOption('-' + self.ShortName, '--' + self.Name, type=self.InputType, action=self.InputAction, help=help_text, default=self.Default) return option class OptparseUsageExample(InterfaceUsageExample): """Provide structure to a usage example""" def _validate_usage_example(self): if self.ShortDesc is None: raise IncompetentDeveloperError("Must define ShortDesc") if self.LongDesc is None: raise IncompetentDeveloperError("Must define LongDesc") if self.Ex is None: raise IncompetentDeveloperError("Must define Ex") class OptparseInterface(Interface): """A command line interface""" DisallowPositionalArguments = True HelpOnNoArguments = True OptionalInputLine = '[] indicates optional input (order unimportant)' RequiredInputLine = '{} indicates required input (order unimportant)' def __init__(self, **kwargs): super(OptparseInterface, self).__init__(**kwargs) def _validate_usage_examples(self, usage_examples): super(OptparseInterface, self)._validate_usage_examples(usage_examples) if len(usage_examples) < 1: raise IncompetentDeveloperError("There are no usage examples " "associated with this command.") def _the_in_validator(self, in_): """Validate input coming from the command line""" if not isinstance(in_, list): raise IncompetentDeveloperError("Unsupported input '%r'. Input " "must be a list." % in_) def _input_handler(self, in_, *args, **kwargs): """Parses command-line input.""" required_opts = [opt for opt in self._get_inputs() if opt.Required] optional_opts = [opt for opt in self._get_inputs() if not opt.Required] # Build the usage and version strings usage = self._build_usage_lines(required_opts) version = 'Version: %prog ' + self._get_version() # Instantiate the command line parser object parser = OptionParser(usage=usage, version=version) # If the command has required options and no input arguments were # provided, print the help string. if len(required_opts) > 0 and self.HelpOnNoArguments and len(in_) == 0: parser.print_usage() return parser.exit(-1) if required_opts: # Define an option group so all required options are grouped # together and under a common header. required = OptionGroup(parser, "REQUIRED options", "The following options must be provided " "under all circumstances.") for ro in required_opts: required.add_option(ro.getOptparseOption()) parser.add_option_group(required) # Add the optional options. for oo in optional_opts: parser.add_option(oo.getOptparseOption()) ##### # THIS IS THE NATURAL BREAKING POINT FOR THIS FUNCTIONALITY ##### # Parse our input. opts, args = parser.parse_args(in_) # If positional arguments are not allowed, and any were provided, raise # an error. if self.DisallowPositionalArguments and len(args) != 0: parser.error("Positional argument detected: %s\n" % str(args[0]) + " Be sure all parameters are identified by their option name.\n" + " (e.g.: include the '-i' in '-i INPUT_DIR')") # Test that all required options were provided. if required_opts: # dest may be different from the original option name because # optparse converts names from dashed to underscored. required_option_ids = [(o.dest, o.get_opt_string()) for o in required.option_list] for required_dest, required_name in required_option_ids: if getattr(opts, required_dest) is None: parser.error('Required option %s omitted.' % required_name) # Build up command input dictionary. This will be passed to # Command.__call__ as kwargs. self._optparse_input = opts.__dict__ cmd_input_kwargs = {} for option in self._get_inputs(): if option.Parameter is not None: param_name = option.getParameterName() optparse_clean_name = \ self._get_optparse_clean_name(option.Name) if option.InputHandler is None: value = self._optparse_input[optparse_clean_name] else: value = option.InputHandler( self._optparse_input[optparse_clean_name]) cmd_input_kwargs[param_name] = value return cmd_input_kwargs def _build_usage_lines(self, required_options): """ Build the usage string from components """ line1 = 'usage: %prog [options] ' + \ '{%s}' % ' '.join(['%s %s' % (str(rp),rp.Name.upper()) for rp in required_options]) formatted_usage_examples = [] for usage_example in self._get_usage_examples(): short_description = usage_example.ShortDesc.strip(':').strip() long_description = usage_example.LongDesc.strip(':').strip() example = usage_example.Ex.strip() if short_description: formatted_usage_examples.append('%s: %s\n %s' % (short_description, long_description, example)) else: formatted_usage_examples.append('%s\n %s' % (long_description,example)) formatted_usage_examples = '\n\n'.join(formatted_usage_examples) lines = (line1, '', # Blank line self.OptionalInputLine, self.RequiredInputLine, '', # Blank line self.CmdInstance.LongDescription, '', # Blank line 'Example usage: ', 'Print help message and exit', ' %prog -h\n', formatted_usage_examples) return '\n'.join(lines) def _output_handler(self, results): """Deal with things in output if we know how""" handled_results = {} for output in self._get_outputs(): rk = output.ResultKey if rk not in results: raise IncompetentDeveloperError("Did not find the expected " "output '%s' in results." % rk) if output.OptionName is None: handled_results[rk] = output.OutputHandler(rk, results[rk]) else: optparse_clean_name = \ self._get_optparse_clean_name(output.OptionName) opt_value = self._optparse_input[optparse_clean_name] handled_results[rk] = output.OutputHandler(rk, results[rk], opt_value) return handled_results def _get_optparse_clean_name(self, name): # optparse converts dashes to underscores in long option names. return name.replace('-', '_') def optparse_factory(command_constructor, usage_examples, inputs, outputs, version): """Optparse command line interface factory command_constructor - a subclass of ``Command`` usage_examples - usage examples for using ``command_constructor`` via a command line interface. inputs - config ``inputs`` or a list of ``OptparseOptions`` outputs - config ``outputs`` or a list of ``OptparseResults`` version - config ``__version__`` (a version string) """ return general_factory(command_constructor, usage_examples, inputs, outputs, version, OptparseInterface) def optparse_main(interface_object, local_argv): """Construct and execute an interface object""" optparse_cmd = interface_object() result = optparse_cmd(local_argv[1:]) return 0 # Definition of PyqiOption option type, a subclass of Option that contains # specific types for filepaths and directory paths. # # This code was derived from PyCogent (http://www.pycogent.org) and QIIME # (http://www.qiime.org), where it was initally developed. # # QIIME and PyCogent are GPL projects, but we obtained permission from the # authors of this code to port it to pyqi (and keep it under pyqi's BSD # license). # # TODO: this code needs to be refactored to better fit the pyqi framework. # Should probably get added to the OptparseInterface class. def check_existing_filepath(option, opt, value): if not exists(value): raise OptionValueError( "option %s: file does not exist: %r" % (opt, value)) elif not isfile(value): raise OptionValueError( "option %s: not a regular file (can't be a directory!): %r" % (opt, value)) else: return value def check_existing_filepaths(option, opt, value): paths = [] for v in value.split(','): fps = glob(v) if len(fps) == 0: raise OptionValueError( "No filepaths match pattern/name '%s'. " "All patterns must be matched at least once." % v) else: paths.extend(fps) values = [] for v in paths: check_existing_filepath(option,opt,v) values.append(v) return values def check_existing_dirpath(option, opt, value): if not exists(value): raise OptionValueError( "option %s: directory does not exist: %r" % (opt, value)) elif not isdir(value): raise OptionValueError( "option %s: not a directory (can't be a file!): %r" % (opt, value)) else: return value def check_new_filepath(option, opt, value): return value def check_new_dirpath(option, opt, value): return value def check_existing_path(option, opt, value): if not exists(value): raise OptionValueError( "option %s: path does not exist: %r" % (opt, value)) return value def check_new_path(option, opt, value): return value def check_multiple_choice(option, opt, value): values = value.split(option.split_char) for v in values: if v not in option.mchoices: choices = ",".join(map(repr, option.mchoices)) raise OptionValueError( "option %s: invalid choice: %r (choose from %s)" % (opt, v, choices)) return values def check_blast_db(option, opt, value): db_dir, db_name = split(abspath(value)) if not exists(db_dir): raise OptionValueError( "option %s: path does not exists: %r" % (opt, db_dir)) elif not isdir(db_dir): raise OptionValueError( "option %s: not a directory: %r" % (opt, db_dir)) return value class PyqiOption(Option): ATTRS = Option.ATTRS + ['mchoices','split_char'] TYPES = Option.TYPES + ("existing_path", "new_path", "existing_filepath", "existing_filepaths", "new_filepath", "existing_dirpath", "new_dirpath", "multiple_choice", "blast_db") TYPE_CHECKER = copy(Option.TYPE_CHECKER) # for cases where the user specifies an existing file or directory # as input, but it can be either a dir or a file TYPE_CHECKER["existing_path"] = check_existing_path # for cases where the user specifies a new file or directory # as output, but it can be either a dir or a file TYPE_CHECKER["new_path"] = check_new_path # for cases where the user passes a single existing file TYPE_CHECKER["existing_filepath"] = check_existing_filepath # for cases where the user passes one or more existing files # as a comma-separated list - paths are returned as a list TYPE_CHECKER["existing_filepaths"] = check_existing_filepaths # for cases where the user is passing a new path to be # create (e.g., an output file) TYPE_CHECKER["new_filepath"] = check_new_filepath # for cases where the user is passing an existing directory # (e.g., containing a set of input files) TYPE_CHECKER["existing_dirpath"] = check_existing_dirpath # for cases where the user is passing a new directory to be # create (e.g., an output dir which will contain many result files) TYPE_CHECKER["new_dirpath"] = check_new_dirpath # for cases where the user is passing one or more values # as comma- or semicolon-separated list # choices are returned as a list TYPE_CHECKER["multiple_choice"] = check_multiple_choice # for cases where the user is passing a blast database option # blast_db is returned as a string TYPE_CHECKER["blast_db"] = check_blast_db def _check_multiple_choice(self): if self.type == "multiple_choice": if self.mchoices is None: raise OptionError( "must supply a list of mchoices for type '%s'" % self.type, self) elif type(self.mchoices) not in (types.TupleType, types.ListType): raise OptionError( "choices must be a list of strings ('%s' supplied)" % str(type(self.mchoices)).split("'")[1], self) if self.split_char is None: self.split_char = ',' elif self.mchoices is not None: raise OptionError( "must not supply mchoices for type %r" % self.type, self) CHECK_METHODS = Option.CHECK_METHODS + [_check_multiple_choice] pyqi-0.2.0/pyqi/core/interfaces/optparse/input_handler.py000066400000000000000000000035431220701352200235720ustar00rootroot00000000000000#!/usr/bin/env python """Command line interface input handlers All input handlers must conform to the following function definittion function(option_value) """ #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.1-dev" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" def command_handler(option_value): """Dynamically load a Python object from a module and return an instance""" module, klass = option_value.rsplit('.',1) mod = __import__(module, fromlist=[klass]) return getattr(mod, klass)() def string_list_handler(option_value=None): """Split a comma-separated string into a list of strings.""" result = None if option_value is not None: result = option_value.split(',') return result def file_reading_handler(option_value=None): """Open a filepath for reading.""" result = None if option_value is not None: result = open(option_value, 'U') return result def load_file_lines(option_value): """Return a list of strings, one per line in the file. Each line will have leading and trailing whitespace stripped from it. """ with open(option_value, 'U') as f: return [line.strip() for line in f] def load_file_contents(option_value): """Return the contents of a file as a single string.""" with open(option_value, 'U') as f: return f.read() pyqi-0.2.0/pyqi/core/interfaces/optparse/output_handler.py000066400000000000000000000045631220701352200237760ustar00rootroot00000000000000#!/usr/bin/env python """Command line interface output handlers All output handlers must conform to the following function definition function(result_key, data, option_value=None) result_key - the corresponding key in the results dictionary data - the actual results option_value - if the handler is tied to an output option, the value of that option is represented here """ #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.1-dev" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from pyqi.core.exception import IncompetentDeveloperError import os def write_string(result_key, data, option_value=None): """Write a string to a file. A newline will be added to the end of the file. """ if option_value is None: raise IncompetentDeveloperError("Cannot write output without a " "filepath.") if os.path.exists(option_value): raise IOError("Output path '%s' already exists." % option_value) with open(option_value, 'w') as f: f.write(data) f.write('\n') def write_list_of_strings(result_key, data, option_value=None): """Write a list of strings to a file, one per line. A newline will be added to the end of the file. """ if option_value is None: raise IncompetentDeveloperError("Cannot write output without a " "filepath.") if os.path.exists(option_value): raise IOError("Output path '%s' already exists." % option_value) with open(option_value, 'w') as f: for line in data: f.write(line) f.write('\n') def print_list_of_strings(result_key, data, option_value=None): """Print a list of strings to stdout, one per line. ``result_key`` and ``option_value`` are ignored. """ for line in data: print line pyqi-0.2.0/pyqi/core/log.py000066400000000000000000000053231220701352200155350ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division from sys import stderr from datetime import datetime __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" class InvalidLoggerError(Exception): pass class Logger(object): """Abstract logging interface""" DEBUG = 'DEBUG' INFO = 'INFO' WARN = 'WARN' FATAL = 'FATAL' def debug(self, msg): """Log at the DEBUG level""" self._debug(msg) self.flush() def info(self, msg): """Log at the INFO level""" self._info(msg) self.flush() def warn(self, msg): """Log at the WARN level""" self._warn(msg) self.flush() def fatal(self, msg): """Log at the FATAL level""" self._fatal(msg) self.flush() def _debug(self, msg): raise NotImplementedError("All subclasses must implement debug.") def _info(self, msg): raise NotImplementedError("All subclasses must implement info.") def _warn(self, msg): raise NotImplementedError("All subclasses must implement warn.") def _fatal(self, msg): raise NotImplementedError("All subclasses must implement fatal.") def flush(self): """Flush buffers as needed""" pass def _get_timestamp(self): """Get an ISO standard timestamp""" return datetime.now().isoformat() def _format_line(self, level, msg): """Construct a logging line""" return '%s %s %s' % (self._get_timestamp(), level, msg) class NullLogger(Logger): """Ignore log messages""" def _debug(self, msg): pass def _info(self, msg): pass def _warn(self, msg): pass def _fatal(self, msg): pass class StdErrLogger(Logger): """Log messages directly to ``stderr``""" def _debug(self, msg): stderr.write(self._format_line(self.DEBUG, msg) + '\n') def _info(self, msg): stderr.write(self._format_line(self.INFO, msg) + '\n') def _warn(self, msg): stderr.write(self._format_line(self.WARN, msg) + '\n') def _fatal(self, msg): stderr.write(self._format_line(self.FATAL, msg) + '\n') pyqi-0.2.0/pyqi/interfaces/000077500000000000000000000000001220701352200155725ustar00rootroot00000000000000pyqi-0.2.0/pyqi/interfaces/__init__.py000066400000000000000000000012701220701352200177030ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com"pyqi-0.2.0/pyqi/interfaces/optparse/000077500000000000000000000000001220701352200174275ustar00rootroot00000000000000pyqi-0.2.0/pyqi/interfaces/optparse/__init__.py000066400000000000000000000012701220701352200215400ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com"pyqi-0.2.0/pyqi/interfaces/optparse/config/000077500000000000000000000000001220701352200206745ustar00rootroot00000000000000pyqi-0.2.0/pyqi/interfaces/optparse/config/__init__.py000066400000000000000000000012341220701352200230050ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" pyqi-0.2.0/pyqi/interfaces/optparse/config/make_bash_completion.py000066400000000000000000000036401220701352200254140ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Jai Ram Rideout", "Doug Wendel", "Greg Caporaso"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from pyqi.core.interfaces.optparse import (OptparseOption, OptparseUsageExample, OptparseResult) from pyqi.core.interfaces.optparse.output_handler import write_string from pyqi.core.command import make_parameter_collection_lookup_f from pyqi.commands.make_bash_completion import CommandConstructor param_lookup = make_parameter_collection_lookup_f(CommandConstructor) usage_examples = [ OptparseUsageExample(ShortDesc="Create a bash completion script", LongDesc="Create a bash completion script for use with a pyqi driver", Ex="%prog --command-config-module pyqi.interfaces.optparse.config --driver-name pyqi -o ~/.bash_completion.d/pyqi") ] inputs = [ OptparseOption(Parameter=param_lookup('command_config_module')), OptparseOption(Parameter=param_lookup('driver_name')), OptparseOption(Parameter=None, InputType='new_filepath', ShortName='o', Name='output-fp', Required=True, Help='output filepath') ] outputs = [ OptparseResult(ResultKey='result', OutputHandler=write_string, OptionName='output-fp') ] pyqi-0.2.0/pyqi/interfaces/optparse/config/make_command.py000066400000000000000000000053741220701352200236720ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from pyqi.core.interfaces.optparse import (OptparseOption, OptparseResult, OptparseUsageExample) from pyqi.core.interfaces.optparse.input_handler import string_list_handler from pyqi.core.interfaces.optparse.output_handler import write_list_of_strings from pyqi.core.command import make_parameter_collection_lookup_f from pyqi.commands.make_command import CommandConstructor param_lookup = make_parameter_collection_lookup_f(CommandConstructor) usage_examples = [ OptparseUsageExample(ShortDesc="Basic Command", LongDesc="Create a basic Command with appropriate attribution", Ex='%prog -n example -a "some author" -c "Copyright 2013, The pyqi project" -e "foo@bar.com" -l BSD --command-version "0.1" --credits "someone else","and another person" -o example.py') ] inputs = [ OptparseOption(Parameter=param_lookup('name'), ShortName='n'), OptparseOption(Parameter=param_lookup('author'), ShortName='a'), OptparseOption(Parameter=param_lookup('email'), ShortName='e'), OptparseOption(Parameter=param_lookup('license'), ShortName='l'), OptparseOption(Parameter=param_lookup('copyright'), ShortName='c'), OptparseOption(Parameter=param_lookup('version'), Name='command-version'), OptparseOption(Parameter=param_lookup('credits'), InputHandler=string_list_handler, Help='comma-separated list of other authors'), OptparseOption(Parameter=param_lookup('test_code'), InputType=None, InputAction='store_true'), OptparseOption(Parameter=None, InputType='new_filepath', ShortName='o', Name='output-fp', Required=True, Help='output filepath to store generated Python code') ] outputs = [ OptparseResult(ResultKey='result', OutputHandler=write_list_of_strings, OptionName='output-fp') ] pyqi-0.2.0/pyqi/interfaces/optparse/config/make_optparse.py000066400000000000000000000072321220701352200241040ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from pyqi.core.command import Command from pyqi.core.interfaces.optparse import (OptparseOption, OptparseUsageExample, OptparseResult) from pyqi.core.interfaces.optparse.input_handler import (command_handler, string_list_handler) from pyqi.core.interfaces.optparse.output_handler import write_list_of_strings from pyqi.core.command import make_parameter_collection_lookup_f from pyqi.commands.make_optparse import CommandConstructor param_lookup = make_parameter_collection_lookup_f(CommandConstructor) usage_examples = [ OptparseUsageExample(ShortDesc="Create an optparse config template", LongDesc="Construct the beginning of an optparse configuration file based on the Parameters required by the Command.", Ex='%prog -c pyqi.commands.make_optparse.MakeOptparse -m pyqi.commands.make_optparse -a "some author" --copyright "Copyright 2013, The pyqi project" -e "foo@bar.com" -l BSD --config-version "0.1" --credits "someone else","and another person" -o pyqi/interfaces/optparse/config/make_optparse.py'), OptparseUsageExample(ShortDesc="Create a different optparse config template", LongDesc="Construct the beginning of an optparse configuration file based on the Parameters required by the Command. This command corresponds to the pyqi tutorial example where a sequence_collection_summarizer command line interface is created for a SequenceCollectionSummarizer Command.", Ex='%prog -c sequence_collection_summarizer.SequenceCollectionSummarizer -m sequence_collection_summarizer -a "Greg Caporaso" --copyright "Copyright 2013, Greg Caporaso" -e "gregcaporaso@gmail.com" -l BSD --config-version 0.0.1 -o summarize_sequence_collection.py') ] inputs = [ OptparseOption(Parameter=param_lookup('command'), ShortName='c', InputHandler=command_handler), OptparseOption(Parameter=param_lookup('command_module'), ShortName='m'), OptparseOption(Parameter=param_lookup('author'), ShortName='a'), OptparseOption(Parameter=param_lookup('email'), ShortName='e'), OptparseOption(Parameter=param_lookup('license'), ShortName='l'), OptparseOption(Parameter=param_lookup('copyright')), OptparseOption(Parameter=param_lookup('version'), Name='config-version'), OptparseOption(Parameter=param_lookup('credits'), InputHandler=string_list_handler, Help='comma-separated list of other authors'), OptparseOption(Parameter=None, InputType='new_filepath', ShortName='o', Name='output-fp', Required=True, Help='output filepath to store generated optparse Python configuration file') ] outputs = [ OptparseResult(ResultKey='result', OutputHandler=write_list_of_strings, OptionName='output-fp') ] pyqi-0.2.0/pyqi/interfaces/optparse/input_handler.py000066400000000000000000000013601220701352200226350ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" ## Store project/inteface specific input handlers herepyqi-0.2.0/pyqi/interfaces/optparse/output_handler.py000066400000000000000000000013611220701352200230370ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" ## Store project/inteface specific output handlers herepyqi-0.2.0/pyqi/util.py000066400000000000000000000066551220701352200150120ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- """Utility functionality for the pyqi project.""" __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" from os import remove from os.path import split, splitext from sys import stdout, stderr from subprocess import Popen, PIPE, STDOUT from pyqi.core.log import StdErrLogger def pyqi_system_call(cmd, shell=True): """Call cmd and return (stdout, stderr, return_value). cmd can be either a string containing the command to be run, or a sequence of strings that are the tokens of the command. Please see Python's subprocess.Popen for a description of the shell parameter and how cmd is interpreted differently based on its value. This function is ported from QIIME (http://www.qiime.org), previously named qiime_system_call. QIIME is a GPL project, but we obtained permission from the authors of this function to port it to pyqi (and keep it under pyqi's BSD license). """ proc = Popen(cmd, shell=shell, universal_newlines=True, stdout=PIPE, stderr=PIPE) # communicate pulls all stdout/stderr from the PIPEs to # avoid blocking -- don't remove this line! stdout, stderr = proc.communicate() return_value = proc.returncode return stdout, stderr, return_value def remove_files(list_of_filepaths, error_on_missing=True): """Remove list of filepaths, optionally raising an error if any are missing This function is ported from PyCogent (http://www.pycogent.org). PyCogent is a GPL project, but we obtained permission from the authors of this function to port it to pyqi (and keep it under pyqi's BSD license). """ missing = [] for fp in list_of_filepaths: try: remove(fp) except OSError: missing.append(fp) if error_on_missing and missing: raise OSError, "Some filepaths were not accessible: %s" % '\t'.join(missing) def old_to_new_command(driver_name, project_title, local_argv): """Deprecate an old-style script. Will only work if the old-style script name matches a command name, and if all option names are the same between old and new. Use like this (put in the script you want to deprecate): #!/usr/bin/env python import sys from pyqi.util import old_to_new_command sys.exit(old_to_new_command('biom', 'BIOM', sys.argv)) """ logger = StdErrLogger() cmd_name = splitext(split(local_argv[0])[1])[0] base_cmd = "%s %s" % (driver_name, cmd_name) command = '%s %s' % (base_cmd, ' '.join(local_argv[1:])) logger.info("This is a new-style %s script. You should now call it with: " "%s" % (project_title, base_cmd)) logger.info("Calling: %s " % command) result_stdout, result_stderr, result_retval = pyqi_system_call(command) stdout.write(result_stdout) stderr.write(result_stderr) return result_retval pyqi-0.2.0/scripts/000077500000000000000000000000001220701352200141545ustar00rootroot00000000000000pyqi-0.2.0/scripts/pyqi000077500000000000000000000141041220701352200150640ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi Project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" import importlib import textwrap from sys import argv, exit, stderr from pyqi.core.interface import get_command_names, get_command_config from pyqi.core.interfaces.optparse import optparse_main, optparse_factory from os.path import basename from string import ljust ### we actually have some flexibility here to make the driver interface agnostic as well TERM_WIDTH = 80 INDENT = 3 def usage(cmd_cfg_mod, command_names): """Modeled after git...""" # limit to a reasonable number of characters valid_cmds = [] invalid_cmds = [] for c in command_names: cmd_cfg, error_msg = get_command_config(cmd_cfg_mod, c, exit_on_failure=False) if cmd_cfg is None: invalid_cmds.append((c, error_msg)) else: desc = cmd_cfg.CommandConstructor.BriefDescription valid_cmds.append((c, desc)) # determine widths max_cmd = max(map(lambda x: len(x[0]), valid_cmds + invalid_cmds)) desc_limit = TERM_WIDTH - (INDENT + max_cmd + INDENT) cmd_end = INDENT + max_cmd + INDENT print "usage: %s []" % argv[0] print print "The currently available commands are:" # format: # indent command indent description for c, desc in valid_cmds: cmd_formatted = ljust(''.join([' ' * INDENT, c]), cmd_end) print ''.join([cmd_formatted, desc[:desc_limit]]) if invalid_cmds: print print "The following commands could not be loaded:" for c, error_msg in invalid_cmds: cmd_formatted = ljust(''.join([' ' * INDENT, c]), cmd_end) print ''.join([cmd_formatted, 'Error: %s' % error_msg]) print print "See '%s help ' for more information on a specific command." % argv[0] exit(0) def get_cmd_obj(cmd_cfg_mod, cmd): """Get a ``Command`` object""" cmd_cfg, _ = get_command_config(cmd_cfg_mod, cmd) return optparse_factory(cmd_cfg.CommandConstructor, cmd_cfg.usage_examples, cmd_cfg.inputs, cmd_cfg.outputs, cmd_cfg.__version__) def help_(cmd_cfg_mod, cmd): """Dump the help for a ``Command``""" cmd_obj = get_cmd_obj(cmd_cfg_mod, cmd) optparse_main(cmd_obj, ['help', '-h']) def assert_command_exists(command_name, command_names, driver_name): if command_name not in command_names: error_msg = '\n'.join(textwrap.wrap("Unrecognized command %s. Please " "make sure that you didn't make a " "typo in the command name." % command_name, TERM_WIDTH)) error_msg += '\n\n' error_msg += '\n'.join(textwrap.wrap("To see a list of all available " "commands, run the following " "command:", TERM_WIDTH)) stderr.write("%s\n\n%s%s\n\n" % (error_msg, ' ' * INDENT, driver_name)) exit(1) if __name__ == '__main__': driver_name = 'pyqi' cmd_cfg_mod = 'pyqi.interfaces.optparse.config' if '--' in argv: stop_idx = argv.index('--') if '--driver-name' in argv[:stop_idx]: idx = argv.index('--driver-name') argv.pop(idx) driver_name = argv[idx] if driver_name.startswith('--'): stderr.write("pyqi driver option --driver-name requires a " "value, e.g. --driver-name mydriver\n") exit(1) argv.pop(idx) stop_idx -= 2 if '--command-config-module' in argv[:stop_idx]: idx = argv.index('--command-config-module') argv.pop(idx) cmd_cfg_mod = argv[idx] if cmd_cfg_mod.startswith('--'): stderr.write("pyqi driver option --command-config-module " "requires a value, e.g. --command-config-module " "my.command.config.module\n") exit(1) argv.pop(idx) stop_idx -= 2 if stop_idx != 1: # We're not pointing at a command name, so there must have been # other stuff that we didn't recognize. stderr.write("Unrecognized pyqi driver option(s): %s\n" % ' '.join(argv[1:stop_idx])) exit(1) argv.pop(stop_idx) command_names = get_command_names(cmd_cfg_mod) if len(argv) == 1: argv[0] = driver_name usage(cmd_cfg_mod, command_names) else: cmd_name = argv[1] if cmd_name.lower() in ['help', '--help', '-?', '-h']: if not len(argv) > 2: argv[0] = driver_name usage(cmd_cfg_mod, command_names) help_cmd = argv[2] assert_command_exists(help_cmd, command_names, driver_name) # tears. # . # this voodoo is to coerce optparse/argparse to dump the program # name at usage and examples correctly. argv[0] = ' '.join([driver_name, help_cmd]) help_(cmd_cfg_mod, help_cmd) else: assert_command_exists(cmd_name, command_names, driver_name) # see the note about crying about tears. argv[0] = ' '.join([driver_name, cmd_name]) cmd_obj = get_cmd_obj(cmd_cfg_mod, cmd_name) # execute FTW optparse_main(cmd_obj, argv[1:]) pyqi-0.2.0/setup.py000066400000000000000000000025161220701352200142030ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Greg Caporaso" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Rob Knight", "Greg Caporaso", "Jai Ram Rideout", "Daniel McDonald", "Doug Wendel"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Greg Caporaso" __email__ = "gregcaporaso@gmail.com" from distutils.core import setup from glob import glob setup(name='pyqi', version=__version__, description='pyqi: expose your interface', author=__maintainer__, author_email=__email__, maintainer=__maintainer__, maintainer_email=__email__, url='http://bipy.github.io/pyqi', packages=['pyqi', 'pyqi/commands', 'pyqi/core', 'pyqi/core/interfaces', 'pyqi/core/interfaces/optparse', 'pyqi/interfaces', 'pyqi/interfaces/optparse', 'pyqi/interfaces/optparse/config', ], scripts=glob('scripts/pyqi*')) pyqi-0.2.0/tests/000077500000000000000000000000001220701352200136275ustar00rootroot00000000000000pyqi-0.2.0/tests/test_commands/000077500000000000000000000000001220701352200164675ustar00rootroot00000000000000pyqi-0.2.0/tests/test_commands/test_code_header_generator.py000066400000000000000000000044421220701352200243740ustar00rootroot00000000000000#!/usr/bin/env python from __future__ import division #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Jai Ram Rideout" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Jai Ram Rideout" __email__ = "jai.rideout@gmail.com" from unittest import TestCase, main from pyqi.commands.code_header_generator import CodeHeaderGenerator class CodeHeaderGeneratorTests(TestCase): def setUp(self): """Set up a CodeHeaderGenerator instance to use in the tests.""" self.cmd = CodeHeaderGenerator() def test_run(self): """Correctly generates header with and without credits.""" obs = self.cmd(author='bob', email='bob@bob.bob', license='very permissive license', copyright='what\'s that?', version='1.0') self.assertEqual(obs.keys(), ['result']) obs = obs['result'] self.assertEqual('\n'.join(obs), exp_header1) # With credits. obs = self.cmd(author='bob', email='bob@bob.bob', license='very permissive license', copyright='what\'s that?', version='1.0', credits=['another person', 'another another person']) self.assertEqual(obs.keys(), ['result']) obs = obs['result'] self.assertEqual('\n'.join(obs), exp_header2) exp_header1 = """#!/usr/bin/env python from __future__ import division __author__ = "bob" __copyright__ = "what's that?" __credits__ = ["bob"] __license__ = "very permissive license" __version__ = "1.0" __maintainer__ = "bob" __email__ = "bob@bob.bob" """ exp_header2 = """#!/usr/bin/env python from __future__ import division __author__ = "bob" __copyright__ = "what's that?" __credits__ = ["bob", "another person", "another another person"] __license__ = "very permissive license" __version__ = "1.0" __maintainer__ = "bob" __email__ = "bob@bob.bob" """ if __name__ == '__main__': main() pyqi-0.2.0/tests/test_commands/test_make_bash_completion.py000066400000000000000000000075101220701352200242460ustar00rootroot00000000000000#!/usr/bin/env python from __future__ import division #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- import sys from os import mkdir from os.path import dirname, join from shutil import copy, rmtree from tempfile import mkdtemp from unittest import TestCase, main import pyqi from pyqi.commands.make_bash_completion import BashCompletion, _get_cfg_module __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Jai Ram Rideout", "Doug Wendel", "Greg Caporaso"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" class BashCompletionTests(TestCase): def setUp(self): """Set up data for unit tests.""" self.cmd = BashCompletion() # Create a temporary directory that we can copy two config files into. # This module can then be passed to BashCompletion so that we don't # have to update the expected output each time we make an interface # change to pyqi. The temporary directory/module structure looks like # the following: # # / # pyqi_test/ # __init__.py # make_bash_completion.py # make_optparse.py # # Note that is added to sys.path in setUp and then removed # in tearDown. Imports can then function as normal, e.g.: # # import pyqi_test.make_bash_completion # ... # self.temp_module_dir = mkdtemp() self.temp_module_name = 'pyqi_test' self.temp_config_dir = join(self.temp_module_dir, self.temp_module_name) mkdir(self.temp_config_dir) module = _get_cfg_module('pyqi.interfaces.optparse.config') module_dir = dirname(module.__file__) make_bash_completion_fp = join(module_dir, 'make_bash_completion.py') copy(make_bash_completion_fp, self.temp_config_dir) make_optparse_fp = join(module_dir, 'make_optparse.py') copy(make_optparse_fp, self.temp_config_dir) with open(join(self.temp_config_dir, '__init__.py'), 'w') as f: f.write('') sys.path.append(self.temp_module_dir) def tearDown(self): sys.path.remove(self.temp_module_dir) rmtree(self.temp_module_dir) def test_get_cfg_module(self): self.assertEqual(_get_cfg_module('pyqi'), pyqi) def test_run(self): params = {'command_config_module':self.temp_module_name, 'driver_name':'pyqi'} obs = self.cmd(**params) self.assertEqual(obs.keys(), ['result']) self.assertEqual(obs['result'], outputandstuff) outputandstuff = """_pyqi_complete() { local cur prev COMPREPLY=() cur=${COMP_WORDS[COMP_CWORD]} prev=${COMP_WORDS[COMP_CWORD-1]} if [ $COMP_CWORD -gt 1 ]; then prev=${COMP_WORDS[1]} fi if [ $COMP_CWORD -eq 1 ]; then COMPREPLY=( $(compgen -W "make-bash-completion make-optparse" -- $cur) ) elif [ $COMP_CWORD -gt 1 ]; then case "$prev" in "make-bash-completion") COMPREPLY=( $(compgen -W "--command-config-module --driver-name --output-fp" -- $cur) ) ;; "make-optparse") COMPREPLY=( $(compgen -W "--author --command --command-module --config-version --copyright --credits --email --license --output-fp" -- $cur) ) ;; *) ;; esac fi return 0 } && complete -F _pyqi_complete -f pyqi """ if __name__ == '__main__': main() pyqi-0.2.0/tests/test_commands/test_make_command.py000066400000000000000000000065051220701352200225210ustar00rootroot00000000000000#!/usr/bin/env python from __future__ import division #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Jai Ram Rideout" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Jai Ram Rideout" __email__ = "jai.rideout@gmail.com" from unittest import TestCase, main from pyqi.commands.make_command import MakeCommand class MakeCommandTests(TestCase): def setUp(self): """Set up a MakeCommand instance to use in the tests.""" self.cmd = MakeCommand() def test_run_command_code_generation(self): """Correctly generates stubbed out Command code.""" obs = self.cmd(name='Test', author='bob', email='bob@bob.bob', license='very permissive license', copyright='what\'s that?', version='1.0') self.assertEqual(obs.keys(), ['result']) obs = obs['result'] self.assertEqual('\n'.join(obs), exp_command_code1) def test_run_test_code_generation(self): """Correctly generates stubbed out unit test code.""" obs = self.cmd(name='Test', author='bob', email='bob@bob.bob', license='very permissive license', copyright='what\'s that?', version='1.0', credits=['another person'], test_code=True) self.assertEqual(obs.keys(), ['result']) obs = obs['result'] self.assertEqual('\n'.join(obs), exp_test_code1) exp_command_code1 = """#!/usr/bin/env python from __future__ import division __author__ = "bob" __copyright__ = "what's that?" __credits__ = ["bob"] __license__ = "very permissive license" __version__ = "1.0" __maintainer__ = "bob" __email__ = "bob@bob.bob" from pyqi.core.command import Command, Parameter, ParameterCollection class Test(Command): BriefDescription = "FILL IN A 1 SENTENCE DESCRIPTION" LongDescription = "GO INTO MORE DETAIL" Parameters = ParameterCollection([ Parameter(Name='foo', DataType=str, Description='some required parameter', Required=True), Parameter(Name='bar', DataType=int, Description='some optional parameter', Required=False, Default=1) ]) def run(self, **kwargs): # EXAMPLE: # return {'result_1': kwargs['foo'] * kwargs['bar'], # 'result_2': "Some output bits"} raise NotImplementedError("You must define this method") CommandConstructor = Test""" exp_test_code1 = """#!/usr/bin/env python from __future__ import division __author__ = "bob" __copyright__ = "what's that?" __credits__ = ["bob", "another person"] __license__ = "very permissive license" __version__ = "1.0" __maintainer__ = "bob" __email__ = "bob@bob.bob" from unittest import TestCase, main from FILL IN MODULE PATH import Test class TestTests(TestCase): def setUp(self): self.cmd_obj = Test() def test_run(self): self.fail() if __name__ == '__main__': main()""" if __name__ == '__main__': main() pyqi-0.2.0/tests/test_commands/test_make_optparse.py000066400000000000000000000154351220701352200227420ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division from pyqi.commands.make_optparse import MakeOptparse from pyqi.core.command import Parameter, ParameterCollection from unittest import TestCase, main __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Jai Ram Rideout", "Doug Wendel", "Greg Caporaso"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" class MakeOptparseTests(TestCase): def setUp(self): self.cmd = MakeOptparse() def test_run(self): exp = win_text pc = Parameter(Name='DUN', Required=True, DataType=str, Description="") bool_param = Parameter(Name='imabool', DataType=bool, Description='zero or one', Required=False) class stubby: Parameters = ParameterCollection([pc, bool_param]) obs = self.cmd(**{'command_module':'foobar', 'command':stubby(), 'author': 'bob', 'email': 'bob@bob.bob', 'license': 'very permissive license', 'copyright': 'what\'s that?', 'version': '1.0' }) self.assertEqual('\n'.join(obs['result']), exp) win_text = """#!/usr/bin/env python from __future__ import division __author__ = "bob" __copyright__ = "what's that?" __credits__ = ["bob"] __license__ = "very permissive license" __version__ = "1.0" __maintainer__ = "bob" __email__ = "bob@bob.bob" from pyqi.core.interfaces.optparse import (OptparseUsageExample, OptparseOption, OptparseResult) from pyqi.core.command import make_parameter_collection_lookup_f from foobar import CommandConstructor # If you need access to input or output handlers provided by pyqi, consider # importing from the following modules: # pyqi.core.interfaces.optparse.input_handler # pyqi.core.interfaces.optparse.output_handler # pyqi.interfaces.optparse.input_handler # pyqi.interfaces.optparse.output_handler # Convenience function for looking up parameters by name. param_lookup = make_parameter_collection_lookup_f(CommandConstructor) # Examples of how the command can be used from the command line using an # optparse interface. usage_examples = [ OptparseUsageExample(ShortDesc="A short single sentence description of the example", LongDesc="A longer, more detailed description", Ex="%prog --foo --bar some_file") ] # inputs map command line arguments and values onto Parameters. It is possible # to define options here that do not exist as parameters, e.g., an output file. inputs = [ # An example option that has a direct relationship with a Parameter. # OptparseOption(Parameter=param_lookup('name_of_a_parameter'), # InputType='existing_filepath', # the optparse type of input # InputAction='store', # the optparse action # InputHandler=None, # Apply a function to the input value to convert it into the type expected by Parameter.DataType # ShortName='n', # a parameter short name, can be None # # Name='foo', # implied by Parameter.Name. Can be overwritten here if desired # # Required=False, # implied by Parameter.Required. Can be promoted by setting True # # Help='help', # implied by Parameter.Description. Can be overwritten here if desired # # Default=None, # implied by Parameter.Default. Can be overwritten here if desired # # DefaultDescription=None, # implied by Parameter.DefaultDescription. Can be overwritten here if desired # convert_to_dashed_name=True), # whether the Name (either implied by Parameter or defined above) should have underscores converted to dashes when displayed to the user # # An example option that does not have an associated Parameter. # OptparseOption(Parameter=None, # InputType='new_filepath', # InputAction='store', # InputHandler=None, # we don't need an InputHandler because this option isn't being converted into a format that a Parameter expects # ShortName='o', # Name='output-fp', # Required=True, # Help='output filepath') OptparseOption(Parameter=param_lookup('DUN'), InputType=, InputAction='store', # default is 'store', change if desired InputHandler=None, # must be defined if desired ShortName=None), # must be defined if desired # Name='DUN', # implied by Parameter # Required=True, # implied by Parameter # Help='', # implied by Parameter OptparseOption(Parameter=param_lookup('imabool'), InputType=None, InputAction='store_true', # default is 'store', change if desired InputHandler=None, # must be defined if desired ShortName=None), # must be defined if desired # Name='imabool', # implied by Parameter # Required=False, # implied by Parameter # Help='zero or one', # implied by Parameter # Default=None, # implied by Parameter # DefaultDescription=None, # implied by Parameter ] # outputs map result keys to output options and handlers. It is not necessary # to supply an associated option, but if you do, it must be an option from the # inputs list (above). outputs = [ # An example option that maps to a result key. # OptparseResult(ResultKey='some_result', # OutputHandler=write_string, # a function applied to the value at ResultKey # # # the name of the option (defined in inputs, above), whose # # value will be made available to OutputHandler. This name # # can be either an underscored or dashed version of the # # option name (e.g., 'output_fp' or 'output-fp') # OptionName='output-fp'), # # An example option that does not map to a result key. # OptparseResult(ResultKey='some_other_result', # OutputHandler=print_string) ]""" if __name__ == '__main__': main() pyqi-0.2.0/tests/test_core/000077500000000000000000000000001220701352200156165ustar00rootroot00000000000000pyqi-0.2.0/tests/test_core/test_command.py000066400000000000000000000106511220701352200206500ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from unittest import TestCase, main from pyqi.core.command import Parameter, ParameterCollection, Command from pyqi.core.exception import (IncompetentDeveloperError, UnknownParameterError, MissingParameterError) class CommandTests(TestCase): def setUp(self): class stubby(Command): Parameters = ParameterCollection([ Parameter('a',int,'', Required=True), Parameter('b',int,'', Required=False, Default=5), Parameter('c',int,'', Required=False, Default=10, ValidateValue=lambda x: x == 10)]) def run(self, **kwargs): return {} self.stubby = stubby def test_init(self): """Jog the init""" c = Command() self.assertEqual(len(c.Parameters), 0) with self.assertRaises(NotImplementedError): _ = c() def test_subclass_init(self): """Exercise the subclassing""" class foo(Command): Parameters = ParameterCollection([Parameter('a', str, 'help1', Required=True), Parameter('b', str, 'help2', Required=False)]) def run(self, **kwargs): return {} obs = foo() self.assertEqual(len(obs.Parameters), 2) self.assertEqual(obs.run(bar={'a':10}), {}) def test_validate_kwargs(self): stub = self.stubby() kwargs = {'a':10, 'b':20} # should work stub._validate_kwargs(kwargs) kwargs = {'b':20} self.assertRaises(MissingParameterError, stub._validate_kwargs, kwargs) kwargs = {'a':10, 'b':20, 'c':10} stub._validate_kwargs(kwargs) kwargs = {'a':10, 'b':20, 'c':20} self.assertRaises(ValueError, stub._validate_kwargs, kwargs) def test_set_defaults(self): stub = self.stubby() kwargs = {'a':10} exp = {'a':10,'b':5,'c':10} stub._set_defaults(kwargs) self.assertEqual(kwargs, exp) class ParameterTests(TestCase): def test_init(self): """Jog the init""" obj = Parameter('a', str, 'help', Required=False) self.assertEqual(obj.Name, 'a') self.assertEqual(obj.DataType, str) self.assertEqual(obj.Description, 'help') self.assertEqual(obj.Required, False) self.assertEqual(obj.Default, None) self.assertEqual(obj.DefaultDescription, None) self.assertRaises(IncompetentDeveloperError, Parameter, 'a', str, 'help', True, 'x') class ParameterCollectionTests(TestCase): def setUp(self): self.pc = ParameterCollection([Parameter('foo',str, 'help')]) def test_init(self): """Jog the init""" params = [Parameter('a', str, 'help', Required=False), Parameter('b', float, 'help2', Required=True)] obj = ParameterCollection(params) self.assertEqual(obj.Parameters, params) self.assertEqual(obj['a'], params[0]) self.assertEqual(obj['b'], params[1]) # Duplicate Parameter names. params.append(Parameter('a', int, 'help3')) with self.assertRaises(IncompetentDeveloperError): _ = ParameterCollection(params) def test_getitem(self): self.assertRaises(UnknownParameterError, self.pc.__getitem__, 'bar') self.assertEqual(self.pc['foo'].Name, 'foo') # make sure we can getitem def test_setitem(self): self.assertRaises(TypeError, self.pc.__setitem__, 'bar', 10) if __name__ == '__main__': main() pyqi-0.2.0/tests/test_core/test_factory.py000066400000000000000000000001301220701352200206700ustar00rootroot00000000000000#!/usr/bin/env python # currently tested in test_interfaces/test_optparse/test_init.py pyqi-0.2.0/tests/test_core/test_interface.py000066400000000000000000000040171220701352200211710ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- from __future__ import division __author__ = "Jai Ram Rideout" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Jai Ram Rideout" __email__ = "jai.rideout@gmail.com" from unittest import TestCase, main from pyqi.core.interface import get_command_names, get_command_config import pyqi.interfaces.optparse.config.make_bash_completion class TopLevelTests(TestCase): def test_get_command_names(self): """Test that command names are returned from a config directory.""" exp = ['make-bash-completion', 'make-command', 'make-optparse'] obs = get_command_names('pyqi.interfaces.optparse.config') self.assertEqual(obs, exp) # Invalid config dir. with self.assertRaises(ImportError): _ = get_command_names('foo.bar.baz-aar') def test_get_command_config(self): """Test successful and unsuccessful module loading.""" cmd_cfg, error_msg = get_command_config( 'pyqi.interfaces.optparse.config', 'make_bash_completion') self.assertEqual(cmd_cfg, pyqi.interfaces.optparse.config.make_bash_completion) self.assertEqual(error_msg, None) cmd_cfg, error_msg = get_command_config( 'hopefully.nonexistent.python.module', 'umm', exit_on_failure=False) self.assertEqual(cmd_cfg, None) self.assertEqual(error_msg, 'No module named hopefully.nonexistent.' 'python.module.umm') if __name__ == '__main__': main() pyqi-0.2.0/tests/test_core/test_interfaces/000077500000000000000000000000001220701352200210005ustar00rootroot00000000000000pyqi-0.2.0/tests/test_core/test_interfaces/test_optparse/000077500000000000000000000000001220701352200236745ustar00rootroot00000000000000pyqi-0.2.0/tests/test_core/test_interfaces/test_optparse/test_init.py000066400000000000000000000132421220701352200262520ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Greg Caporaso", "Daniel McDonald", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.2.0" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from unittest import TestCase, main from pyqi.core.interfaces.optparse import (OptparseResult, OptparseOption, OptparseUsageExample, OptparseInterface, optparse_factory, optparse_main) from pyqi.core.exception import IncompetentDeveloperError from pyqi.core.command import Command, Parameter, ParameterCollection class OptparseResultTests(TestCase): # Nothing to test yet pass class OptparseOptionTests(TestCase): def setUp(self): p = Parameter('number', int, 'some int', Required=False, Default=42, DefaultDescription='forty-two') # With associated parameter. self.opt1 = OptparseOption(p, int) # Without associated parameter. self.opt2 = OptparseOption(None, int, InputHandler=None, ShortName='n', Name='number', Required=False, Help='help!!!') def test_init(self): self.assertEqual(self.opt1.InputType, int) self.assertEqual(self.opt1.Help, 'some int') self.assertEqual(self.opt1.Name, 'number') self.assertEqual(self.opt1.Default, 42) self.assertEqual(self.opt1.DefaultDescription, 'forty-two') self.assertEqual(self.opt1.ShortName, None) self.assertEqual(self.opt1.Required, False) def test_str(self): exp = '--number' obs = str(self.opt1) self.assertEqual(obs, exp) exp = '-n/--number' obs = str(self.opt2) self.assertEqual(obs, exp) class OptparseUsageExampleTests(TestCase): def test_init(self): obj = OptparseUsageExample(ShortDesc='a', LongDesc='b', Ex='c') self.assertEqual(obj.ShortDesc, 'a') self.assertEqual(obj.LongDesc, 'b') self.assertEqual(obj.Ex, 'c') with self.assertRaises(IncompetentDeveloperError): _ = OptparseUsageExample('a', 'b', Ex=None) def oh(key, data, opt_value=None): return data * 2 class OptparseInterfaceTests(TestCase): def setUp(self): self.interface = fabulous() def test_init(self): self.assertRaises(IncompetentDeveloperError, OptparseInterface) def test_validate_usage_examples(self): with self.assertRaises(IncompetentDeveloperError): _ = NoUsageExamples() # TODO: this test should be migrated to tests for the Interface baseclass # if/when they exist. def test_validate_inputs(self): with self.assertRaises(IncompetentDeveloperError): _ = DuplicateOptionMappings() def test_input_handler(self): obs = self.interface._input_handler(['--c','foo']) self.assertEqual(obs.items(), [('c', 'foo')]) def test_build_usage_lines(self): obs = self.interface._build_usage_lines([]) self.assertEqual(obs, usage_lines) def test_output_handler(self): results = {'itsaresult':20} obs = self.interface._output_handler(results) self.assertEqual(obs, {'itsaresult':40}) class GeneralTests(TestCase): def setUp(self): self.obj = optparse_factory(ghetto, [OptparseUsageExample('a','b','c')], [OptparseOption(InputType=str, Parameter=ghetto.Parameters['c'], ShortName='n')], [OptparseResult(ResultKey='itsaresult', OutputHandler=oh)], '2.0-dev') def test_optparse_factory(self): # exercise it _ = self.obj() def test_optparse_main(self): # exercise it _ = optparse_main(self.obj, ['testing', '--c', 'bar']) class ghetto(Command): Parameters = ParameterCollection([Parameter('c', str, 'b')]) def run(self, **kwargs): return {'itsaresult':10} class fabulous(OptparseInterface): CommandConstructor = ghetto def _get_inputs(self): return [OptparseOption(InputType=str, Parameter=self.CommandConstructor.Parameters['c'], ShortName='n')] def _get_usage_examples(self): return [OptparseUsageExample('a','b','c')] def _get_outputs(self): return [OptparseResult(ResultKey='itsaresult', OutputHandler=oh)] def _get_version(self): return '2.0-dev' # Doesn't have any usage examples... class NoUsageExamples(fabulous): def _get_usage_examples(self): return [] # More than one option mapping to the same Parameter... class DuplicateOptionMappings(fabulous): def _get_inputs(self): return [ OptparseOption(InputType=str, Parameter=self.CommandConstructor.Parameters['c'], ShortName='n'), OptparseOption(Parameter=self.CommandConstructor.Parameters['c'], Name='i-am-a-duplicate') ] usage_lines = """usage: %prog [options] {} [] indicates optional input (order unimportant) {} indicates required input (order unimportant) Example usage: Print help message and exit %prog -h a: b c""" if __name__ == '__main__': main() pyqi-0.2.0/tests/test_core/test_interfaces/test_optparse/test_input_handler.py000066400000000000000000000021611220701352200301410ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.1-dev" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" from unittest import TestCase, main from pyqi.core.interfaces.optparse.input_handler import command_handler from pyqi.commands.make_optparse import MakeOptparse class OptparseInputHandlerTests(TestCase): def setUp(self): pass def test_command_handler(self): exp = MakeOptparse() obs = command_handler('pyqi.commands.make_optparse.MakeOptparse') self.assertEqual(type(obs), type(exp)) if __name__ == '__main__': main() pyqi-0.2.0/tests/test_core/test_interfaces/test_optparse/test_output_handler.py000066400000000000000000000053071220701352200303470ustar00rootroot00000000000000#!/usr/bin/env python #----------------------------------------------------------------------------- # Copyright (c) 2013, The BiPy Development Team. # # Distributed under the terms of the Modified BSD License. # # The full license is in the file COPYING.txt, distributed with this software. #----------------------------------------------------------------------------- __author__ = "Daniel McDonald" __copyright__ = "Copyright 2013, The pyqi project" __credits__ = ["Daniel McDonald", "Greg Caporaso", "Doug Wendel", "Jai Ram Rideout"] __license__ = "BSD" __version__ = "0.1-dev" __maintainer__ = "Daniel McDonald" __email__ = "mcdonadt@colorado.edu" import os import sys from StringIO import StringIO from shutil import rmtree from tempfile import mkdtemp from unittest import TestCase, main from pyqi.core.interfaces.optparse.output_handler import (write_string, write_list_of_strings, print_list_of_strings) from pyqi.core.exception import IncompetentDeveloperError class OutputHandlerTests(TestCase): def setUp(self): self.output_dir = mkdtemp() self.fp = os.path.join(self.output_dir, 'test_file.txt') def tearDown(self): rmtree(self.output_dir) def test_write_string(self): """Correctly writes a string to file.""" # can't write without a path self.assertRaises(IncompetentDeveloperError, write_string, 'a','b') write_string('foo', 'bar', self.fp) with open(self.fp, 'U') as obs_f: obs = obs_f.read() self.assertEqual(obs, 'bar\n') def test_write_list_of_strings(self): """Correctly writes a list of strings to file.""" # can't write without a path self.assertRaises(IncompetentDeveloperError, write_list_of_strings, 'a', ['b', 'c']) write_list_of_strings('foo', ['bar', 'baz'], self.fp) with open(self.fp, 'U') as obs_f: obs = obs_f.read() self.assertEqual(obs, 'bar\nbaz\n') def test_print_list_of_strings(self): """Correctly prints a list of strings.""" # Save stdout and replace it with something that will capture the print # statement. Note: this code was taken from here: # http://stackoverflow.com/questions/4219717/how-to-assert-output- # with-nosetest-unittest-in-python/4220278#4220278 saved_stdout = sys.stdout try: out = StringIO() sys.stdout = out print_list_of_strings('this is ignored', ['foo', 'bar', 'baz']) exp = 'foo\nbar\nbaz\n' obs = out.getvalue() self.assertEqual(obs, exp) finally: sys.stdout = saved_stdout if __name__ == '__main__': main()