PyRRD-0.1.0/000755 000765 000024 00000000000 11635462441 013443 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/ChangeLog000644 000765 000024 00000031602 11635456135 015222 0ustar00oubiwannstaff000000 000000 2009.12.19 * Updated the mapper map function to check for the mode: in read mode, it sucks in XML data; in write mode, it uses what's already been written to the objects. This fixes issue #18. * Added support for catching command-line errors (Modified fix from Joseph Heck). This fixes issue #19. * Added missing exceptions module as well as backend tests. * Moved the error unit test into the external test case, where it belongs. * Removed redunant ElementTree import check, pulling instead from the util module. 2009.12.18 * Added a placeholder for native Python parsing of RRD file data. * Added a sandbox with some files for exploring Python-native RRD-read support. * Updated some doctests that were too sensive to version differences (point brought up by Joseph Heck). * Added more checks for existence in the doctests (Joseph Heck). * Updated acknowledgements. * Changed the DSs and RRAs in the mapper to extend, not append (Fix provided by Joseph Heck). * Updated the mappers to not use class attributes for lists (should fix issue #18, but that still needs to be tested). * Reverted the append->extend change. * Changed the way that the mappers stored their ds and rra data. * Added an XML dump file for testing. * Added unit tests for the XML dumps. * Enabled the RRA XML node to get the xff value by name. * Added unit tests for the RRD Mapper. * Moved the NaN class into the util module. * Moved a chunk of the logic from the external backend info function into the RRDMapper.printInfo method. 2009.10.29 * Added a first draft of a NaN fix for Windows users. * Applied partial patch from Denis Fortin (the part of the patch not included was a Linux-incompatible change for NaNs). * Changed a filename check in the doctest, due to the patch. * Fixed the doctests that were broken after the patch. * Updated pyrrd.testing.suite to do more widespread and intelligent optional test skipping. * Added an option parser to the test running so that users may not indicate verbosity level and skip files from the command line. * Added a base unittest test case. 2009.10.27 * Fixed failing epoch test. * Sync'ed Launchpad and Google repos. * Updated TODO. * Fixed buildParameters in backend.common. * Removed unused backend.external.buildParameters function. * Fixed parameters in external backend. * Fixed failing test in bindings backend. * Fixed graph formatting. * More external backend fixes. * Fixed unit tests in rrd module. * More unit test cleanups. * Added testing utilities. * Added new test runner that using the new testing module. * Removed old test runners. * Added unit tests for RRD isntantiation. * Tweaked test method names. 2009.07.02 * Added a test file for pushing up to launchpad. 2009.07.01 * Updated TODO. * Updated the admin defs to use the right google username. * Applied a fix for graphs on Windows machines (colons in filenames causing problems); patch supplied by Denis Fortin. * Applied a patch from Denis Fortin for letting Popen handle spaces in paramaters. * Applied a fix for an issue that Denis Fortin opened a ticket for regarding boolean flags not being handled properly. * Fixed the spelling on Denis' name! 2009.03.30 * Added update function to bindings wrapper. * Stubbed out fetch, dump, and load functions. * Added a graph function to the bindings wrapper. * Added dump and load functions to the bindings wrapper. * Added fetch wrapper for bindings. * Completely rearranged the info method/function/dispatch for symmetry between the two backends. * Added an info wrapper for bindings. * Finished touching up the bindings dispatcher. * Added doctests for bindings-enabled RRD to the bindings module. * Updated the README with a note about using the Graph class with the Python bindings. * Cleaned up an outdated docstring. Updated the "Features" and "Introduction" sections of the README. * Fixed typo in kwds dict (Mladen Milankovic). * Updated acknowledgements. 2009.03.29 * Created a branch for doing the python RRDtool bindings integration. * Added admin scripts for convenient branch management. * Updated TODO. * Added doctest for buildParameters. * Updated the RRD class to take a backend parameter in the constructor and modified the class to reference this assigned attribute in the rest of its methods. * Started putting together the bindings module. * Cleaned up the part of the README that has the new bindings instructions in it. * Updated the Graph class to take a backend parameter in its constructor. * Updated the README to include how to use graph.Graph with the python bindings backend. * Renamed backend.py to common.py for a container of backend functions that are common to both types. * Moved coerce and iterparse into common. * Cleaned up imports. * Fixed bindings.buildParameters and moved it into common. * Updated external.buildParameters to use the one from common. 2009.03.26 * Moved meta data out of setup.py and into pyrrd.meta. * Added docs directory and split docs out of README and added more info. * Moved pyrrd/utils.py to pyrrd/util/__init__.py. * Added pyrrd.util.dist. * Updated setup.py to use meta as well as the new dist utility. * Moved INSTALL content into README. * Moved acknowledgements out of README and into their own file. * Added file cleanups in doctests. * Updated MANIFEST to include new docs and removed references to files that no longer exist. * Tweaked setup.py and dist module. * Cleaned up parts of README that were causing problems in ReST rendering on PyPI. * Created an uploader for PyPI dists. 2009.03.23 * Added a file manifest so that files don't get left out during build process. * Updated setup.py. * Updated TODO. * Incremented microversion to 0.0.6 since this is a bug fix (manifest file). * Really incremented version. 2009.03.21 * Received the following fixes from Aaron Westendorf of Agora Games: - Moved pyrrd.xml to pyrrd.__xml to not conflict with stdlib xml namespace. - Fixed loading of existing RRDs when expected attributes are not there, specifically when RRAs are for Holt-Winters. * Renamed __xml to node. * Updated unit tests. * Added missing docstrings. * Formatted contributed code. * Removed the Agora notes file * Updated version. * Added a dist-building script. * Updated the clean script. * Incremented microversion number to 0.0.5 (committed this change earlier). * Updated acknowledgements section in README. * Updated install instructions with a note about a conditional dependency. 2009.01.31 * Removed ez_setup dependence. * Fixed old parameter name in example. * Added shell script for testing example code. * Updated TODO. * Updated the RRD class so that manual calls to load are not necessary if an RRD instance is created with the mode="r" parameter. * Updated TODO. 2009.01.30 * Changed the rrd.* classes to inherit from the mapper.* classes. * Added last update tracking. * Updated RRD.load to use the mapper map method. * Cleaned up some variable names and comments. * Added missing mapper module. * Added printInfo method to the base mapper class. * Updated the RRD.info method to use the printInfo mapper method. * Added support for printing info of loaded RRD files. * Updated TODO. * Updated example4 to use current times instead of the times for when the examples were originally written (2005). * Added a string coercion before doing a split on dateOrData. * Removed old docstrings. * Improved getData mapper methods to descend into sub-objects. * Improved the output format of the external fetch function. * Fixed the external fetch and made the output format more usable. * Added support for a fetch method on the RRD class. 2009.01.29 * Updated the TODO with thoughts/plans for implementing an info method. * Started stubbing out the code that will support loading data from files (and thus an info method). * Added a reference directory with the dump dtd in it. * Added acknowledgements to README. * Removed the testing sandbox in trunk. * Renamed "read" to "load". * Added dump function with doctests. * Added working code and doctests for loading an RRD XML dump file as an ElementTree instance. * Added an XML module for extracting data from RRD dumps. * Removed stubbed mapper objects from rrd module. * Added mapper module for mapping ElementTree XML nodes to Python objects with attributes. * With the use of slots, removed __dict__ references. 2009.01.28 * Updated the old admin scripts to use bazaar. * Added more scripts, including definitions for pushing back up to Google Code (Subversion). * Added a fix for Windows users. Fixes: http://code.google.com/p/pyrrd/issues/detail?id=1 * Cleaned up the backend module a tiny bit. * White space cleanup. * Added a fix for named parameters in the DataSource class. Fixes: http://code.google.com/p/pyrrd/issues/detail?id=2 * Added Google Code to the admin push script. * Fixed missing named parameter in RRA class. * Fixed missing named parameter in RRD class. * Added a NotImplementedError for the fetch method. * Updated TODO. * Added missing named parameters in VariableDefinition class. * Added missing named parameters in CalculationDefinition class. * Updated test runner. * Updated README with links. * Updated commit script to run test runner. * Added project resource files (images). * Updated the TODO. * Fixed a typo. * Incremented micro version number. * Added placeholder for info implementation. * Updated formatting. 2008.04.11 * Moved sandbox. * Created admin directory. * Moved stat script into admin directory. * Added commit script. * Fixed date in ChangeLog. * Commented out sync code in commit.sh. * Renamed lib directory to pyrrd for easier use with non-installed environments. * Incremented micro version number. * Added test runner. * Fixed bad unit test result in pyrrd.utils. * Adjusted stat.sh script. * Updated setup.py. * Updated date in LICENSE. * Updated vars in example 1. * Fixed var names in rrd.external. * Renamed generateResultLines to iterParse and added some unit tests. * Cleaned up testmod call. * Cleaned up vars in rrd and graph modules. * Fixed example1 to accord with var name changes. * Fixed a missed var in rrd.DataSource, a test in graph, and updated example2. * Updated example3. * Updated example4. * Updated example5. * Updated TODO, based on email feedback from Markus Juenemann. 2006.01.29 * Added code to example5.py for generating RRD files and graphs from the sample data. * Updated the commends in example5.py and extended the 3-day archive to a 7-day archive. * Corrected the vname in cdef1 of example5.py. 2006.01.28 * Added a couple doctests to the utils module. * Added template support to rrd.RRD.update(). * Added a data file for the 5th example. * Extended the flexibility of rrd.RRD.bufferValue() such that it can now take a single parameter, and if so, that parameter will be considered a "data chunk", i.e., a group of one or more time:data strings. * Added missing step attribute in rrd.RRD.__init__(). 2006.01.24 * Setup all graph.Graph parameters to be initialized. * Added code for graph.Print and graph.GraphPrint. * Added code for graph.Comment. * Added an examples directory. * Added code for graph.Line * Added code for graph.Area. * Finised graph.Graph. * Added graph.ColorAttributes class for easy OOP-configuration of a graph's overall color attributes. * As a first example, added the graph-generating code from the doctests. * Split the first example into two in order to show the color differences as a separate example. * Added a third example. * Updated rrd.RRD.bufferValue() to handle multiple values per line (for instances where more than one DS is getting updated in an RRD). * Added debug options to graph.Graph.write() and rrd.RRD.update(). * Added a fourth example with a little more math for additional data sets. * Added INSTALL and README files. * Removed cruft from sandbox and removed "old" from lib. * Updated to newest ez_setup.py. 2006.01.23 * Added code for VDEF. * Added TODO. * Added code for CDEF. * Created external.py for using the command-line rrdtool binary. * Added create, update and fetch (with doctests) to external.py. * Moved old files into an 'old' dir. * Added graph (with doctests) to external.py. * Added a place-holder file for supporting the rrdtool python bindings. * Added a backends.py module for selecting which backend to use. * Added an rrd.py module for abstracting the database aspect of RRD. * Added a utils module with a funtion for getting the epoch date of a datetime object. * Finished most of rrd.py. * Added LICENSE, VERSION, setup.py and ez_setup.py. 2006.01.22 * Added old rrdtool sandbox code from CoyMon repository. * Added unfinished CoyMon rrd lib dir to PyRRD lib dir. * Added ChangeLog. * Added some basic library files with stubbs. * Added most of the class documentation for pyrrd.graph.* and the beginnings of some implementation. * Created graph.validateVName and graph.GraphDefinition, a.k.a. graph.DEF and added doctests to each. * Added escaping for colons in timedata. * Added checks for required attributes. [From the CoyMon ChangeLog] 2004.11.01 * Reorganized RRD libs in adytum.app.coymon.rrd. * Copied the RRD wrapper in my sandbox to adytum.app.coymon.rrd.graphs. PyRRD-0.1.0/docs/000755 000765 000024 00000000000 11635462441 014373 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/examples/000755 000765 000024 00000000000 11635462441 015261 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/LICENSE000644 000765 000024 00000003000 11635456135 014444 0ustar00oubiwannstaff000000 000000 Copyright (c) 2004-2008, AdytumSolutions, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of AdytumSolutions, Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. PyRRD-0.1.0/MANIFEST.in000644 000765 000024 00000000237 11635456135 015206 0ustar00oubiwannstaff000000 000000 recursive-include pyrrd *.py recursive-include examples *.py *.png recursive-include docs *.txt include ChangeLog include README include LICENSE include TODO PyRRD-0.1.0/PKG-INFO000644 000765 000024 00000037234 11635462441 014551 0ustar00oubiwannstaff000000 000000 Metadata-Version: 1.0 Name: PyRRD Version: 0.1.0 Summary: An Object-Oriented Python Interface for RRDTool Home-page: http://code.google.com/p/pyrrd/ Author: Duncan McGreggor Author-email: duncan@canonical.com License: BSD Description: ~~~~~ PyRRD ~~~~~ .. contents:: :depth: 1 ======== Features ======== PyRRD lets you use RRDTool from Python code that takes advantage of standard object-oriented patterns. The makes the programmatic usage of RRDTool much easier and reusable. A quick review of features is available at the project wiki [#]_ . Example code with graph image output is also available on the wiki [#]_ . ============ Introduction ============ PyRRD is an object-oriented wrapper for the command line graphing and round-robin database utility, rrdtool [#]_ . PyRRD originally had two design goals: 1. provide an interface to rrdtool that Python programmers would love, and 2. not depend upon the Python bindings for rrdtool. The reasons for the former are obvious. The motivation for the latter were the many people who had difficulty compiling the rrdtool bindings on their operating system of choice. Even though PyRRD's original purpose was to help those without the bingins, the project now offers support for those with the bindings installed. As such, users may enjoy both the speed benefits from the bindings as well as the API usability from PyRRD. For docs, see the docstrings at the beginning of each class (and many of the functions). They not only contain many of the standard RRDTool docs, but they contain doctests which give you a hands-on, how-it-works understanding of actual usage. For those curious about the motivation for creating PyRRD, perhaps some background would be in order. Originally, there were only two ways to use RRDTool from Python: 1. using the Python bindings which were difficult to compile and use, or 2. making system calls to the "rrdtool" executable from Python, passing it all the parameters it needed. Option #1 was often difficult or impossible for many folks to get running on their preferred operating system. But even if one was able to compile it and run it, usage was very cumbersome and designed to work like the command line tool and the C interface, not like most people typically use Python. Now, with PyRRD, there are two additional ways to use RRDTool from Python: 3. an object-oriented interface that wraps system calls (Popen) to the rrdtool binary, and 4. the same object-oriented interface that wraps the cumbersome rrdtool Python bindings. ============ Dependencies ============ Some parts of PyRRD make use of ElementTree for XML processing. If you have Python 2.5 or greater, PyRRD will use xml.etree. If your Python version is less than 2.5 and you want to use features that depend on XML processing (such as dump function and the fetch/info methods), you will need to install the ElementTree library [#]_ . ============ Installation ============ PyRRD is installed in the usual way:: python setup.py install You may also use PyRRD without installing it as long as you have ./ in your PYTHONPATH and you are in the top-level directory (which has the pyrrd child directory). ===== Usage ===== Create an RRD file programmatically:: >>> from pyrrd.rrd import DataSource, RRA, RRD >>> filename = '/tmp/test.rrd' >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) >>> myRRD = RRD( ... filename, ds=dataSources, rra=roundRobinArchives, start=920804400) >>> myRRD.create() Let's check to see that the file exists:: >>> import os >>> os.path.isfile(filename) True Let's see how big it is (depending upon RRDTool version, the byte count can change, so we'll just get a general sense):: >>> bytes = len(open(filename).read()) >>> 800 < bytes < 1200 True In order to save writes to disk, PyRRD buffers values and then writes the values to the RRD file at one go:: >>> myRRD.bufferValue('920805600', '12363') >>> myRRD.bufferValue('920805900', '12363') >>> myRRD.bufferValue('920806200', '12373') >>> myRRD.bufferValue('920806500', '12383') >>> myRRD.update() Let's add some more data:: >>> myRRD.bufferValue('920806800', '12393') >>> myRRD.bufferValue('920807100', '12399') >>> myRRD.bufferValue('920807400', '12405') >>> myRRD.bufferValue('920807700', '12411') >>> myRRD.bufferValue('920808000', '12415') >>> myRRD.bufferValue('920808300', '12420') >>> myRRD.bufferValue('920808600', '12422') >>> myRRD.bufferValue('920808900', '12423') >>> myRRD.update() If you're curious, you can take a look at your rrd file with the following:: myRRD.info() The output of that isn't printed here, 'cause it take up too much space. However, it is very similar to the output of the similarly named rrdtool command. In order to create a graph, we'll need some data definitions. We'll also throw in some calculated definitions and variable definitions for good meansure:: >>> from pyrrd.graph import DEF, CDEF, VDEF, LINE, AREA, GPRINT >>> def1 = DEF(rrdfile=myRRD.filename, vname='myspeed', ... dsName=dataSource.name) >>> cdef1 = CDEF(vname='kmh', rpn='%s,3600,*' % def1.vname) >>> cdef2 = CDEF(vname='fast', rpn='kmh,100,GT,kmh,0,IF') >>> cdef3 = CDEF(vname='good', rpn='kmh,100,GT,0,kmh,IF') >>> vdef1 = VDEF(vname='mymax', rpn='%s,MAXIMUM' % def1.vname) >>> vdef2 = VDEF(vname='myavg', rpn='%s,AVERAGE' % def1.vname) >>> line1 = LINE(value=100, color='#990000', legend='Maximum Allowed') >>> area1 = AREA(defObj=cdef3, color='#006600', legend='Good Speed') >>> area2 = AREA(defObj=cdef2, color='#CC6633', legend='Too Fast') >>> line2 = LINE(defObj=vdef2, color='#000099', legend='My Average', ... stack=True) >>> gprint1 = GPRINT(vdef2, '%6.2lf kph') Color is the spice of life. Let's spice it up a little:: >>> from pyrrd.graph import ColorAttributes >>> ca = ColorAttributes() >>> ca.back = '#333333' >>> ca.canvas = '#333333' >>> ca.shadea = '#000000' >>> ca.shadeb = '#111111' >>> ca.mgrid = '#CCCCCC' >>> ca.axis = '#FFFFFF' >>> ca.frame = '#AAAAAA' >>> ca.font = '#FFFFFF' >>> ca.arrow = '#FFFFFF' Now we can create a graph for the data in our RRD file:: >>> from pyrrd.graph import Graph >>> graphfile = "/tmp/rrdgraph.png" >>> g = Graph(graphfile, start=920805000, end=920810000, ... vertical_label='km/h', color=ca) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Let's make sure it's there:: >>> os.path.isfile(graphfile) True Let's get a sense of the byte size:: >>> bytes = len(open(graphfile).read()) >>> bytes != 0 True >>> 8000 < bytes < 10400 True Open that up in your favorite image browser and confirm that the appropriate RRD graph is generated. Let's clean up the files we've put in the temp directory:: >>> os.unlink(filename) >>> os.unlink(graphfile) =============== Python Bindings =============== In addition to the command line tool, PyRRD also supports using the Python bindings, if you have them installed. Let's set some stuff up like we did in the previous example:: >>> filename = '/tmp/test.rrd' >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) Usage is identical to the standard PyRRD usage, with the exception of object construction:: >>> from pyrrd.backend import bindings >>> myRRD = RRD(filename, ds=dataSources, rra=roundRobinArchives, ... backend=bindings) >>> myRRD.create() Note that since the Graph module is its own beast, you will need to indicate whether you want to use the bindings or the external backend when you graph as well:: >>> from pyrrd.graph import Graph >>> graphfile = "/tmp/rrdgraph.png" >>> g = Graph(graphfile, start=920805000, end=920810000, ... vertical_label='km/h', color=ca, backend=bindings) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Everything else is the same. Let's check to see that the file exists, and then we'll cleanup:: >>> import os >>> os.path.isfile(filename) True >>> os.unlink(filename) ========== Known Bugs ========== * http://code.google.com/p/pyrrd/issues/list ==== TODO ==== Near Term --------- * Move test code around (testing and admin/testRunner). * Fix breaking tests. * Add wiki examples for using info and fetch * Improve the wrapper for the Python RRDTool bindings * a lot of the code in PyRRD was written quite a while ago (circa 2004), and needs to be refactored (removing redundancies, use better idioms, etc.) * Allow for users to supply their own fd to pyrrd.graph. * Update all examples for recent dates like example4 has been updated. * Stop using actual file writes and doctests for file tests; use unit tests (and StringIO) instead. Future ------ * Add an RPN class. * Add a DS collection class that has a get() method for getting a particular DS by name. * Add support for atomic operations. ======= Changes ======= From 0.0.7 to 0.1.0 ------------------- This version marks a significant update to the code base. Some 70+ commits have been made to trunk since the last release in the period from March 2009 to September 2011. Some of the most signficant changes include the following: * Added a wrapper for the Python bindings. * Improvements in tests and testing infrastructure. * Improved exception handling. * Many changes and improvements to the RRD -> Python object mapper. * Improved support in Windows. * Graph formatting fixes. * Many bug fixes from community members. From 0.0.6 to 0.0.7 ------------------- * Packaging improvements and loads of documentation. From 0.0.5 to 0.0.6 ------------------- * Bug fix release (missing files in source package). From 0.0.4 to 0.0.5 ------------------- * Added support for retrieving and displaying RRD from RRD files. * Added an object mapper for RRD data (via XML files). * Added community-contributed improvements. From 0.0.3 to 0.0.4 ------------------- * Updated all the examples to work with the latest code. * Added community-contributed bug fix for Windows users. From 0.0.2 to 0.0.3 ------------------- * Minor code reorg. * Fixed doctests. * Various bug fixes. * Examples updates. From 0.0.1 to 0.0.2 ------------------- * Added license. * Added unit tests. * Added more examples. From 0 to 0.0.1 --------------- * Reorganized RRD code as donated from the CoyMon project. * Got basic rrdtool functionality represented as Python classes. * Code cleanup. ================ Acknowledgements ================ The following members of the community have provided valuable contributions to this project: * Ravi Bhalotia, Allen Lerner, Mike Carrick and the U.S. Department of Veterans Affairs * AdytumSolutions, Inc., E-Secure Systems * nasvos, Leem Smit, Aaron Westendorf and Agora Games * Mladen Milankovic, Denis Fortin * Joseph Heck, Jean-Baptiste Quenot * Pavel Shramov, Brad Beattie, Colin Horsington Thanks! ========== References ========== .. [#] http://code.google.com/p/pyrrd/wiki/FrontPage?tm=6 .. [#] http://code.google.com/p/pyrrd/wiki/FullWorkingExamples .. [#] http://oss.oetiker.ch/rrdtool/ .. [#] http://effbot.org/zone/element-index.htm Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Developers Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: Programming Language :: Python Classifier: Topic :: Database Classifier: Topic :: Database :: Front-Ends Classifier: Topic :: Multimedia :: Graphics Classifier: Topic :: System :: Networking :: Monitoring Classifier: Topic :: System :: Systems Administration Classifier: License :: OSI Approved :: BSD License PyRRD-0.1.0/pyrrd/000755 000765 000024 00000000000 11635462441 014603 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/PyRRD.egg-info/000755 000765 000024 00000000000 11635462441 016075 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/README000644 000765 000024 00000020652 11635456135 014333 0ustar00oubiwannstaff000000 000000 ======== Features ======== PyRRD lets you use RRDTool from Python code that takes advantage of standard object-oriented patterns. The makes the programmatic usage of RRDTool much easier and reusable. A quick review of features is available at the project wiki [#]_ . Example code with graph image output is also available on the wiki [#]_ . ============ Introduction ============ PyRRD is an object-oriented wrapper for the command line graphing and round-robin database utility, rrdtool [#]_ . PyRRD originally had two design goals: 1. provide an interface to rrdtool that Python programmers would love, and 2. not depend upon the Python bindings for rrdtool. The reasons for the former are obvious. The motivation for the latter were the many people who had difficulty compiling the rrdtool bindings on their operating system of choice. Even though PyRRD's original purpose was to help those without the bingins, the project now offers support for those with the bindings installed. As such, users may enjoy both the speed benefits from the bindings as well as the API usability from PyRRD. For docs, see the docstrings at the beginning of each class (and many of the functions). They not only contain many of the standard RRDTool docs, but they contain doctests which give you a hands-on, how-it-works understanding of actual usage. For those curious about the motivation for creating PyRRD, perhaps some background would be in order. Originally, there were only two ways to use RRDTool from Python: 1. using the Python bindings which were difficult to compile and use, or 2. making system calls to the "rrdtool" executable from Python, passing it all the parameters it needed. Option #1 was often difficult or impossible for many folks to get running on their preferred operating system. But even if one was able to compile it and run it, usage was very cumbersome and designed to work like the command line tool and the C interface, not like most people typically use Python. Now, with PyRRD, there are two additional ways to use RRDTool from Python: 3. an object-oriented interface that wraps system calls (Popen) to the rrdtool binary, and 4. the same object-oriented interface that wraps the cumbersome rrdtool Python bindings. ============ Dependencies ============ Some parts of PyRRD make use of ElementTree for XML processing. If you have Python 2.5 or greater, PyRRD will use xml.etree. If your Python version is less than 2.5 and you want to use features that depend on XML processing (such as dump function and the fetch/info methods), you will need to install the ElementTree library [#]_ . ============ Installation ============ PyRRD is installed in the usual way:: python setup.py install You may also use PyRRD without installing it as long as you have ./ in your PYTHONPATH and you are in the top-level directory (which has the pyrrd child directory). ===== Usage ===== Create an RRD file programmatically:: >>> from pyrrd.rrd import DataSource, RRA, RRD >>> filename = '/tmp/test.rrd' >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) >>> myRRD = RRD( ... filename, ds=dataSources, rra=roundRobinArchives, start=920804400) >>> myRRD.create() Let's check to see that the file exists:: >>> import os >>> os.path.isfile(filename) True Let's see how big it is (depending upon RRDTool version, the byte count can change, so we'll just get a general sense):: >>> bytes = len(open(filename).read()) >>> 800 < bytes < 1200 True In order to save writes to disk, PyRRD buffers values and then writes the values to the RRD file at one go:: >>> myRRD.bufferValue('920805600', '12363') >>> myRRD.bufferValue('920805900', '12363') >>> myRRD.bufferValue('920806200', '12373') >>> myRRD.bufferValue('920806500', '12383') >>> myRRD.update() Let's add some more data:: >>> myRRD.bufferValue('920806800', '12393') >>> myRRD.bufferValue('920807100', '12399') >>> myRRD.bufferValue('920807400', '12405') >>> myRRD.bufferValue('920807700', '12411') >>> myRRD.bufferValue('920808000', '12415') >>> myRRD.bufferValue('920808300', '12420') >>> myRRD.bufferValue('920808600', '12422') >>> myRRD.bufferValue('920808900', '12423') >>> myRRD.update() If you're curious, you can take a look at your rrd file with the following:: myRRD.info() The output of that isn't printed here, 'cause it take up too much space. However, it is very similar to the output of the similarly named rrdtool command. In order to create a graph, we'll need some data definitions. We'll also throw in some calculated definitions and variable definitions for good meansure:: >>> from pyrrd.graph import DEF, CDEF, VDEF, LINE, AREA, GPRINT >>> def1 = DEF(rrdfile=myRRD.filename, vname='myspeed', ... dsName=dataSource.name) >>> cdef1 = CDEF(vname='kmh', rpn='%s,3600,*' % def1.vname) >>> cdef2 = CDEF(vname='fast', rpn='kmh,100,GT,kmh,0,IF') >>> cdef3 = CDEF(vname='good', rpn='kmh,100,GT,0,kmh,IF') >>> vdef1 = VDEF(vname='mymax', rpn='%s,MAXIMUM' % def1.vname) >>> vdef2 = VDEF(vname='myavg', rpn='%s,AVERAGE' % def1.vname) >>> line1 = LINE(value=100, color='#990000', legend='Maximum Allowed') >>> area1 = AREA(defObj=cdef3, color='#006600', legend='Good Speed') >>> area2 = AREA(defObj=cdef2, color='#CC6633', legend='Too Fast') >>> line2 = LINE(defObj=vdef2, color='#000099', legend='My Average', ... stack=True) >>> gprint1 = GPRINT(vdef2, '%6.2lf kph') Color is the spice of life. Let's spice it up a little:: >>> from pyrrd.graph import ColorAttributes >>> ca = ColorAttributes() >>> ca.back = '#333333' >>> ca.canvas = '#333333' >>> ca.shadea = '#000000' >>> ca.shadeb = '#111111' >>> ca.mgrid = '#CCCCCC' >>> ca.axis = '#FFFFFF' >>> ca.frame = '#AAAAAA' >>> ca.font = '#FFFFFF' >>> ca.arrow = '#FFFFFF' Now we can create a graph for the data in our RRD file:: >>> from pyrrd.graph import Graph >>> graphfile = "/tmp/rrdgraph.png" >>> g = Graph(graphfile, start=920805000, end=920810000, ... vertical_label='km/h', color=ca) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Let's make sure it's there:: >>> os.path.isfile(graphfile) True Let's get a sense of the byte size:: >>> bytes = len(open(graphfile).read()) >>> bytes != 0 True >>> 8000 < bytes < 10400 True Open that up in your favorite image browser and confirm that the appropriate RRD graph is generated. Let's clean up the files we've put in the temp directory:: >>> os.unlink(filename) >>> os.unlink(graphfile) =============== Python Bindings =============== In addition to the command line tool, PyRRD also supports using the Python bindings, if you have them installed. Let's set some stuff up like we did in the previous example:: >>> filename = '/tmp/test.rrd' >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) Usage is identical to the standard PyRRD usage, with the exception of object construction:: >>> from pyrrd.backend import bindings >>> myRRD = RRD(filename, ds=dataSources, rra=roundRobinArchives, ... backend=bindings) >>> myRRD.create() Note that since the Graph module is its own beast, you will need to indicate whether you want to use the bindings or the external backend when you graph as well:: >>> from pyrrd.graph import Graph >>> graphfile = "/tmp/rrdgraph.png" >>> g = Graph(graphfile, start=920805000, end=920810000, ... vertical_label='km/h', color=ca, backend=bindings) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Everything else is the same. Let's check to see that the file exists, and then we'll cleanup:: >>> import os >>> os.path.isfile(filename) True >>> os.unlink(filename) PyRRD-0.1.0/setup.cfg000644 000765 000024 00000000073 11635462441 015264 0ustar00oubiwannstaff000000 000000 [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 PyRRD-0.1.0/setup.py000644 000765 000024 00000002113 11635462311 015146 0ustar00oubiwannstaff000000 000000 from setuptools import setup from pyrrd import meta from pyrrd.util import dist setup( name=meta.display_name, version=meta.version, description=meta.description, author=meta.author, author_email=meta.author_email, url=meta.url, license=meta.license, packages=dist.findPackages(), long_description=dist.catReST( "docs/PRELUDE.txt", "README", "TODO", "docs/HISTORY.txt", "docs/ACKNOWLEDGEMENTS.txt", "docs/FOOTNOTES.txt", stop_on_errors=True, out=True), classifiers=[ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "Programming Language :: Python", "Topic :: Database", "Topic :: Database :: Front-Ends", "Topic :: Multimedia :: Graphics", "Topic :: System :: Networking :: Monitoring", "Topic :: System :: Systems Administration", "License :: OSI Approved :: BSD License", ], ) PyRRD-0.1.0/TODO000644 000765 000024 00000001524 11635461001 014124 0ustar00oubiwannstaff000000 000000 ========== Known Bugs ========== * http://code.google.com/p/pyrrd/issues/list ==== TODO ==== Near Term --------- * Move test code around (testing and admin/testRunner). * Fix breaking tests. * Add wiki examples for using info and fetch * Improve the wrapper for the Python RRDTool bindings * a lot of the code in PyRRD was written quite a while ago (circa 2004), and needs to be refactored (removing redundancies, use better idioms, etc.) * Allow for users to supply their own fd to pyrrd.graph. * Update all examples for recent dates like example4 has been updated. * Stop using actual file writes and doctests for file tests; use unit tests (and StringIO) instead. Future ------ * Add an RPN class. * Add a DS collection class that has a get() method for getting a particular DS by name. * Add support for atomic operations. PyRRD-0.1.0/PyRRD.egg-info/dependency_links.txt000644 000765 000024 00000000001 11635462441 022143 0ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/PyRRD.egg-info/PKG-INFO000644 000765 000024 00000037234 11635462441 017203 0ustar00oubiwannstaff000000 000000 Metadata-Version: 1.0 Name: PyRRD Version: 0.1.0 Summary: An Object-Oriented Python Interface for RRDTool Home-page: http://code.google.com/p/pyrrd/ Author: Duncan McGreggor Author-email: duncan@canonical.com License: BSD Description: ~~~~~ PyRRD ~~~~~ .. contents:: :depth: 1 ======== Features ======== PyRRD lets you use RRDTool from Python code that takes advantage of standard object-oriented patterns. The makes the programmatic usage of RRDTool much easier and reusable. A quick review of features is available at the project wiki [#]_ . Example code with graph image output is also available on the wiki [#]_ . ============ Introduction ============ PyRRD is an object-oriented wrapper for the command line graphing and round-robin database utility, rrdtool [#]_ . PyRRD originally had two design goals: 1. provide an interface to rrdtool that Python programmers would love, and 2. not depend upon the Python bindings for rrdtool. The reasons for the former are obvious. The motivation for the latter were the many people who had difficulty compiling the rrdtool bindings on their operating system of choice. Even though PyRRD's original purpose was to help those without the bingins, the project now offers support for those with the bindings installed. As such, users may enjoy both the speed benefits from the bindings as well as the API usability from PyRRD. For docs, see the docstrings at the beginning of each class (and many of the functions). They not only contain many of the standard RRDTool docs, but they contain doctests which give you a hands-on, how-it-works understanding of actual usage. For those curious about the motivation for creating PyRRD, perhaps some background would be in order. Originally, there were only two ways to use RRDTool from Python: 1. using the Python bindings which were difficult to compile and use, or 2. making system calls to the "rrdtool" executable from Python, passing it all the parameters it needed. Option #1 was often difficult or impossible for many folks to get running on their preferred operating system. But even if one was able to compile it and run it, usage was very cumbersome and designed to work like the command line tool and the C interface, not like most people typically use Python. Now, with PyRRD, there are two additional ways to use RRDTool from Python: 3. an object-oriented interface that wraps system calls (Popen) to the rrdtool binary, and 4. the same object-oriented interface that wraps the cumbersome rrdtool Python bindings. ============ Dependencies ============ Some parts of PyRRD make use of ElementTree for XML processing. If you have Python 2.5 or greater, PyRRD will use xml.etree. If your Python version is less than 2.5 and you want to use features that depend on XML processing (such as dump function and the fetch/info methods), you will need to install the ElementTree library [#]_ . ============ Installation ============ PyRRD is installed in the usual way:: python setup.py install You may also use PyRRD without installing it as long as you have ./ in your PYTHONPATH and you are in the top-level directory (which has the pyrrd child directory). ===== Usage ===== Create an RRD file programmatically:: >>> from pyrrd.rrd import DataSource, RRA, RRD >>> filename = '/tmp/test.rrd' >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) >>> myRRD = RRD( ... filename, ds=dataSources, rra=roundRobinArchives, start=920804400) >>> myRRD.create() Let's check to see that the file exists:: >>> import os >>> os.path.isfile(filename) True Let's see how big it is (depending upon RRDTool version, the byte count can change, so we'll just get a general sense):: >>> bytes = len(open(filename).read()) >>> 800 < bytes < 1200 True In order to save writes to disk, PyRRD buffers values and then writes the values to the RRD file at one go:: >>> myRRD.bufferValue('920805600', '12363') >>> myRRD.bufferValue('920805900', '12363') >>> myRRD.bufferValue('920806200', '12373') >>> myRRD.bufferValue('920806500', '12383') >>> myRRD.update() Let's add some more data:: >>> myRRD.bufferValue('920806800', '12393') >>> myRRD.bufferValue('920807100', '12399') >>> myRRD.bufferValue('920807400', '12405') >>> myRRD.bufferValue('920807700', '12411') >>> myRRD.bufferValue('920808000', '12415') >>> myRRD.bufferValue('920808300', '12420') >>> myRRD.bufferValue('920808600', '12422') >>> myRRD.bufferValue('920808900', '12423') >>> myRRD.update() If you're curious, you can take a look at your rrd file with the following:: myRRD.info() The output of that isn't printed here, 'cause it take up too much space. However, it is very similar to the output of the similarly named rrdtool command. In order to create a graph, we'll need some data definitions. We'll also throw in some calculated definitions and variable definitions for good meansure:: >>> from pyrrd.graph import DEF, CDEF, VDEF, LINE, AREA, GPRINT >>> def1 = DEF(rrdfile=myRRD.filename, vname='myspeed', ... dsName=dataSource.name) >>> cdef1 = CDEF(vname='kmh', rpn='%s,3600,*' % def1.vname) >>> cdef2 = CDEF(vname='fast', rpn='kmh,100,GT,kmh,0,IF') >>> cdef3 = CDEF(vname='good', rpn='kmh,100,GT,0,kmh,IF') >>> vdef1 = VDEF(vname='mymax', rpn='%s,MAXIMUM' % def1.vname) >>> vdef2 = VDEF(vname='myavg', rpn='%s,AVERAGE' % def1.vname) >>> line1 = LINE(value=100, color='#990000', legend='Maximum Allowed') >>> area1 = AREA(defObj=cdef3, color='#006600', legend='Good Speed') >>> area2 = AREA(defObj=cdef2, color='#CC6633', legend='Too Fast') >>> line2 = LINE(defObj=vdef2, color='#000099', legend='My Average', ... stack=True) >>> gprint1 = GPRINT(vdef2, '%6.2lf kph') Color is the spice of life. Let's spice it up a little:: >>> from pyrrd.graph import ColorAttributes >>> ca = ColorAttributes() >>> ca.back = '#333333' >>> ca.canvas = '#333333' >>> ca.shadea = '#000000' >>> ca.shadeb = '#111111' >>> ca.mgrid = '#CCCCCC' >>> ca.axis = '#FFFFFF' >>> ca.frame = '#AAAAAA' >>> ca.font = '#FFFFFF' >>> ca.arrow = '#FFFFFF' Now we can create a graph for the data in our RRD file:: >>> from pyrrd.graph import Graph >>> graphfile = "/tmp/rrdgraph.png" >>> g = Graph(graphfile, start=920805000, end=920810000, ... vertical_label='km/h', color=ca) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Let's make sure it's there:: >>> os.path.isfile(graphfile) True Let's get a sense of the byte size:: >>> bytes = len(open(graphfile).read()) >>> bytes != 0 True >>> 8000 < bytes < 10400 True Open that up in your favorite image browser and confirm that the appropriate RRD graph is generated. Let's clean up the files we've put in the temp directory:: >>> os.unlink(filename) >>> os.unlink(graphfile) =============== Python Bindings =============== In addition to the command line tool, PyRRD also supports using the Python bindings, if you have them installed. Let's set some stuff up like we did in the previous example:: >>> filename = '/tmp/test.rrd' >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) Usage is identical to the standard PyRRD usage, with the exception of object construction:: >>> from pyrrd.backend import bindings >>> myRRD = RRD(filename, ds=dataSources, rra=roundRobinArchives, ... backend=bindings) >>> myRRD.create() Note that since the Graph module is its own beast, you will need to indicate whether you want to use the bindings or the external backend when you graph as well:: >>> from pyrrd.graph import Graph >>> graphfile = "/tmp/rrdgraph.png" >>> g = Graph(graphfile, start=920805000, end=920810000, ... vertical_label='km/h', color=ca, backend=bindings) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Everything else is the same. Let's check to see that the file exists, and then we'll cleanup:: >>> import os >>> os.path.isfile(filename) True >>> os.unlink(filename) ========== Known Bugs ========== * http://code.google.com/p/pyrrd/issues/list ==== TODO ==== Near Term --------- * Move test code around (testing and admin/testRunner). * Fix breaking tests. * Add wiki examples for using info and fetch * Improve the wrapper for the Python RRDTool bindings * a lot of the code in PyRRD was written quite a while ago (circa 2004), and needs to be refactored (removing redundancies, use better idioms, etc.) * Allow for users to supply their own fd to pyrrd.graph. * Update all examples for recent dates like example4 has been updated. * Stop using actual file writes and doctests for file tests; use unit tests (and StringIO) instead. Future ------ * Add an RPN class. * Add a DS collection class that has a get() method for getting a particular DS by name. * Add support for atomic operations. ======= Changes ======= From 0.0.7 to 0.1.0 ------------------- This version marks a significant update to the code base. Some 70+ commits have been made to trunk since the last release in the period from March 2009 to September 2011. Some of the most signficant changes include the following: * Added a wrapper for the Python bindings. * Improvements in tests and testing infrastructure. * Improved exception handling. * Many changes and improvements to the RRD -> Python object mapper. * Improved support in Windows. * Graph formatting fixes. * Many bug fixes from community members. From 0.0.6 to 0.0.7 ------------------- * Packaging improvements and loads of documentation. From 0.0.5 to 0.0.6 ------------------- * Bug fix release (missing files in source package). From 0.0.4 to 0.0.5 ------------------- * Added support for retrieving and displaying RRD from RRD files. * Added an object mapper for RRD data (via XML files). * Added community-contributed improvements. From 0.0.3 to 0.0.4 ------------------- * Updated all the examples to work with the latest code. * Added community-contributed bug fix for Windows users. From 0.0.2 to 0.0.3 ------------------- * Minor code reorg. * Fixed doctests. * Various bug fixes. * Examples updates. From 0.0.1 to 0.0.2 ------------------- * Added license. * Added unit tests. * Added more examples. From 0 to 0.0.1 --------------- * Reorganized RRD code as donated from the CoyMon project. * Got basic rrdtool functionality represented as Python classes. * Code cleanup. ================ Acknowledgements ================ The following members of the community have provided valuable contributions to this project: * Ravi Bhalotia, Allen Lerner, Mike Carrick and the U.S. Department of Veterans Affairs * AdytumSolutions, Inc., E-Secure Systems * nasvos, Leem Smit, Aaron Westendorf and Agora Games * Mladen Milankovic, Denis Fortin * Joseph Heck, Jean-Baptiste Quenot * Pavel Shramov, Brad Beattie, Colin Horsington Thanks! ========== References ========== .. [#] http://code.google.com/p/pyrrd/wiki/FrontPage?tm=6 .. [#] http://code.google.com/p/pyrrd/wiki/FullWorkingExamples .. [#] http://oss.oetiker.ch/rrdtool/ .. [#] http://effbot.org/zone/element-index.htm Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Developers Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: Programming Language :: Python Classifier: Topic :: Database Classifier: Topic :: Database :: Front-Ends Classifier: Topic :: Multimedia :: Graphics Classifier: Topic :: System :: Networking :: Monitoring Classifier: Topic :: System :: Systems Administration Classifier: License :: OSI Approved :: BSD License PyRRD-0.1.0/PyRRD.egg-info/SOURCES.txt000644 000765 000024 00000002521 11635462441 017761 0ustar00oubiwannstaff000000 000000 ChangeLog LICENSE MANIFEST.in README TODO setup.py PyRRD.egg-info/PKG-INFO PyRRD.egg-info/SOURCES.txt PyRRD.egg-info/dependency_links.txt PyRRD.egg-info/top_level.txt docs/ACKNOWLEDGEMENTS.txt docs/FOOTNOTES.txt docs/HISTORY.txt docs/PRELUDE.txt examples/example1.png examples/example1.py examples/example2.png examples/example2.py examples/example3-large.png examples/example3.png examples/example3.py examples/example4-large.png examples/example4.png examples/example4.py examples/example5-10800-large.png examples/example5-10800.png examples/example5-259200-large.png examples/example5-259200.png examples/example5-43200-large.png examples/example5-43200.png examples/example5-86400-large.png examples/example5-86400.png examples/example5.py pyrrd/__init__.py pyrrd/exceptions.py pyrrd/graph.py pyrrd/mapper.py pyrrd/meta.py pyrrd/node.py pyrrd/rrd.py pyrrd/backend/__init__.py pyrrd/backend/bindings.py pyrrd/backend/common.py pyrrd/backend/external.py pyrrd/backend/native/__init__.py pyrrd/backend/native/format.py pyrrd/backend/tests/__init__.py pyrrd/backend/tests/test_bindings.py pyrrd/backend/tests/test_external.py pyrrd/testing/__init__.py pyrrd/testing/base.py pyrrd/testing/dump.py pyrrd/testing/suite.py pyrrd/tests/__init__.py pyrrd/tests/test_mapper.py pyrrd/tests/test_node.py pyrrd/tests/test_rrd.py pyrrd/util/__init__.py pyrrd/util/dist.pyPyRRD-0.1.0/PyRRD.egg-info/top_level.txt000644 000765 000024 00000000006 11635462441 020623 0ustar00oubiwannstaff000000 000000 pyrrd PyRRD-0.1.0/pyrrd/__init__.py000644 000765 000024 00000000000 11635456135 016705 0ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/backend/000755 000765 000024 00000000000 11635462441 016172 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/exceptions.py000644 000765 000024 00000000131 11635456135 017334 0ustar00oubiwannstaff000000 000000 class PyRRDError(Exception): pass class ExternalCommandError(PyRRDError): pass PyRRD-0.1.0/pyrrd/graph.py000644 000765 000024 00000074264 11635456135 016276 0ustar00oubiwannstaff000000 000000 import os import re from pyrrd.backend import external def validateVName(name): ''' RRDTool vnames must be made up strings of the following characters: A-Z, a-z, 0-9, -,_ and have a maximum length of 255 characters. >>> vname = validateVName('Zaphod Beeble-Brox!') Traceback (most recent call last): ValueError: Names must consist only of the characters A-Z, a-z, 0-9, -, _ >>> vname = validateVName('Zaphod_Beeble-Brox') >>> vname = validateVName('a'*32) >>> vname = validateVName('a'*254) >>> vname = validateVName('a'*255) >>> vname = validateVName('a'*256) Traceback (most recent call last): ValueError: Names must be shorter than 255 characters ''' if name != re.sub('[^A-Za-z0-9_-]', '', name): raise ValueError, "Names must consist only of the characters " + \ "A-Z, a-z, 0-9, -, _" if len(name) > 255: raise ValueError, "Names must be shorter than 255 characters" return name def escapeColons(data): ''' Time data in RRD parameters that have colons need to escape them, due to the fact that RRDTool uses colons as separators. Additionally, comments have to be colon-escaped as well. >>> print escapeColons('now') now >>> print escapeColons('end-8days8hours') end-8days8hours >>> print escapeColons('13:00') 13\:00 ''' return re.sub(':', '\:', data) def validateObjectType(instance, objType): ''' >>> my_list = [1,2,3,4,5] >>> validateObjectType(my_list, list) [1, 2, 3, 4, 5] >>> validateObjectType(my_list, dict) Traceback (most recent call last): TypeError: list instance is not of type dict ''' if isinstance(instance, objType): return instance raise TypeError, "%s instance is not of type %s" % ( type(instance).__name__, objType.__name__) def validateImageFormat(format): ''' >>> validateImageFormat('txt') Traceback (most recent call last): ValueError: The image format must be one of the following: PNG SVG EPS PDF >>> validateImageFormat('jpg') Traceback (most recent call last): ValueError: The image format must be one of the following: PNG SVG EPS PDF >>> validateImageFormat('png') 'PNG' ''' format = format.upper() valid = ['PNG', 'SVG', 'EPS', 'PDF'] if format in valid: return format else: valid = ' '.join(valid) raise ValueError, 'The image format must be one of the ' + \ 'following: %s' % valid class DataDefinition(object): ''' This object causes data to be fetched from the RRD file. The virtual name vname can then be used throughout the rest of the script. By default, an RRA which contains the correct consolidated data at an appropriate resolution will be chosen. The resolution can be overridden with the --step option. The resolution can again be overridden by specifying the step size. The time span of this data is the same as for the graph by default, you can override this by specifying start and end. Remember to escape colons in the time specification! If the resolution of the data is higher than the resolution of the graph, the data will be further consolidated. This may result in a graph that spans slightly more time than requested. Ideally each point in the graph should correspond with one CDP from an RRA. For instance, if your RRD has an RRA with a resolution of 1800 seconds per CDP, you should create an image with width 400 and time span 400*1800 seconds (use appropriate start and end times, such as --start end-8days8hours). If consolidation needs to be done, the CF of the RRA specified in the DEF itself will be used to reduce the data density. This behaviour can be changed using :reduce=. This optional parameter specifies the CF to use during the data reduction phase. >>> def1 = DataDefinition(vname='ds0a', ... rrdfile='/home/rrdtool/data/router1.rrd', dsName='ds0', ... cdef='AVERAGE') >>> def1 DEF:ds0a=/home/rrdtool/data/router1.rrd:ds0:AVERAGE >>> def1.__repr__() 'DEF:ds0a=/home/rrdtool/data/router1.rrd:ds0:AVERAGE' >>> def2 = DataDefinition(rrdfile='/home/rrdtool/data/router1.rrd') >>> def2.vname = 'ds0b' >>> def2.dsName = 'ds0' >>> def2.cdef = 'AVERAGE' >>> def2.step = 1800 >>> def2 DEF:ds0b=/home/rrdtool/data/router1.rrd:ds0:AVERAGE:step=1800 >>> def3 = DEF(vname='ds0c', dsName='ds0', step=7200) >>> def3.rrdfile = '/home/rrdtool/data/router1.rrd' >>> def3 DEF:ds0c=/home/rrdtool/data/router1.rrd:ds0:AVERAGE:step=7200 >>> def4 = DEF() >>> def4 Traceback (most recent call last): ValueError: vname, rrdfile, dsName, and cdef are all required attributes and cannot be None. >>> def4.rrdfile = '/home/rrdtool/data/router2.rrd' >>> def4 Traceback (most recent call last): ValueError: vname, rrdfile, dsName, and cdef are all required attributes and cannot be None. >>> def4.vname = 'ds1a' >>> def4 Traceback (most recent call last): ValueError: vname, rrdfile, dsName, and cdef are all required attributes and cannot be None. >>> def4.dsName = 'ds1' >>> def4 DEF:ds1a=/home/rrdtool/data/router2.rrd:ds1:AVERAGE ''' def __init__(self, vname='', rrdfile='', dsName='', cdef='AVERAGE', step=None, start=None, end=None, reduce=None): self.vname = validateVName(vname) self.rrdfile = escapeColons(rrdfile) self.dsName = dsName self.cdef = cdef self.step = step self.start = start self.end = end self.reduce = reduce def __repr__(self): ''' We override this method for preparing the class's data for use with RRDTool. Time representations must have their ':'s escaped, since the colon is the RRDTool separator for parameters. ''' if not (self.vname and self.rrdfile and self.dsName and self.cdef): msg = ("vname, rrdfile, dsName, and cdef " + "are all required attributes and cannot be None.") raise ValueError, msg main = 'DEF:%(vname)s=%(rrdfile)s:%(dsName)s:%(cdef)s' % ( self.__dict__) tail = '' if self.step: tail += ':step=%s' % self.step if self.start: tail += ':start=%s' % escapeColons(self.start) if self.end: tail += ':end=%s' % escapeColons(self.end) if self.reduce: tail += ':reduce=%s' % self.reduce return main+tail DEF = DataDefinition class VariableDefinition(object): ''' This object has two attributes: vname rpn_expr It generates a value and/or a time according to the RPN statements used. The resulting vname will, depending on the functions used, have a value and a time component. When you use this vname in another RPN expression, you are effectively inserting its value just as if you had put a number at that place. The variable can also be used in the various graph and print elements. Note that currently only agregation functions work in VDEF rpn expressions (a limitation of RRDTool, not PyRRD). >>> def1 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds0a', dsName='ds0') >>> def2 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds1a', dsName='ds1') >>> rpnmax = '%s,MAXIMUM' >>> rpnmin = '%s,MINIMUM' >>> rpnavg = '%s,AVERAGE' >>> rpnpct = '%s,%s,PERCENT' >>> vdef1 = VariableDefinition(vname='ds0max', ... rpn=rpnmax % def1.dsName) >>> vdef1 VDEF:ds0max=ds0,MAXIMUM >>> vdef2 = VDEF(vname='ds0avg', rpn=rpnavg % def1.dsName) >>> vdef2 VDEF:ds0avg=ds0,AVERAGE >>> vdef3 = VDEF(vname='ds0min', rpn=rpnmin % def1.dsName) >>> vdef3 VDEF:ds0min=ds0,MINIMUM >>> vdef4 = VDEF(vname='ds1pct', rpn=rpnpct % (def2.dsName, 95)) >>> vdef4 VDEF:ds1pct=ds1,95,PERCENT ''' def __init__(self, vname=None, rpn=None): if vname == None: raise ValueError, "You must provide a variable definition name." if rpn == None: raise ValueError, "You must provide an RPN statement(s)." self.vname = validateVName(vname) self.rpn = rpn self.abbr = 'VDEF' def __repr__(self): ''' We override this method for preparing the class's data for use with RRDTool. Time representations must have their ':'s escaped, since the colon is the RRDTool separator for parameters. ''' main = self.abbr+':%(vname)s=%(rpn)s' % ( self.__dict__) return main VDEF = VariableDefinition class CalculationDefinition(VariableDefinition): ''' This object creates a new set of data points (in memory only, not in the RRD file) out of one or more other data series. It has two attributes: vname rpn_expr The RPN instructions are used to evaluate a mathematical function on each data point. The resulting vname can then be used further on in the script, just as if it were generated by a DEF instruction. >>> someDSN = 'mydata' >>> cdef1 = CDEF(vname='mydatabits', rpn='%s,8,*' % someDSN) >>> cdef1 CDEF:mydatabits=mydata,8,* ''' def __init__(self, vname=None, rpn=None): super(CalculationDefinition, self).__init__(vname, rpn) self.abbr = 'CDEF' CDEF = CalculationDefinition class Print(object): ''' Depending on the context, either the value component or the time component of a VDEF is printed using format. It is an error to specify a vname generated by a DEF or CDEF. Any text in format is printed literally with one exception: The percent character introduces a formatter string. This string can be: For printing values: %% just prints a literal '%' character %#.#le prints numbers like 1.2346e+04. The optional integers # denote field width and decimal precision. %#.#lf prints numbers like 12345.6789, with optional field width and precision. %s place this after %le, %lf or %lg. This will be replaced by the appropriate SI magnitude unit and the value will be scaled accordingly (123456 -> 123.456 k). %S is similar to %s. It does, however, use a previously defined magnitude unit. If there is no such unit yet, it tries to define one (just like %s) unless the value is zero, in which case the magnitude unit stays undefined. Thus, formatter strings using %S and no %s will all use the same magnitude unit except for zero values. For printing times: %% just prints a literal '%' character %a, %A print the abbreviated or full name of the day of the week. %b, %B print the abbreviated or full name of the month. %d, %m, %y, %H, %M, %S print day, month, year, hour, minute, and second in two-digit format. %Y prints the year in 4-digit format. %I, %p print the hour (01..12), 'am' or 'pm'. %j, %w print day of the week (0..6), day of the year (1..366) %c, %x, %X print date+time, date only, time only. %U, %W number of the week of the current year, with either the first Sunday (%U) or the first Monday (%W) determining the first week. %Z prints the time zone. This object takes as parameters: a VDEF instance a format, per defined above This is for printing to stdout. See GraphPrint for printing to the generated graphs. >>> def1 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds0a', dsName='ds0') >>> vdef1 = VariableDefinition(vname='ds0max', ... rpn='%s,MAXIMUM' % def1.dsName) >>> prnFmt = "%6.2lf %Sbps" >>> prn = Print(vdef1, prnFmt) >>> prn PRINT:ds0max:"%6.2lf %Sbps" ''' def __init__(self, vdefObj, format): vdefObj = validateObjectType(vdefObj, VariableDefinition) self.vname = vdefObj.vname self.format = format self.abbr = 'PRINT' def __repr__(self): ''' We override this method for preparing the class's data for use with RRDTool. Time representations must have their ':'s escaped, since the colon is the RRDTool separator for parameters. ''' main = self.abbr+':%s:"%s"' % (self.vname, escapeColons(self.format)) return main PRINT = Print class GraphPrint(Print): ''' This is the same as PRINT, but printed inside the graph. >>> def1 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds0a', dsName='ds0') >>> vdef1 = VariableDefinition(vname='ds0max', ... rpn='%s,MAXIMUM' % def1.dsName) >>> prnFmt = '%6.2lf %Sbps' >>> prn = GraphPrint(vdef1, prnFmt) >>> prn GPRINT:ds0max:"%6.2lf %Sbps" ''' def __init__(self, vdefObj, format): super(GraphPrint, self).__init__(vdefObj, format) self.abbr = 'GPRINT' GPRINT = GraphPrint class GraphComment(object): ''' Text is printed literally in the legend section of the graph. Note that in RRDtool 1.2 you have to escape colons in COMMENT text in the same way you have to escape them in *PRINT commands by writing '\:'. >>> cmt = GraphComment('95th percentile') >>> len(str(cmt)) 26 >>> cmt = GraphComment('95th percentile', autoNewline=False) >>> len(str(cmt)) 25 >>> print cmt COMMENT:"95th percentile" ''' def __init__(self, comment, autoNewline=True): self.autoNewline = autoNewline self.comment = comment def __repr__(self): ''' We override this method for preparing the class's data for use with RRDTool. Time representations must have their ':'s escaped, since the colon is the RRDTool separator for parameters. ''' newLine = '\n' if not self.autoNewline: newLine = '' main = 'COMMENT:"%s"%s' % ( self.comment, newLine) return main COMMENT = GraphComment class GraphVerticalLine(object): ''' Draw a vertical line at time. Its color is composed from three hexadecimal numbers specifying the rgb color components (00 is off, FF is maximum) red, green and blue. Optionally, a legend box and string is printed in the legend section. time may be a number or a variable from a VDEF. It is an error to use vnames from DEF or CDEF here. ''' # XXX TODO VRULE = GraphVerticalLine class Line(object): ''' Draw a line of the specified width onto the graph. Width can be a floating point number. If the color is not specified, the drawing is done 'invisibly'. This is useful when stacking something else on top of this line. Also optional is the legend box and string which will be printed in the legend section if specified. The value can be generated by DEF, VDEF, and CDEF. If the optional STACK modifier is used, this line is stacked on top of the previous element which can be a LINE or an AREA. When you do not specify a color, you cannot specify a legend. Should you want to use STACK, use the ``LINEx:::STACK'' form. >>> def1 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds0a', dsName='ds0') >>> vdef1 = VariableDefinition(vname='ds0max', ... rpn='%s,MAXIMUM' % def1.dsName) # Now let's do some lines... >>> line = Line(1, value='ds0max', color='#00ff00', ... legend="Max") >>> line LINE1:ds0max#00ff00:"Max" >>> LINE(2, defObj=def1, color='#0000ff') LINE2:ds0a#0000ff >>> LINE(1, defObj=vdef1, color='#ff0000') LINE1:ds0max#ff0000 >>> LINE(1, color='#ff0000') Traceback (most recent call last): Exception: You must provide either a value or a definition object. >>> LINE(1, value=vdef1, color='#ff0000') Traceback (most recent call last): ValueError: The parameter 'value' must be either a string or an integer. ''' def __init__(self, width=None, value=None, defObj=None, color=None, legend='', stack=False): ''' If a DEF, VDEF, or CDEF object as passed, the vname will be automatically extraced from the object and used. ''' self.width = width self.color = color self.legend = legend self.stack = stack if value: if not (isinstance(value, str) or isinstance(value, int)): raise ValueError, "The parameter 'value' must be " + \ "either a string or an integer." else: if not defObj: raise Exception, "You must provide either a value " + \ "or a definition object." else: value = defObj.vname self.vname = value self.abbr = 'LINE' def __repr__(self): ''' We override this method for preparing the class's data for use with RRDTool. ''' main = self.abbr if self.width: main += unicode(self.width) main += ':%s' % self.vname if self.color: main += self.color if self.legend: main += ':"%s"' % self.legend if self.stack: main += ':STACK' return main LINE = Line class Area(Line): ''' See LINE, however the area between the x-axis and the line will be filled. >>> def1 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds0a', dsName='ds0') >>> vdef1 = VariableDefinition(vname='ds0max', ... rpn='%s,MAXIMUM' % def1.dsName) # Now let's do some areas... >>> Area(value='ds0a', color='#cccccc', legend='Raw Router Data') AREA:ds0a#cccccc:"Raw Router Data" >>> AREA(defObj=vdef1, color='#cccccc', legend='Max Router Data', ... stack=True) AREA:ds0max#cccccc:"Max Router Data":STACK ''' def __init__(self, width=None, value=None, defObj=None, color=None, legend='', stack=False): ''' If a DEF, VDEF, or CDEF object as passed, the vname will be automatically extraced from the object and used. ''' super(Area, self).__init__(value=value, defObj=defObj, color=color, legend=legend, stack=stack) self.abbr = 'AREA' AREA = Area class GraphTick(object): ''' Plot a tick mark (a vertical line) for each value of vname that is non-zero and not *UNKNOWN*. The fraction argument specifies the length of the tick mark as a fraction of the y-axis; the default value is 0.1 (10% of the axis). Note that the color specification is not optional. >>> def1 = DEF(rrdfile='/home/rrdtool/data/router1.rrd', ... vname='ds0a', dsName='ds0') >>> GraphTick(def1,'#ffffff',0.3,'Alarm!') TICK:ds0a#ffffff:0.3:"Alarm!" ''' def __init__(self, defObj=None, color=None, fraction=None, legend=''): if not defObj: raise Exception, "You must provide either a value " + \ "or a definition object." else: value1 = defObj.vname if fraction: if not (isinstance(fraction, float) or isinstance(fraction, int)): raise TypeError, "The parameter 'fraction' must" + \ "be a float value between 0 and 1." else: if 0 <= fraction <= 1: value2 = fraction else: raise ValueError, "The parameter 'fraction' must" + \ "be a value between 0 and 1." if not color: raise ValueError, "Missing required parameter color" self.vname = value1 self.color = color self.fraction = value2 self.legend = legend self.abbr = 'TICK' def __repr__(self): main = self.abbr main += ':%s' % self.vname if self.color: main += self.color if self.fraction: main += ':%s' % self.fraction if self.legend: main += ':"%s"' % self.legend return main TICK = GraphTick class GraphShift(object): ''' Using this command RRDtool will graph the following elements with the specified offset. For instance, you can specify an offset of ( 7*24*60*60 = ) 604'800 seconds to ``look back'' one week. Make sure to tell the viewer of your graph you did this ... As with the other graphing elements, you can specify a number or a variable here. ''' # XXX TODO SHIFT = GraphShift class GraphXGrid(object): ''' The x-axis label is quite complex to configure. If you don't have very special needs it is probably best to rely on the autoconfiguration to get this right. You can specify the string none to suppress the grid and labels altogether. The grid is defined by specifying a certain amount of time in the ?TM positions. You can choose from SECOND, MINUTE, HOUR, DAY, WEEK, MONTH or YEAR. Then you define how many of these should pass between each line or label. This pair (?TM:?ST) needs to be specified for the base grid (G??), the major grid (M??) and the labels (L??). For the labels you also must define a precision in LPR and a strftime format string in LFM. LPR defines where each label will be placed. If it is zero, the label will be placed right under the corresponding line (useful for hours, dates etcetera). If you specify a number of seconds here the label is centered on this interval (useful for Monday, January etcetera). --x-grid MINUTE:10:HOUR:1:HOUR:4:0:%X This places grid lines every 10 minutes, major grid lines every hour, and labels every 4 hours. The labels are placed under the major grid lines as they specify exactly that time. --x-grid HOUR:8:DAY:1:DAY:1:0:%A This places grid lines every 8 hours, major grid lines and labels each day. The labels are placed exactly between two major grid lines as they specify the complete day and not just midnight. ''' class GraphYGrid(object): ''' Y-axis grid lines appear at each grid step interval. Labels are placed every label factor lines. You can specify -y none to suppress the grid and labels altogether. The default for this option is to automatically select sensible values. ''' class ColorAttributes(object): ''' This class is repr'ed without a leading '--color' because that will be provided by the graph class when it's color attribute is set to an instance of this class. >>> ColorAttributes(background='#000000', axis='#FFFFFF') AXIS#FFFFFF --color BACK#000000 >>> ca = ColorAttributes() >>> ca.back = '#333333' >>> ca.canvas = '#333333' >>> ca.shadea = '#000000' >>> ca.shadeb = '#111111' >>> ca.mgrid = '#CCCCCC' >>> ca.axis = '#FFFFFF' >>> ca.frame = '#AAAAAA' >>> ca.font = '#FFFFFF' >>> ca.arrow = '#FFFFFF' >>> ca ARROW#FFFFFF --color AXIS#FFFFFF --color BACK#333333 --color CANVAS#333333 --color FONT#FFFFFF --color FRAME#AAAAAA --color MGRID#CCCCCC --color SHADEA#000000 --color SHADEB#111111 ''' def __init__(self, background=None, canvas=None, lefttop_border=None, rightbottom_border=None, major_grid=None, font=None, axis=None, frame=None, arrow=None): ''' Each of the parameters that gets pass when initializing this class take only a hexidecimal color as a value. ''' self.back = background self.canvas = canvas self.shadea = lefttop_border self.shadeb = rightbottom_border self.mgrid = major_grid self.font = font self.axis = axis self.frame = frame self.arror = arrow def __repr__(self): joiner = ' --color ' params = self.__dict__.items() params.sort() attrs = [ name.upper()+color for name,color in params if color ] return joiner.join(attrs) class Graph(object): ''' rrdtool graph needs data to work with, so you must use one or more data definition statements to collect this data. You are not limited to one database, it's perfectly legal to collect data from two or more databases (one per statement, though). If you want to display averages, maxima, percentiles, etcetera it is best to collect them now using the variable definition statement. Currently this makes no difference, but in a future version of rrdtool you may want to collect these values before consolidation. The data fetched from the RRA is then consolidated so that there is exactly one datapoint per pixel in the graph. If you do not take care yourself, RRDtool will expand the range slightly if necessary. Note, in that case the first and/or last pixel may very well become unknown! Sometimes data is not exactly in the format you would like to display it. For instance, you might be collecting bytes per second, but want to display bits per second. This is what the data calculation command is designed for. After consolidating the data, a copy is made and this copy is modified using a rather powerful RPN command set. When you are done fetching and processing the data, it is time to graph it (or print it). This ends the rrdtool graph sequence. # Let's create and RRD file and dump some data in it >>> import tempfile >>> from rrd import RRD, RRA, DS >>> dss = [] >>> rras = [] >>> tfile = tempfile.NamedTemporaryFile() >>> filename = tfile.name >>> ds1 = DS(dsName='speed', dsType='COUNTER', heartbeat=600) >>> dss.append(ds1) >>> rra1 = RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24) >>> rra2 = RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10) >>> rras.extend([rra1, rra2]) >>> my_rrd = RRD(filename, ds=dss, rra=rras, start=920804400) >>> my_rrd.create() >>> import os >>> os.path.exists(filename) True >>> my_rrd.bufferValue('920805600', '12363') >>> my_rrd.bufferValue('920805900', '12363') >>> my_rrd.bufferValue('920806200', '12373') >>> my_rrd.bufferValue('920806500', '12383') >>> my_rrd.update() >>> my_rrd.bufferValue('920806800', '12393') >>> my_rrd.bufferValue('920807100', '12399') >>> my_rrd.bufferValue('920807400', '12405') >>> my_rrd.bufferValue('920807700', '12411') >>> my_rrd.bufferValue('920808000', '12415') >>> my_rrd.bufferValue('920808300', '12420') >>> my_rrd.bufferValue('920808600', '12422') >>> my_rrd.bufferValue('920808900', '12423') >>> my_rrd.update() # Let's set up the objects that will be added to the graph >>> def1 = DEF(rrdfile=my_rrd.filename, vname='myspeed', dsName=ds1.name) >>> cdef1 = CDEF(vname='kmh', rpn='%s,3600,*' % def1.vname) >>> cdef2 = CDEF(vname='fast', rpn='kmh,100,GT,kmh,0,IF') >>> cdef3 = CDEF(vname='good', rpn='kmh,100,GT,0,kmh,IF') >>> vdef1 = VDEF(vname='mymax', rpn='%s,MAXIMUM' % def1.vname) >>> vdef2 = VDEF(vname='myavg', rpn='%s,AVERAGE' % def1.vname) >>> line1 = LINE(value=100, color='#990000', legend='Maximum Allowed') >>> area1 = AREA(defObj=cdef3, color='#006600', legend='Good Speed') >>> area2 = AREA(defObj=cdef2, color='#CC6633', legend='Too Fast') >>> line2 = LINE(defObj=vdef2, color='#000099', legend='My Average', stack=True) >>> gprint1 = GPRINT(vdef2, '%6.2lf kph') # Let's configure some custom colors for the graph >>> ca = ColorAttributes() >>> ca.back = '#333333' >>> ca.canvas = '#333333' >>> ca.shadea = '#000000' >>> ca.shadeb = '#111111' >>> ca.mgrid = '#CCCCCC' >>> ca.axis = '#FFFFFF' >>> ca.frame = '#AAAAAA' >>> ca.font = '#FFFFFF' >>> ca.arrow = '#FFFFFF' # Now that we've got everything set up, let's make a graph >>> graphfile = tempfile.NamedTemporaryFile() >>> g = Graph(graphfile.name, start=920805000, end=920810000, ... vertical_label='km/h', color=ca, imgformat='png') >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, ... area1, area2, line2, gprint1]) >>> g.write() >>> os.path.exists(graphfile.name) True ''' # Note that we don't use the Twisted camel case convention for the # parameters in the following method signature due to the fact that these # are what is used by RRDTool. Stuff will break if we don't. def __init__(self, filename, start=None, end=None, step=None, title='', vertical_label='', width=None, height=None, only_graph=None, upper_limit=None, lower_limit=None, rigid=False, alt_autoscale=None, alt_autoscale_max=None, no_gridfit=False, x_grid=None, y_grid=None, alt_y_grid=False, logarithmic=False, units_exponent=None, units_length=None, lazy=False, imginfo=None, color=None, zoom=None, font=None, font_render_mode=None, font_smoothing_threshold=None, slope_mode=None, imgformat='', interlaced=False, no_legend=False, force_rules_legend=False, tabwidth=None, base=None, backend=external): self.filename = filename if not imgformat: fn, ext = os.path.splitext(filename) ext = ext.strip(os.extsep) imgformat = ext self.imgformat = validateImageFormat(imgformat) self.start = start self.end = end self.step = step self.title = title self.vertical_label = vertical_label self.width = width self.height = height self.only_graph = only_graph self.upper_limit = upper_limit self.lower_limit = lower_limit self.rigid = rigid self.alt_autoscale = alt_autoscale self.alt_autoscale_max = alt_autoscale_max self.no_gridfit = no_gridfit self.x_grid = x_grid self.y_grid = y_grid self.alt_y_grid = alt_y_grid self.logarithmic = logarithmic self.units_exponent = units_exponent self.color = color self.zoom = zoom self.font = font self.font_render_mode = font_render_mode self.interlaced = interlaced self.no_legend = no_legend self.force_rules_legend = force_rules_legend self.tabwidth = tabwidth self.base = base self.slope_mode = slope_mode self.backend = backend if filename.strip() == '-': # send to stdout pass self.data = [] def write(self, debug=False): ''' ''' data = self.backend.prepareObject('graph', self) if debug: print data self.backend.graph(*data) if __name__ == '__main__': import doctest doctest.testmod() PyRRD-0.1.0/pyrrd/mapper.py000644 000765 000024 00000012052 11635456135 016444 0ustar00oubiwannstaff000000 000000 from pyrrd.node import RRDXMLNode class DSMixin(object): def __init__(self): self.ds = [] class Mapper(object): """ """ __slots__ = [] __skip_repr__ = [] def setAttributes(self, attributes): for name, value in attributes.items(): if name not in self.__skip_repr__: setattr(self, name, value) def getData(self): items = {} for name in self.__slots__: if name not in self.__skip_repr__: items[name] = getattr(self, name, None) return items def map(self, node): """ """ self.setAttributes(node.attributes) def printInfo(self): for name, value in self.getData().items(): if value is None: continue print "%s = %s" % (name, unicode(value)) class RowMapper(Mapper): """ """ __slots__ = ["v"] class DatabaseMapper(Mapper): """ """ __slots__ = ["rows"] __skip_repr__ = ["rows"] def __init__(self): self.rows = [] class CDPrepDSMapper(Mapper): """ """ __slots__ = [ "primary_value", "secondary_value", "value", "unknown_datapoints", ] def printInfo(self, prefix, index): for name, value in self.getData().items(): if value is None: continue print "%s.cdp_prep[%s].%s = %s" % ( prefix, index, name, unicode(value)) class CDPPrepMapper(Mapper, DSMixin): """ """ __slots__ = ["ds"] __skip_repr__ = ["ds"] class RRAMapper(Mapper, DSMixin): """ """ __slots__ = [ "cf", "pdp_per_row", "xff", "cdp_prep", "ds", "steps", "rows", "alpha", "beta", "seasonal_period", "rra_num", "gamma", "threshold", "window_length", "database", ] __skip_repr__ = ["ds"] def map(self, node): super(RRAMapper, self).map(node) for subNode in node.cdp_prep.ds: ds = CDPrepDSMapper() ds.map(subNode) self.ds.append(ds) def getData(self): data = super(RRAMapper, self).getData() data["ds"] = [ds.getData() for ds in self.ds] return data def printInfo(self, index): prefix = "rra[%s]" % index for name, value in self.getData().items(): if value is None: continue print "%s.%s = %s" % (prefix, name, unicode(value)) for index, ds in enumerate(self.ds): ds.printInfo(prefix, index) class DSMapper(Mapper): """ """ __slots__ = [ "name", "type", "minimal_heartbeat", "min", "max", "last_ds", "value", "unknown_sec", "rpn", ] def printInfo(self): for name, value in self.getData().items(): if value is None: continue if name != self.name: print "ds[%s].%s = %s" % (self.name, name, unicode(value)) class RRDMapper(Mapper, DSMixin): """ """ __slots__ = [ "version", "step", "lastupdate", "ds", "rra", "values", "start", "mode", "filename", ] __skip_repr__ = ["ds", "rra", "mode"] def __init__(self): self.mode = None super(RRDMapper, self).__init__() self.rra = [] def getData(self): """ """ if not (self.ds or self.rra): self.map() data = super(RRDMapper, self).getData() data["ds"] = [ds.getData() for ds in self.ds] data["rra"] = [rra.getData() for rra in self.rra] return data def printInfo(self): super(RRDMapper, self).printInfo() for ds in self.ds: ds.printInfo() for index, rra in enumerate(self.rra): rra.printInfo(index) def map(self): """ The map method does several things: 1) if the RRD object (instantiated from this class or a subclass) is in "write" mode, there is no need to parse the XML and map it; there is already an object representation. In this case, the majority of this method is skipped. 2) if the RRD object is in "read" mode, it needs to pull data out of the rrd file; it does this by loading (which dumps to XML and then reads in the XML). 3) once the XML has been parsed, it maps the XML to objects. """ if self.mode == "w": return # The backend is defined by the subclass of this class, as is the # filename. tree = self.backend.load(self.filename) node = RRDXMLNode(tree) super(RRDMapper, self).map(node) for subNode in node.ds: ds = DSMapper() ds.map(subNode) self.ds.append(ds) for subNode in node.rra: rra = RRAMapper() rra.map(subNode) self.rra.append(rra) PyRRD-0.1.0/pyrrd/meta.py000644 000765 000024 00000000372 11635456505 016111 0ustar00oubiwannstaff000000 000000 display_name = "PyRRD" library_name = "pyrrd" version = "0.1.0" author = "Duncan McGreggor" author_email = "duncan@canonical.com" license = "BSD" url = "http://code.google.com/p/pyrrd/" description = "An Object-Oriented Python Interface for RRDTool" PyRRD-0.1.0/pyrrd/node.py000644 000765 000024 00000007762 11635456135 016121 0ustar00oubiwannstaff000000 000000 """ This module's classes are used to represent RRD data in XML format. The mapper module uses this format to establish a relationship between RRD files (and their exports) and Python objects. """ class XMLNode(object): """ A base class. Not used directly. """ def __init__(self, tree, attribute_names): self.tree = tree self.attributes = {} for name, cast, default in attribute_names: try: value = cast(self.getAttribute(name)) except ValueError: value = default if not value: value = default self.attributes[name] = value def getAttribute(self, attrName): """ """ node = self.tree.find(attrName) if node != None: return node.text.strip() raise ValueError() class DSXMLNode(XMLNode): """ An object abstraction for the node in the XML RRD export. This is a child of the node, and thus this class is used in the RRDXMLNode class. Currently provides no featres beyond those of the base XML node class. """ class CDPPrepXMLNode(XMLNode): """ An object abstraction for the node in the XML RRD export. The nodes are children node of an node. """ def __init__(self, node): self.ds = [] dsAttributes = [ ("primary_value", float, 0.0), ("secondary_value", float, 0.0), ("value", float, 0.0), ("unknown_datapoints", int, 0), ] for ds in node.findall("ds"): self.ds.append(DSXMLNode(ds, dsAttributes)) class DatabaseNode(XMLNode): """ An object abstraction for the node in the XML RRD export. Currently unimplemented. """ def __init__(self, node): super(DatabaseNode, self).__init__(node, []) self.row = [] class RRAXMLNode(XMLNode): """ An object abstraction for the node in the XML RRD export. The nodes are children of the node. """ def __init__(self, tree, attributes, includeData=False): super(RRAXMLNode, self).__init__(tree, attributes) self.database = None xff = self.tree.find('params').find('xff') if xff!=None: xff = float(xff.text) self.attributes["xff"] = xff self.cdp_prep = CDPPrepXMLNode(self.tree.find("cdp_prep")) if includeData: db = self.tree.get("database") self.database = DatabaseNode(db) def getAttribute(self, attrName): """ """ if attrName.lower() == "xff": return self.tree.findtext("params/xff").strip() else: return super(RRAXMLNode, self).getAttribute(attrName) class RRDXMLNode(XMLNode): """ An object abstraction for the node in the XML RRD export. This is the top-level node in the XML RRD export. """ def __init__(self, tree, includeData=False): attributes = [ ("version", int, 0), ("step", int, 300), ("lastupdate", int, 0), ] super(RRDXMLNode, self).__init__(tree, attributes) dsAttributes = [ ("name", str, ""), ("type", str, "GAUGE"), ("minimal_heartbeat", int, 300), ("min", int, "NaN"), ("max", int, "NaN"), ("last_ds", int, 0), ("value", float, 0.0), ("unknown_sec", int, 0), ] rraAttributes = [ ("cf", str, "AVERAGE"), ("pdp_per_row", int, 0), ] self.ds = [] self.rra = [] for ds in self.getDataSources(): self.ds.append(DSXMLNode(ds, dsAttributes)) for rra in self.getRRAs(): self.rra.append(RRAXMLNode(rra, rraAttributes, includeData)) def getDataSources(self): """ """ return self.tree.findall("ds") def getRRAs(self): """ """ return self.tree.findall("rra") PyRRD-0.1.0/pyrrd/rrd.py000644 000765 000024 00000042134 11635456135 015753 0ustar00oubiwannstaff000000 000000 import re from datetime import datetime from pyrrd import mapper from pyrrd import util from pyrrd.backend import external def validateDSName(name): """ >>> vname = validateDSName('Zaphod Beeble-Brox!') Traceback (most recent call last): ValueError: Names must consist only of the characters A-Z, a-z, 0-9, _ >>> vname = validateDSName('Zaphod_Bee_Brox') >>> vname = validateDSName('a'*18) >>> vname = validateDSName('a'*19) Traceback (most recent call last): ValueError: Names must be shorter than 19 characters """ if name != re.sub('[^A-Za-z0-9_]', '', name): raise ValueError, "Names must consist only of the characters " + \ "A-Z, a-z, 0-9, _" if len(name) > 18: raise ValueError, "Names must be shorter than 19 characters" def validateDSType(dsType): """ >>> validateDSType('counter') 'COUNTER' >>> validateDSType('ford prefect') Traceback (most recent call last): ValueError: A data source type must be one of the following: GAUGE COUNTER DERIVE ABSOLUTE COMPUTE """ dsType = dsType.upper() valid = ['GAUGE', 'COUNTER', 'DERIVE', 'ABSOLUTE', 'COMPUTE'] if dsType in valid: return dsType else: valid = ' '.join(valid) raise ValueError, 'A data source type must be one of the ' + \ 'following: %s' % valid def validateRRACF(consolidationFunction): """ >>> validateRRACF('Max') 'MAX' >>> validateRRACF('Maximum') Traceback (most recent call last): ValueError: An RRA's consolidation function must be one of the following: AVERAGE MIN MAX LAST HWPREDICT SEASONAL DEVSEASONAL DEVPREDICT FAILURES >>> validateRRACF('Trisha MacMillan') Traceback (most recent call last): ValueError: An RRA's consolidation function must be one of the following: AVERAGE MIN MAX LAST HWPREDICT SEASONAL DEVSEASONAL DEVPREDICT FAILURES """ cf = consolidationFunction.upper() valid = ['AVERAGE', 'MIN', 'MAX', 'LAST', 'HWPREDICT', 'SEASONAL', 'DEVSEASONAL', 'DEVPREDICT', 'FAILURES'] if cf in valid: return cf else: valid = ' '.join(valid) raise ValueError, "An RRA's consolidation function must be " + \ "one of the following: %s" % valid class RRD(mapper.RRDMapper): """ >>> import os, tempfile >>> >>> dss = [] >>> rras = [] >>> rrdfile = tempfile.NamedTemporaryFile() >>> dss.append(DataSource(dsName='speed', dsType='COUNTER', heartbeat=600)) >>> rras.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> rras.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) >>> rrd = RRD(rrdfile.name, ds=dss, rra=rras, start=920804400) >>> rrd.create() >>> os.path.exists(rrdfile.name) True >>> rrd.bufferValue('920805600', '12363') >>> rrd.bufferValue('920805900', '12363') >>> rrd.bufferValue('920806200', '12373') >>> rrd.bufferValue('920806500', '12383') >>> rrd.update() >>> rrd.bufferValue('920806800', '12393') >>> rrd.bufferValue('920807100', '12399') >>> rrd.bufferValue('920807400', '12405') >>> rrd.bufferValue('920807700', '12411') >>> rrd.bufferValue('920808000', '12415') >>> rrd.bufferValue('920808300', '12420') >>> rrd.bufferValue('920808600', '12422') >>> rrd.bufferValue('920808900', '12423') >>> rrd.update() >>> len(rrd.values) 0 """ def __init__(self, filename=None, start=None, step=300, ds=None, rra=None, mode="w", backend=external): super(RRD, self).__init__() if filename == None: raise ValueError, "You must provide a filename." self.filename = filename if not start or isinstance(start, datetime): self.start = util.epoch(start) else: self.start = start if not ds: ds = [] if not rra: rra = [] self.ds = ds self.rra = rra self.values = [] self.step = step self.lastupdate = None self.mode = mode # the backend attribute needs to be defined before the load call, since # the load method (super class) expects the backend attribute self.backend = backend if self.mode == "r": self.load() def bufferValue(self, timeOrData, *values): """ The parameter 'values' can either be a an n-tuple, but it is assumed that the order in which the values are sent is the order in which they will be applied to the DSs (i.e., respectively... i.e., in the order that the DSs were added to the RRD). >>> my_rrd = RRD('somefile') >>> my_rrd.bufferValue('1000000', 'value') >>> my_rrd.update(debug=True, dryRun=True) ('somefile', [u'1000000:value']) >>> my_rrd.update(template='ds0', debug=True, dryRun=True) ('somefile', ['--template', u'ds0', u'1000000:value']) >>> my_rrd.values = [] >>> my_rrd.bufferValue('1000000:value') >>> my_rrd.update(debug=True, dryRun=True) ('somefile', ['1000000:value']) >>> my_rrd.update(template='ds0', debug=True, dryRun=True) ('somefile', ['--template', u'ds0', '1000000:value']) >>> my_rrd.values = [] >>> my_rrd.bufferValue('1000000', 'value1', 'value2') >>> my_rrd.bufferValue('1000001', 'value3', 'value4') >>> my_rrd.update(debug=True, dryRun=True) ('somefile', [u'1000000:value1:value2', u'1000001:value3:value4']) >>> my_rrd.update(template=u'ds1:ds0', debug=True, dryRun=True) ('somefile', ['--template', u'ds1:ds0', u'1000000:value1:value2', u'1000001:value3:value4']) >>> my_rrd.values = [] >>> my_rrd.bufferValue('1000000:value') >>> my_rrd.bufferValue('1000001:anothervalue') >>> my_rrd.update(debug=True, dryRun=True) ('somefile', ['1000000:value', '1000001:anothervalue']) >>> my_rrd.update(template='ds0', debug=True, dryRun=True) ('somefile', ['--template', u'ds0', '1000000:value', '1000001:anothervalue']) >>> my_rrd.values = [] """ values = ':'.join([unicode(x) for x in values]) self.values.append((timeOrData, values)) self.lastupdate = float(unicode(timeOrData).split(":")[0]) # for backwards compatibility bufferValues = bufferValue def create(self, debug=False): data = self.backend.prepareObject('create', self) if debug: print data self.backend.create(*data) # XXX this can be uncommented when we're doing full database imports with # the loads method and storing those values in the python objects #def write(self, filename, debug=False): # self.filename = filename # if not os.path.exists(filename): # self.create(debug) # for rra in self.rra: # for row in rra.database.rows: # time, data = # self.bufferValue(time, data) # self.update() def update(self, debug=False, template=None, dryRun=False): """ """ # XXX this needs a lot more testing with different data # sources and values self.template = template if self.values: data = self.backend.prepareObject('update', self) if debug: print data if not dryRun: self.backend.update(debug=debug, *data) self.values = [] def fetch(self, cf="AVERAGE", resolution=None, start=None, end=None, returnStyle="ds", useBindings=False): """ By default, fetch returns a dict of data source names whose associated values are lists. The list for each DS contains (time, data) tuples. Optionally, one may pass returnStyle="time" and one will instead get a dict of times whose associated values are dicts. These associated dicts have a key for every defined DS and a corresponding value that is the data associated with that DS at the given time. # XXX add a doctest that creates an RRD with multiple DSs and RRAs """ attributes = util.Attributes() attributes.filename = self.filename attributes.cf = cf attributes.resolution = resolution attributes.start = start attributes.end = end data = self.backend.prepareObject('fetch', attributes) if useBindings: kwds = {"useBindings": useBindings} return self.backend.fetch(*data, **kwds) return self.backend.fetch(*data)[returnStyle] def info(self, useBindings=False, rawData=False, stream=None): """ For this method, the info is rendered to stdout, unless rawData is set to True. """ data = self.backend.prepareObject('info', self) kwds = { "useBindings": useBindings, "rawData": rawData, "stream": stream} result = self.backend.info(*data, **kwds) if rawData: return result def load(self, filename=None, includeData=False): """ # Create an empty file: >>> import os, tempfile >>> >>> dss = [] >>> rras = [] >>> rrdfile = tempfile.NamedTemporaryFile() >>> dss.append(DataSource(dsName='speed', dsType='COUNTER', ... heartbeat=600)) >>> rras.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> rras.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) >>> rrd = RRD(rrdfile.name, ds=dss, rra=rras, start=920804400) >>> rrd.create() >>> os.path.exists(rrdfile.name) True # Add some values: >>> rrd.bufferValue('920805600', '12363') >>> rrd.bufferValue('920805900', '12363') >>> rrd.bufferValue('920806200', '12373') >>> rrd.bufferValue('920806500', '12383') >>> rrd.update() # Let's create another one, using the source file we just created. Note # that by passing the "read" mode, were letting the RRD class know that # it should call load() immediately, thus giving us read-access to the # file's data. >>> rrd2 = RRD(rrdfile.name, mode="r") # Now let's load the data from self.filename: >>> top_level_attrs = rrd2.getData() >>> top_level_attrs["lastupdate"] 920806500 >>> top_level_attrs["filename"] == rrdfile.name True >>> top_level_attrs["step"] 300 >>> len(rrd2.ds) 1 >>> len(rrd2.rra) 2 >>> sorted(rrd2.ds[0].getData().keys()) ['last_ds', 'max', 'min', 'minimal_heartbeat', 'name', 'rpn', 'type', 'unknown_sec', 'value'] >>> sorted(rrd2.rra[1].getData().keys()) ['alpha', 'beta', 'cdp_prep', 'cf', 'database', 'ds', 'gamma', 'pdp_per_row', 'rows', 'rra_num', 'seasonal_period', 'steps', 'threshold', 'window_length', 'xff'] # Finally, a comparison: >>> rrd.lastupdate == rrd2.lastupdate True >>> rrd.filename == rrd2.filename True >>> rrd.step == rrd2.step True """ # XXX this should only be enabled once we have the data from the loaded # RRD file updating the RRD object #if filename: # self.filename = filename # this re-maps all attributes of this object (self) based on what is # read in from self.filename self.map() # XXX add support for loading data from the database XML tag; when this # is implemented, we will also need to come up with the best way to # write this data back to disk (write the individual rows of data that # get read in, that is) if includeData: pass class DataSource(mapper.DSMapper): """ A single RRD can accept input from several data sources (DS), for example incoming and outgoing traffic on a specific communication line. With the DS configuration option you must define some basic properties of each data source you want to store in the RRD. ds-name is the name you will use to reference this particular data source from an RRD. A ds-name must be 1 to 19 characters long in the characters [a-zA-Z0-9_]. DST defines the Data Source Type. The remaining arguments of a data source entry depend on the data source type. For GAUGE, COUNTER, DERIVE, and ABSOLUTE the format for a data source entry is: DS:ds-name:GAUGE | COUNTER | DERIVE | ABSOLUTE:heartbeat:min:max For COMPUTE data sources, the format is: DS:ds-name:COMPUTE:rpn-expression >>> ds = DataSource(dsName='speed', dsType='COUNTER', heartbeat=600) >>> ds DS:speed:COUNTER:600:U:U """ def __init__(self, dsName=None, dsType=None, heartbeat=None, minval='U', maxval='U', rpn=None): super(DataSource, self).__init__() if dsName == None: raise ValueError, "You must provide a name for the data source." if dsType == None: raise ValueError, "You must provide a type for the data source." self.name = dsName self.type = dsType self.minimal_heartbeat = heartbeat self.min = minval self.max = maxval self.rpn = rpn def __repr__(self): """ We override this method for preparing the class's data for use with RRDTool. Time representations must have their ':'s escaped, since the colon is the RRDTool separator for parameters. """ main = 'DS:%s:%s' % (self.name, self.type) tail = '' if self.type == 'COMPUTE': tail += ':%s' % self.rpn else: tail += ':%s:%s:%s' % ( self.minimal_heartbeat, self.min, self.max) return main + tail DS = DataSource class RRA(mapper.RRAMapper): """ The purpose of an RRD is to store data in the round robin archives (RRA). An archive consists of a number of data values or statistics for each of the defined data-sources (DS) and is defined with an RRA line. When data is entered into an RRD, it is first fit into time slots of the length defined with the -s option, thus becoming a primary data point. The data is also processed with the consolidation function (CF) of the archive. There are several consolidation functions that consolidate primary data points via an aggregate function: AVERAGE, MIN, MAX, LAST. The format of RRA line for these consolidation functions is: RRA:AVERAGE | MIN | MAX | LAST:xff:steps:rows xff The xfiles factor defines what part of a consolidation interval may be made up from *UNKNOWN* data while the consolidated value is still regarded as known. steps defines how many of these primary data points are used to build a consolidated data point which then goes into the archive. rows defines how many generations of data values are kept in an RRA. >>> rra1 = RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24) >>> rra1 RRA:AVERAGE:0.5:1:24 >>> rra2 = RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10) >>> rra2 RRA:AVERAGE:0.5:6:10 """ def __init__(self, cf=None, xff=None, steps=None, rows=None, alpha=None, beta=None, seasonal_period=None, rra_num=None, gamma=None, threshold=None, window_length=None, cdpPrepObject=None, databaseObject=None): super(RRA, self).__init__() if cf == None: msg = "You must provide a value for the consolidation function." raise ValueError, msg self.cf = cf self.xff = xff self.steps = steps self.rows = rows self.alpha = alpha self.beta = beta self.gamma = gamma self.seasonal_period = seasonal_period self.rra_num = rra_num self.threshold = threshold self.window_length = window_length # for object mapping self.cdp_prep = cdpPrepObject self.database = databaseObject def __repr__(self): """ We override this method for preparing the class's data for use with RRDTool. Time representations must have their ':'s escaped, since the colon is the RRDTool separator for parameters. """ main = 'RRA:%s' % self.cf tail = '' if self.cf in ['AVERAGE', 'MIN', 'MAX', 'LAST']: tail += ':%s:%s:%s' % (self.xff, self.steps, self.rows) elif self.cf == 'HWPREDICT': tail += ':%s:%s:%s:%s' % ( self.rows, self.alpha, self.beta, self.seasonal_period) if self.rra_num != None: tail += ':%s' % (self.rra_num) elif self.cf == 'SEASONAL': tail += ':%s:%s:%s' % ( self.seasonal_period, self.gamma, self.rra_num) elif self.cf == 'DEVSEASONAL': tail += ':%s:%s:%s' % ( self.seasonal_period, self.gamma, self.rra_num) elif self.cf == 'DEVPREDICT': tail += ':%s:%s' % (self.rows, self.rra_num) elif self.cf == 'FAILURES': tail += ':%s:%s:%s:%s' % ( self.rows, self.threshold, self.window_length, self.rra_num) return main+tail class Query(object): pass if __name__ == '__main__': from doctest import testmod testmod() PyRRD-0.1.0/pyrrd/testing/000755 000765 000024 00000000000 11635462441 016260 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/tests/000755 000765 000024 00000000000 11635462441 015745 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/util/000755 000765 000024 00000000000 11635462441 015560 5ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/util/__init__.py000644 000765 000024 00000001153 11635456135 017674 0ustar00oubiwannstaff000000 000000 from time import mktime from datetime import datetime try: from xml.etree import ElementTree except ImportError: from elementtree import ElementTree XML = ElementTree.XML def epoch(dt_obj=None): ''' >>> dt = datetime(1972, 8, 17) >>> epoch(dt) 82879200 >>> now = epoch() >>> type(now) ''' if not dt_obj: dt_obj = datetime.now() return int(mktime(dt_obj.timetuple())) class Attributes(object): """ A simple object for storing attributes. """ class NaN(float): def __repr__(self): return "nan" __str__ = __repr__ PyRRD-0.1.0/pyrrd/util/dist.py000644 000765 000024 00000006354 11635456135 017110 0ustar00oubiwannstaff000000 000000 import os import re from pyrrd import meta legalReSTFiles = [ 'README', 'TODO', 'DEPENDENCIES', ] def setup(*args, **kwds): """ Compatibility wrapper. """ try: from setuptools import setup except ImportError: from distutils.core import setup return setup(*args, **kwds) def findPackages(): """ Compatibility wrapper. Taken from storm setup.py. """ try: from setuptools import find_packages return find_packages() except ImportError: pass packages = [] for directory, subdirectories, files in os.walk(meta.library_name): if '__init__.py' in files: packages.append(directory.replace(os.sep, '.')) return packages def hasDocutils(): """ Check to see if docutils is installed. """ try: import docutils return True except ImportError: return False def _validateReST(text): """ Make sure that the given ReST text is valid. Taken from Zope Corp's zc.twist setup.py. """ import docutils.utils import docutils.parsers.rst import StringIO doc = docutils.utils.new_document('validator') # our desired settings doc.reporter.halt_level = 5 doc.reporter.report_level = 1 stream = doc.reporter.stream = StringIO.StringIO() # docutils buglets (?) doc.settings.tab_width = 2 doc.settings.pep_references = doc.settings.rfc_references = False doc.settings.trim_footnote_reference_space = None # and we're off... parser = docutils.parsers.rst.Parser() parser.parse(text, doc) return stream.getvalue() def validateReST(text): """ A wrapper that ensafens the validation for pythons that are not embiggened with docutils. """ if hasDocutils(): return _validateReST(text) print " *** No docutils; can't validate ReST." return '' def catReST(*args, **kwds): """ Concatenate the contents of one or more ReST files. Taken from Zope Corp's zc.twist setup.py. """ # note: distutils explicitly disallows unicode for setup values :-/ # http://docs.python.org/dist/meta-data.html tmp = [] for arg in args: if arg in legalReSTFiles or arg.endswith('.txt'): f = open(os.path.join(*arg.split('/'))) tmp.append(f.read()) f.close() tmp.append('\n\n') else: print "Warning: '%s' not a legal ReST filename." tmp.append(arg) if len(tmp) == 1: res = tmp[0] else: res = ''.join(tmp) out = kwds.get('out') stop_on_errors = kwds.get('stop_on_errors') if out is True: out = 'CHECK_THIS_BEFORE_UPLOAD.txt' if out: f = open(out, 'w') f.write(res) f.close() report = validateReST(res) if report: print report if stop_on_errors: print 'ReST validation error' print print 'See the following:' print ' http://docutils.sourceforge.net/docs/user/rst/cheatsheet.txt' print ' http://docutils.sourceforge.net/docs/user/rst/quickstart.html' print raise ValueError('ReST validation error') return res PyRRD-0.1.0/pyrrd/tests/__init__.py000644 000765 000024 00000000000 11635456135 020047 0ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/tests/test_mapper.py000644 000765 000024 00000004374 11635456135 020655 0ustar00oubiwannstaff000000 000000 from unittest import TestCase from pyrrd.mapper import RRDMapper from pyrrd.testing import dump from pyrrd.util import NaN, XML class FakeBackend(object): def __init__(self, tree): self.tree = tree def load(self, filename): return self.tree class RRDMapperTestCase(TestCase): def setUp(self): self.tree = XML(dump.simpleDump01) def makeMapper(self): rrd = RRDMapper() rrd.filename = None rrd.backend = FakeBackend(self.tree) rrd.map() return rrd def test_map(self): rrd = self.makeMapper() self.assertEquals(rrd.version, 3) self.assertEquals(rrd.step, 300) self.assertEquals(rrd.lastupdate, 920804400) def test_mapDS(self): rrd = self.makeMapper() self.assertEquals(len(rrd.ds), 1) ds = rrd.ds[0] self.assertEquals(ds.name, "speed") self.assertEquals(ds.type, "COUNTER") self.assertEquals(ds.minimal_heartbeat, 600) self.assertEquals(ds.min, "NaN") self.assertEquals(ds.max, "NaN") def test_mapRRA(self): rrd = self.makeMapper() self.assertEquals(len(rrd.rra), 2) rra1 = rrd.rra[0] self.assertEquals(rra1.cf, "AVERAGE") self.assertEquals(rra1.pdp_per_row, 1) rra2 = rrd.rra[1] self.assertEquals(rra2.cf, "AVERAGE") self.assertEquals(rra2.pdp_per_row, 6) def test_mapRRAParams(self): rrd = self.makeMapper() rra1 = rrd.rra[0] self.assertEquals(rra1.xff, 0.5) rra2 = rrd.rra[1] self.assertEquals(rra2.xff, 0.5) def test_mapRRACDPPrep(self): rrd = self.makeMapper() ds1 = rrd.rra[0].ds self.assertEquals(len(ds1), 1) self.assertEquals(ds1[0].primary_value, 0.0) self.assertEquals(ds1[0].secondary_value, 0.0) self.assertEquals(str(ds1[0].value), str(NaN())) self.assertEquals(ds1[0].unknown_datapoints, 0) ds2 = rrd.rra[1].ds self.assertEquals(len(ds2), 1) self.assertEquals(len(ds1), 1) self.assertEquals(ds2[0].primary_value, 0.0) self.assertEquals(ds2[0].secondary_value, 0.0) self.assertEquals(str(ds2[0].value), str(NaN())) self.assertEquals(ds2[0].unknown_datapoints, 0) PyRRD-0.1.0/pyrrd/tests/test_node.py000644 000765 000024 00000005075 11635456135 020315 0ustar00oubiwannstaff000000 000000 from unittest import TestCase from pyrrd.node import RRDXMLNode from pyrrd.testing import dump from pyrrd.util import XML class RRDXMLNodeTestCase(TestCase): def setUp(self): self.tree = XML(dump.simpleDump01) def test_creation(self): rrd = RRDXMLNode(self.tree) self.assertEquals(rrd.getAttribute("version"), "0003") self.assertEquals(rrd.getAttribute("step"), "300") self.assertEquals(rrd.getAttribute("lastupdate"), "920804400") def test_creationDS(self): dsChecks = [ ("name", "speed"), ("type", "COUNTER"), ("minimal_heartbeat", "600"), ("min", "NaN"), ("max", "NaN"), ("last_ds", "UNKN"), ("value", "0.0000000000e+00"), ("unknown_sec", "0")] rrd = RRDXMLNode(self.tree) self.assertEquals(len(rrd.ds), 1) ds = rrd.ds[0] for name, value in dsChecks: self.assertEquals(ds.getAttribute(name), value) def test_creationRRA(self): rra1Checks = [ ("cf", "AVERAGE"), ("pdp_per_row", "1")] rra2Checks = [ ("cf", "AVERAGE"), ("pdp_per_row", "6")] rrd = RRDXMLNode(self.tree) self.assertEquals(len(rrd.rra), 2) rra1 = rrd.rra[0] for name, value in rra1Checks: self.assertEquals(rra1.getAttribute(name), value) rra2 = rrd.rra[1] for name, value in rra2Checks: self.assertEquals(rra2.getAttribute(name), value) def test_creationRRAParams(self): rrd = RRDXMLNode(self.tree) self.assertEquals(len(rrd.rra), 2) rra1 = rrd.rra[0] self.assertEquals(rra1.getAttribute("xff"), "5.0000000000e-01") rra2 = rrd.rra[1] self.assertEquals(rra2.getAttribute("xff"), "5.0000000000e-01") def test_creationRRACDPPrep(self): dsChecks = [ ("primary_value", "0.0000000000e+00"), ("secondary_value", "0.0000000000e+00"), ("value", "NaN"), ("unknown_datapoints", "0")] rrd = RRDXMLNode(self.tree) cdpPrep1 = rrd.rra[0].cdp_prep self.assertEquals(len(cdpPrep1.ds), 1) for name, value in dsChecks: self.assertEquals(cdpPrep1.ds[0].getAttribute(name), value) cdpPrep2 = rrd.rra[1].cdp_prep self.assertEquals(len(cdpPrep2.ds), 1) for name, value in dsChecks: self.assertEquals(cdpPrep2.ds[0].getAttribute(name), value) def test_creationIncludeData(self): rrd = RRDXMLNode(self.tree, includeData=True) PyRRD-0.1.0/pyrrd/tests/test_rrd.py000644 000765 000024 00000004712 11635456135 020154 0ustar00oubiwannstaff000000 000000 from unittest import TestCase from pyrrd.rrd import DataSource, RRA, RRD class RRDTestCase(TestCase): def test_creationDefaults(self): filename = '/tmp/test.rrd' rrd = RRD(filename, start=920804400) self.assertEquals(rrd.filename, filename) self.assertEquals(rrd.ds, []) self.assertEquals(rrd.rra, []) self.assertEquals(rrd.values, []) self.assertEquals(rrd.step, 300) self.assertEquals(rrd.lastupdate, None) def test_creation(self): dss = [] rras = [] filename = '/tmp/test.rrd' dss.append(DataSource(dsName='speed', dsType='COUNTER', heartbeat=600)) rras.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) rras.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) rrd = RRD(filename, ds=dss, rra=rras, start=920804400) self.assertEquals(rrd.filename, filename) self.assertEquals(repr(rrd.ds), "[DS:speed:COUNTER:600:U:U]") self.assertEquals( repr(rrd.rra), "[RRA:AVERAGE:0.5:1:24, RRA:AVERAGE:0.5:6:10]") self.assertEquals(rrd.values, []) self.assertEquals(rrd.step, 300) self.assertEquals(rrd.lastupdate, None) def test_creationDSsAndRRAs(self): dss1 = [] rras1 = [] filename = '/tmp/test1.rrd' dss1.append(DataSource(dsName='speed', dsType='COUNTER', heartbeat=600)) rras1.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) rras1.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) rrd1 = RRD(filename, ds=dss1, rra=rras1, start=920804400) self.assertEquals(repr(rrd1.ds), "[DS:speed:COUNTER:600:U:U]") self.assertEquals( repr(rrd1.rra), "[RRA:AVERAGE:0.5:1:24, RRA:AVERAGE:0.5:6:10]") filename = '/tmp/test2.rrd' rrd2 = RRD(filename, start=920804400) self.assertEquals(rrd2.ds, []) self.assertEquals(rrd2.rra,[]) dss3 = [] rras3 = [] filename = '/tmp/test3.rrd' dss3.append(DataSource(dsName='speed', dsType='COUNTER', heartbeat=300)) rras3.append(RRA(cf='AVERAGE', xff=0.5, steps=2, rows=24)) rras3.append(RRA(cf='AVERAGE', xff=0.5, steps=12, rows=10)) rrd3 = RRD(filename, ds=dss3, rra=rras3, start=920804400) self.assertEquals(repr(rrd3.ds), "[DS:speed:COUNTER:300:U:U]") self.assertEquals( repr(rrd3.rra), "[RRA:AVERAGE:0.5:2:24, RRA:AVERAGE:0.5:12:10]") PyRRD-0.1.0/pyrrd/testing/__init__.py000644 000765 000024 00000000000 11635456135 020362 0ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/testing/base.py000644 000765 000024 00000000712 11635456135 017547 0ustar00oubiwannstaff000000 000000 import tempfile from unittest import TestCase from pyrrd.backend.external import create class RRDBaseTestCase(TestCase): def setUp(self): self.rrdfile = tempfile.NamedTemporaryFile() self.filename = self.rrdfile.name parameters = ( "--start 920804400 " "DS:speed:COUNTER:600:-10.1:10.3 " "RRA:AVERAGE:0.5:1:24 " "RRA:AVERAGE:0.5:6:10") create(self.filename, parameters) PyRRD-0.1.0/pyrrd/testing/dump.py000644 000765 000024 00000007660 11635456135 017613 0ustar00oubiwannstaff000000 000000 # This dump is from an rrd file with one data source and two RRAs. simpleDump01 = """ 0003 300 920804400 speed COUNTER 600 NaN NaN UNKN 0.0000000000e+00 0 AVERAGE 1 5.0000000000e-01 0.0000000000e+00 0.0000000000e+00 NaN 0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN AVERAGE 6 5.0000000000e-01 0.0000000000e+00 0.0000000000e+00 NaN 0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN """ PyRRD-0.1.0/pyrrd/testing/suite.py000644 000765 000024 00000007375 11635456135 020002 0ustar00oubiwannstaff000000 000000 """ Utility functions for testing. """ import os import unittest import doctest def importModule(name): mod = __import__(name) components = name.split('.') for comp in components[1:]: mod = getattr(mod, comp) return mod def fileCheck(path, skipFiles=[]): if not os.path.isfile(path): return False filename = os.path.basename(path) if filename in skipFiles: return False if path in skipFiles: print "skip it!" return False return True def fileIsTest(path, skipFiles=[]): result = fileCheck(path, skipFiles) if not result: return False filename = os.path.basename(path) if filename.startswith('test') and filename.endswith('.py'): return True def fileHasDoctests(path, skipFiles=[]): result = fileCheck(path, skipFiles) if result and path.endswith('.py'): fh = open(path) if '>>>' in fh.read(): result = True else: result = False fh.close() else: result = False return result def find(start, func, skip=[]): for item in [os.path.join(start, x) for x in os.listdir(start)]: if func(item, skip): yield item if os.path.isdir(item): for subItem in find(item, func, skip): yield subItem def findTests(startDir, skipFiles=[]): return find(startDir, fileIsTest, skipFiles) def findDoctests(startDir, skipFiles=[]): return find(startDir, fileHasDoctests, skipFiles) def _buildDoctestSuiteFromModules(modules, skip): suite = unittest.TestSuite() for modname in modules: mod = importModule(modname) if mod not in skip: suite.addTest(doctest.DocTestSuite(mod)) return suite def _buildDoctestSuiteFromFiles(files, skip): suite = [] for file in files: if file not in skip: suite.append(DocFileSuite(file)) return suite def _buildDoctestSuiteFromPaths(paths=[], skip=[]): """ paths: a list of directories to search skip: a list of file names to skip """ suite = unittest.TestSuite() loader = unittest.TestLoader() for startDir in paths: for testFile in findDoctests(startDir, skip): modBase = os.path.splitext(testFile)[0] name = modBase.replace(os.path.sep, '.') if name in skip: continue mod = importModule(name) suite.addTest(doctest.DocTestSuite(mod)) return suite def buildDoctestSuites(modules=[], files=[], paths=[], skip=[]): suite = [] if modules: suite.extend(_buildDoctestSuiteFromModules(modules, skip)) if files: suite.extend(_buildDoctestSuiteFromFiles(files, skip)) if paths: suite.extend(_buildDoctestSuiteFromPaths(paths, skip)) return suite def buildUnittestSuites(paths=[], skip=[]): """ paths: a list of directories to search skip: a list of file names to skip """ suites = [] loader = unittest.TestLoader() for startDir in paths: for testFile in findTests(startDir, skip): modBase = os.path.splitext(testFile)[0] name = modBase.replace(os.path.sep, '.') # import the testFile as a module mod = importModule(name) # iterate through module objects, checking for TestCases for objName in dir(mod): if not objName.endswith('TestCase'): continue obj = getattr(mod, objName) if not issubclass(obj, unittest.TestCase): continue # create a suite from any test cases suite = loader.loadTestsFromTestCase(obj) # append to suites list suites.append(suite) return suites PyRRD-0.1.0/pyrrd/backend/__init__.py000644 000765 000024 00000000000 11635456135 020274 0ustar00oubiwannstaff000000 000000 PyRRD-0.1.0/pyrrd/backend/bindings.py000644 000765 000024 00000037236 11635456135 020357 0ustar00oubiwannstaff000000 000000 """ The following exercises the RRD class with this backend: Create an RRD file programmatically:: >>> import tempfile >>> from pyrrd.rrd import DataSource, RRA, RRD >>> from pyrrd.backend import bindings >>> rrdfile = "/tmp/tmprrdfile.rrd" >>> dataSources = [] >>> roundRobinArchives = [] >>> dataSource = DataSource( ... dsName='speed', dsType='COUNTER', heartbeat=600) >>> dataSources.append(dataSource) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=1, rows=24)) >>> roundRobinArchives.append(RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)) >>> myRRD = RRD(rrdfile, ds=dataSources, rra=roundRobinArchives, ... start=920804400, backend=bindings) >>> myRRD.create() Let's check to see that the file exists:: >>> import os >>> os.path.isfile(rrdfile) True Let's see how big it is:: >>> bytes = len(open(rrdfile).read()) >>> 800 < bytes < 1200 True In order to save writes to disk, PyRRD buffers values and then writes the values to the RRD file at one go:: >>> myRRD.bufferValue('920805600', '12363') >>> myRRD.bufferValue('920805900', '12363') >>> myRRD.bufferValue('920806200', '12373') >>> myRRD.bufferValue('920806500', '12383') >>> myRRD.update() Let's add some more data:: >>> myRRD.bufferValue('920806800', '12393') >>> myRRD.bufferValue('920807100', '12399') >>> myRRD.bufferValue('920807400', '12405') >>> myRRD.bufferValue('920807700', '12411') >>> myRRD.bufferValue('920808000', '12415') >>> myRRD.bufferValue('920808300', '12420') >>> myRRD.bufferValue('920808600', '12422') >>> myRRD.bufferValue('920808900', '12423') >>> myRRD.update() Info checks when the RRD object is in write mode:: >>> myRRD.info() # doctest:+ELLIPSIS lastupdate = 920808900 rra = [{'rows': 24, 'database': None, 'cf': 'AVERAGE', 'cdp_prep': None, 'beta': None, 'seasonal_period': None, 'steps': 1, 'window_length': None, 'threshold': None, 'alpha': None, 'pdp_per_row': None, 'xff': 0.5, 'ds': [], 'gamma': None, 'rra_num': None}, {'rows': 10, 'database': None, 'cf': 'AVERAGE', 'cdp_prep': None, 'beta': None, 'seasonal_period': None, 'steps': 6, 'window_length': None, 'threshold': None, 'alpha': None, 'pdp_per_row': None, 'xff': 0.5, 'ds': [], 'gamma': None, 'rra_num': None}] filename = /tmp/... start = 920804400 step = 300 values = [] ds = [{'name': 'speed', 'min': 'U', 'max': 'U', 'unknown_sec': None, 'minimal_heartbeat': 600, 'value': None, 'rpn': None, 'type': 'COUNTER', 'last_ds': None}] ds[speed].name = speed ds[speed].min = U ds[speed].max = U ds[speed].minimal_heartbeat = 600 ds[speed].type = COUNTER rra[0].rows = 24 rra[0].cf = AVERAGE rra[0].steps = 1 rra[0].xff = 0.5 rra[0].ds = [] rra[1].rows = 10 rra[1].cf = AVERAGE rra[1].steps = 6 rra[1].xff = 0.5 rra[1].ds = [] In order to create a graph, we'll need some data definitions. We'll also throw in some calculated definitions and variable definitions for good meansure:: >>> from pyrrd.graph import DEF, CDEF, VDEF, LINE, AREA, GPRINT >>> def1 = DEF(rrdfile=myRRD.filename, vname='myspeed', ... dsName=dataSource.name) >>> cdef1 = CDEF(vname='kmh', rpn='%s,3600,*' % def1.vname) >>> cdef2 = CDEF(vname='fast', rpn='kmh,100,GT,kmh,0,IF') >>> cdef3 = CDEF(vname='good', rpn='kmh,100,GT,0,kmh,IF') >>> vdef1 = VDEF(vname='mymax', rpn='%s,MAXIMUM' % def1.vname) >>> vdef2 = VDEF(vname='myavg', rpn='%s,AVERAGE' % def1.vname) >>> line1 = LINE(value=100, color='#990000', legend='Maximum Allowed') >>> area1 = AREA(defObj=cdef3, color='#006600', legend='Good Speed') >>> area2 = AREA(defObj=cdef2, color='#CC6633', legend='Too Fast') >>> line2 = LINE(defObj=vdef2, color='#000099', legend='My Average', ... stack=True) >>> gprint1 = GPRINT(vdef2, '%6.2lf kph') Color is the spice of life. Let's spice it up a little:: >>> from pyrrd.graph import ColorAttributes >>> ca = ColorAttributes() >>> ca.back = '#333333' >>> ca.canvas = '#333333' >>> ca.shadea = '#000000' >>> ca.shadeb = '#111111' >>> ca.mgrid = '#CCCCCC' >>> ca.axis = '#FFFFFF' >>> ca.frame = '#AAAAAA' >>> ca.font = '#FFFFFF' >>> ca.arrow = '#FFFFFF' Now we can create a graph for the data in our RRD file:: >>> from pyrrd.graph import Graph >>> graphfile = tempfile.NamedTemporaryFile(suffix=".png") >>> g = Graph(graphfile.name, start=920805000, end=920810000, ... vertical_label='km/h', color=ca, backend=bindings) >>> g.data.extend([def1, cdef1, cdef2, cdef3, vdef1, vdef2, line1, area1, ... area2, line2, gprint1]) >>> g.write() Let's make sure it's there:: >>> os.path.isfile(graphfile.name) True Let's see how big it is:: >>> bytes = len(open(graphfile.name).read()) >>> bytes != 0 True >>> 8000 < bytes < 10700 True Open that up in your favorite image browser and confirm that the appropriate RRD graph is generated. # Cleanup: >>> os.unlink(rrdfile) >>> os.path.exists(rrdfile) False """ import rrdtool from pyrrd.backend import external from pyrrd.backend.common import buildParameters def _cmd(command, args, debug=False): function = getattr(rrdtool, command) # XXX fucntion calls barf if args aren't strings (can't handle unicode # right now) args = [str(x) for x in args] if debug: print "function:", function print "args:", args return function(*args) def create(filename, parameters): """ >>> rrdfile = '/tmp/test.rrd' >>> parameters = [ ... '--start', ... '920804400', ... 'DS:speed:COUNTER:600:U:U', ... 'RRA:AVERAGE:0.5:1:24', ... 'RRA:AVERAGE:0.5:6:10'] >>> create(rrdfile, parameters) # Check that the file's there: >>> import os >>> os.path.exists(rrdfile) True # Cleanup: >>> os.unlink(rrdfile) >>> os.path.exists(rrdfile) False """ parameters.insert(0, filename) output = _cmd('create', parameters, debug=True) def update(filename, parameters, debug=False): """ >>> rrdfile = '/tmp/test.rrd' >>> parameters = [ ... '--start', ... '920804400', ... 'DS:speed:COUNTER:600:U:U', ... 'RRA:AVERAGE:0.5:1:24', ... 'RRA:AVERAGE:0.5:6:10'] >>> create(rrdfile, parameters) >>> import os >>> os.path.exists(rrdfile) True >>> parameters = ['920804700:12345', '920805000:12357', '920805300:12363'] >>> update(rrdfile, parameters) >>> parameters = ['920805600:12363', '920805900:12363','920806200:12373'] >>> update(rrdfile, parameters) >>> parameters = ['920806500:12383', '920806800:12393','920807100:12399'] >>> update(rrdfile, parameters) >>> parameters = ['920807400:12405', '920807700:12411', '920808000:12415'] >>> update(rrdfile, parameters) >>> parameters = ['920808300:12420', '920808600:12422','920808900:12423'] >>> update(rrdfile, parameters) >>> os.unlink(rrdfile) >>> os.path.exists(rrdfile) False """ parameters.insert(0, filename) if debug: _cmd('updatev', parameters) else: _cmd('update', parameters) def fetch(filename, parameters, useBindings=False): """ By default, this function does not use the bindings for fetch. The reason for this is we want default compatibility with the data output/results from the fetch method for both the external and bindings modules. If a developer really wants to use the native bindings to get the fetch data, they may do so by explicitly setting the useBindings parameter. This will return data in the Python Python bindings format, though. Do be aware, though, that the PyRRD format is much easier to get data out of in a sensible manner (unless you really like the RRDTool approach). >>> rrdfile = '/tmp/test.rrd' >>> parameters = [ ... '--start', ... '920804400', ... 'DS:speed:COUNTER:600:U:U', ... 'RRA:AVERAGE:0.5:1:24', ... 'RRA:AVERAGE:0.5:6:10'] >>> create(rrdfile, parameters) >>> import os >>> os.path.exists(rrdfile) True >>> parameters = ['920804700:12345', '920805000:12357', '920805300:12363'] >>> update(rrdfile, parameters) >>> parameters = ['920805600:12363', '920805900:12363','920806200:12373'] >>> update(rrdfile, parameters) >>> parameters = ['920806500:12383', '920806800:12393','920807100:12399'] >>> update(rrdfile, parameters) >>> parameters = ['920807400:12405', '920807700:12411', '920808000:12415'] >>> update(rrdfile, parameters) >>> parameters = ['920808300:12420', '920808600:12422','920808900:12423'] >>> update(rrdfile, parameters) >>> parameters = ['AVERAGE', '--start', '920804400', '--end', '920809200'] >>> results = fetch(rrdfile, parameters, useBindings=True) >>> results[0] (920804400, 920809500, 300) >>> results[1] ('speed',) >>> len(results[2]) 17 # For more info on the PyRRD data format, see the docstring for # pyrrd.external.fetch. >>> parameters = ['AVERAGE', '--start', '920804400', '--end', '920809200'] >>> results = fetch(rrdfile, parameters, useBindings=False) >>> sorted(results["ds"].keys()) ['speed'] >>> os.unlink(rrdfile) >>> os.path.exists(rrdfile) False """ if useBindings: parameters.insert(0, filename) return _cmd('fetch', parameters) else: return external.fetch(filename, external.concat(parameters)) def dump(filename, outfile="", parameters=[]): """ The rrdtool Python bindings don't have support for dump, so we need to use the external dump function. >>> rrdfile = '/tmp/test.rrd' >>> parameters = [ ... '--start', ... '920804400', ... 'DS:speed:COUNTER:600:U:U', ... 'RRA:AVERAGE:0.5:1:24', ... 'RRA:AVERAGE:0.5:6:10'] >>> create(rrdfile, parameters) >>> xml = dump(rrdfile) >>> xmlBytes = len(xml) >>> 3300 < xmlBytes < 4000 True >>> xmlCommentCheck = ' 60/1 * 24 rra1 = RRA(cf='AVERAGE', xff=0, steps=1, rows=1440) # 7 days-worth of five-minute samples --> 60/5 * 24 * 7 rra2 = RRA(cf='AVERAGE', xff=0, steps=5, rows=2016) # 30 days-worth of five-minute samples --> 60/60 * 24 * 30 rra3 = RRA(cf='AVERAGE', xff=0, steps=60, rows=720) rras.extend([rra1, rra2, rra3]) # With those setup, we can now created the RRD myRRD = RRD(filename, step=step, ds=dss, rra=rras, start=startTime) myRRD.create(debug=False) # Let's suck in that data... the data file has the following format: # DS TIME:VALUE [TIME:VALUE [TIME:VALUE]...] # and the lines are in a completely arbitrary order. data = {} # First, we need to get everything indexed by time for line in open(datafile).readlines(): line = line.strip() lineParts = line.split(' ') dsName = lineParts[0] for timedatum in lineParts[1:]: time, datum = timedatum.split(':') # For each time index, let's have a dict data.setdefault(time, {}) # Now let's add the DS names and its data for this time to the # dict we just created data[time].setdefault(dsName, datum) # Sort everything by time counter = 0 sortedData = [ (i,data[i]) for i in sorted(data.keys()) ] for time, dsNames in sortedData: counter += 1 val1 = dsNames.get(ds1.name) or 'U' val2 = dsNames.get(ds2.name) or 'U' val3 = dsNames.get(ds3.name) or 'U' val4 = dsNames.get(ds4.name) or 'U' # Add the values myRRD.bufferValue(time, val1, val2, val3, val4) # Lets update the RRD/purge the buffer ever 100 entires if counter % 100 == 0: myRRD.update(debug=False) # Add anything remaining in the buffer myRRD.update(debug=False) # Let's set up the objects that will be added to the graph def1 = DEF(rrdfile=myRRD.filename, vname='in', dsName=ds1.name) def2 = DEF(rrdfile=myRRD.filename, vname='out', dsName=ds2.name) # Here we're just going to mulitply the in bits by 100, solely for # the purpose of display cdef1 = CDEF(vname='hundredin', rpn='%s,%s,*' % (def1.vname, 100)) cdef2 = CDEF(vname='negout', rpn='%s,-1,*' % def2.vname) area1 = AREA(defObj=cdef1, color='#FFA902', legend='Bits In') area2 = AREA(defObj=cdef2, color='#A32001', legend='Bits Out') # Let's configure some custom colors for the graph ca = ColorAttributes() ca.back = '#333333' ca.canvas = '#333333' ca.shadea = '#000000' ca.shadeb = '#111111' ca.mgrid = '#CCCCCC' ca.axis = '#FFFFFF' ca.frame = '#AAAAAA' ca.font = '#FFFFFF' ca.arrow = '#FFFFFF' # Now that we've got everything set up, let's make a graph g = Graph('dummy.png', end=endTime, vertical_label='Bits', color=ca) g.data.extend([def1, def2, cdef1, cdef2, area2, area1]) g.title = '"In- and Out-bound Traffic Across Local Router"' #g.logarithmic = ' ' # Iterate through the different resoltions for which we want to # generate graphs. for time, step in times: # First, the small graph g.filename = graphfile % (exampleNum, time) g.width = 400 g.height = 100 g.start=endTime - time g.step = step g.write(debug=False) # Then the big one g.filename = graphfileLg % (exampleNum, time) g.width = 800 g.height = 400 g.write() PyRRD-0.1.0/docs/ACKNOWLEDGEMENTS.txt000644 000765 000024 00000000720 11635460704 017465 0ustar00oubiwannstaff000000 000000 ================ Acknowledgements ================ The following members of the community have provided valuable contributions to this project: * Ravi Bhalotia, Allen Lerner, Mike Carrick and the U.S. Department of Veterans Affairs * AdytumSolutions, Inc., E-Secure Systems * nasvos, Leem Smit, Aaron Westendorf and Agora Games * Mladen Milankovic, Denis Fortin * Joseph Heck, Jean-Baptiste Quenot * Pavel Shramov, Brad Beattie, Colin Horsington Thanks! PyRRD-0.1.0/docs/FOOTNOTES.txt000644 000765 000024 00000000364 11635456135 016522 0ustar00oubiwannstaff000000 000000 ========== References ========== .. [#] http://code.google.com/p/pyrrd/wiki/FrontPage?tm=6 .. [#] http://code.google.com/p/pyrrd/wiki/FullWorkingExamples .. [#] http://oss.oetiker.ch/rrdtool/ .. [#] http://effbot.org/zone/element-index.htm PyRRD-0.1.0/docs/HISTORY.txt000644 000765 000024 00000003032 11635461264 016275 0ustar00oubiwannstaff000000 000000 ======= Changes ======= From 0.0.7 to 0.1.0 ------------------- This version marks a significant update to the code base. Some 70+ commits have been made to trunk since the last release in the period from March 2009 to September 2011. Some of the most signficant changes include the following: * Added a wrapper for the Python bindings. * Improvements in tests and testing infrastructure. * Improved exception handling. * Many changes and improvements to the RRD -> Python object mapper. * Improved support in Windows. * Graph formatting fixes. * Many bug fixes from community members. From 0.0.6 to 0.0.7 ------------------- * Packaging improvements and loads of documentation. From 0.0.5 to 0.0.6 ------------------- * Bug fix release (missing files in source package). From 0.0.4 to 0.0.5 ------------------- * Added support for retrieving and displaying RRD from RRD files. * Added an object mapper for RRD data (via XML files). * Added community-contributed improvements. From 0.0.3 to 0.0.4 ------------------- * Updated all the examples to work with the latest code. * Added community-contributed bug fix for Windows users. From 0.0.2 to 0.0.3 ------------------- * Minor code reorg. * Fixed doctests. * Various bug fixes. * Examples updates. From 0.0.1 to 0.0.2 ------------------- * Added license. * Added unit tests. * Added more examples. From 0 to 0.0.1 --------------- * Reorganized RRD code as donated from the CoyMon project. * Got basic rrdtool functionality represented as Python classes. * Code cleanup. PyRRD-0.1.0/docs/PRELUDE.txt000644 000765 000024 00000000056 11635456135 016240 0ustar00oubiwannstaff000000 000000 ~~~~~ PyRRD ~~~~~ .. contents:: :depth: 1