pax_global_header00006660000000000000000000000064130622210350014504gustar00rootroot0000000000000052 comment=c238d00a8bf2558e97e79a75309712dc0805fbd9 python-nxs/000077500000000000000000000000001306222103500131775ustar00rootroot00000000000000python-nxs/.gitignore000066400000000000000000000001151306222103500151640ustar00rootroot00000000000000*.swp *.pyc build dist NeXus.egg-info MANIFEST dist /.project /.pydevproject python-nxs/MANIFEST.in000066400000000000000000000000231306222103500147300ustar00rootroot00000000000000include README.rst python-nxs/README.rst000066400000000000000000000116141306222103500146710ustar00rootroot00000000000000================ Nexus Python API ================ Overview ======== NeXus Python Api binds the NeXus libraries to Python. It brings functionality of the NeXus API to Python for reading, writing and modifying NeXus Files. Python NeXus API imitates the functionality NeXus API though with a more object oriented flavour. * Information on NeXus Dataformat: http://www.nexusformat.org/Introduction Installation ============ Requirements ~~~~~~~~~~~~ The following software has to be installed on your system in order to successfully install and use the Python bindings for NAPI * Python 2.5 or higher * ``setuptools`` (replaces the old ``distutils`` code) * ``sphinx`` to build the documentation * a working installation of the runtime binaries of ``libNeXus`` * ``numpy``-package Supported operating systems are: Windows, OS X and Linux. The bindings should be easily modified for any version of Python which supports ctypes and numpy. In order to build the documentation `sphinx` is required. Building and Installing ~~~~~~~~~~~~~~~~~~~~~~~ This package uses the standard distutils installer for python .. code-block:: bash $ python setup.py install You will also need to make sure that `libNeXus` can be found. In order to build the documentation use .. code-block:: bash $ python setup.py build_sphinx To run the tests use .. code-block:: bash $ python setup.py test Using API from Python ===================== Test Files ~~~~~~~~~~ The Python NeXus-API includes nxstest.py, which provides similar tests to the original C api file napi_test.c. After installing, you can run the test using: .. code-block:: bash $ python [options] [formats] where options are ``-q`` for quiet and ``-x`` for external, and formats are ``hdf4``, ``hdf5`` and ``xml``. The default is to test ``hdf5`` format read/write. Using The API And An Example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The API's functions aim to reproduce the funtionality of the C API closely. Some low level functionality has been hidden from the user. Memory allocation functions NXmalloc and NXfree are done automatically in the API when needed. The file handle is an object with methods rather than a parameter to functions. Instead of checking status codes, errors raise exceptions. The input and returned values match the format of the data in the files. On return, python creates values of the correct type. However on input, numeric types must be created correctly using ``numpy.array(...,dtype='type')``. The matching datatypes are: ============== =============== NeXus datatype Python Datatype ============== =============== NX_CHAR char NX_FLOAT32 float32 NX_FLOAT64 float64 NX_UINT8 uint8 NX_INT16 int16 NX_UINT16 uint16 NX_INT32 int32 NX_UINT32 uint32 ============== =============== Here is simple example program that demonstrates the basic functions and most important differences between the C Nexus Api and the Python Nexus API. * Creates a NeXus file with access method HDF5 * adds datagroups * makes a data array of data type NX_INT32 * puts data to the array * reads the data and attributes * prints data and attribute value< * closes the groups and the file. .. code-block:: python import nxs,numpy # Access method accepts strings or integer (e.g., nxs.ACC_CREATE5) f = nxs.open("test.h5", 'w5') f.makegroup("testgroup", "NXentry") f.opengroup("testgroup", "NXentry") f.makegroup("anothergroup", "NXentry") # Some data to store in the file, this of type int16 data = numpy.array([[0,1,2,3],[4,5,6,7],[8,9,10,11],[12,13,14,15] ],'int16') # Make a data set for the array. Note that this could also # be done as f.makedata('data1','int16',[4,4]) f.makedata('data1', dtype=data.dtype, shape=data.shape) f.opendata("data1") f.putdata(data) # Attribute type can be inferred from the data or specified. If inferred, it # must match the type of the data. Attributes are scalars or strings, with # string length inferred from value. f.putattr('integer-attribute', 42, 'int16') f.putattr('double-attribute', 3.14159) f.closedata() # NeXus returns arrays from getattr/getdata/getslab f.opendata("data1") print 'data :',f.getdata() # getnext functions return tuples attrname,length,type = f.getnextattr () value = f.getattr(attrname, length, type) print 'first attribute: ', value # ... or you can use iterators for attrs and entries print 'all attributes' for attr,value in f.attrs(): print " %s: %s"%(attr,value) f.closedata() f.closegroup() f.close() NeXus API Routines ~~~~~~~~~~~~~~~~~~ Documentation for the individual methods, and how they differ from the basic NAPI methods is available from the Python command line. Rather than duplicate it here, use the following in Python: .. code-block:: python import nxs help(nxs) python-nxs/doc/000077500000000000000000000000001306222103500137445ustar00rootroot00000000000000python-nxs/doc/Makefile000066400000000000000000000152031306222103500154050ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Python-nxs.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Python-nxs.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/Python-nxs" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Python-nxs" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." python-nxs/doc/make.bat000066400000000000000000000150761306222103500153620ustar00rootroot00000000000000@ECHO OFF REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set BUILDDIR=build set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% source set I18NSPHINXOPTS=%SPHINXOPTS% source if NOT "%PAPER%" == "" ( set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% ) if "%1" == "" goto help if "%1" == "help" ( :help echo.Please use `make ^` where ^ is one of echo. html to make standalone HTML files echo. dirhtml to make HTML files named index.html in directories echo. singlehtml to make a single large HTML file echo. pickle to make pickle files echo. json to make JSON files echo. htmlhelp to make HTML files and a HTML help project echo. qthelp to make HTML files and a qthelp project echo. devhelp to make HTML files and a Devhelp project echo. epub to make an epub echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter echo. text to make text files echo. man to make manual pages echo. texinfo to make Texinfo files echo. gettext to make PO message catalogs echo. changes to make an overview over all changed/added/deprecated items echo. xml to make Docutils-native XML files echo. pseudoxml to make pseudoxml-XML files for display purposes echo. linkcheck to check all external links for integrity echo. doctest to run all doctests embedded in the documentation if enabled goto end ) if "%1" == "clean" ( for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i del /q /s %BUILDDIR%\* goto end ) %SPHINXBUILD% 2> nul if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.http://sphinx-doc.org/ exit /b 1 ) if "%1" == "html" ( %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/html. goto end ) if "%1" == "dirhtml" ( %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. goto end ) if "%1" == "singlehtml" ( %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml if errorlevel 1 exit /b 1 echo. echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. goto end ) if "%1" == "pickle" ( %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the pickle files. goto end ) if "%1" == "json" ( %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can process the JSON files. goto end ) if "%1" == "htmlhelp" ( %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run HTML Help Workshop with the ^ .hhp project file in %BUILDDIR%/htmlhelp. goto end ) if "%1" == "qthelp" ( %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp if errorlevel 1 exit /b 1 echo. echo.Build finished; now you can run "qcollectiongenerator" with the ^ .qhcp project file in %BUILDDIR%/qthelp, like this: echo.^> qcollectiongenerator %BUILDDIR%\qthelp\Python-nxs.qhcp echo.To view the help file: echo.^> assistant -collectionFile %BUILDDIR%\qthelp\Python-nxs.ghc goto end ) if "%1" == "devhelp" ( %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp if errorlevel 1 exit /b 1 echo. echo.Build finished. goto end ) if "%1" == "epub" ( %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub if errorlevel 1 exit /b 1 echo. echo.Build finished. The epub file is in %BUILDDIR%/epub. goto end ) if "%1" == "latex" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex if errorlevel 1 exit /b 1 echo. echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdf" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "latexpdfja" ( %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex cd %BUILDDIR%/latex make all-pdf-ja cd %BUILDDIR%/.. echo. echo.Build finished; the PDF files are in %BUILDDIR%/latex. goto end ) if "%1" == "text" ( %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text if errorlevel 1 exit /b 1 echo. echo.Build finished. The text files are in %BUILDDIR%/text. goto end ) if "%1" == "man" ( %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man if errorlevel 1 exit /b 1 echo. echo.Build finished. The manual pages are in %BUILDDIR%/man. goto end ) if "%1" == "texinfo" ( %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo if errorlevel 1 exit /b 1 echo. echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. goto end ) if "%1" == "gettext" ( %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale if errorlevel 1 exit /b 1 echo. echo.Build finished. The message catalogs are in %BUILDDIR%/locale. goto end ) if "%1" == "changes" ( %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes if errorlevel 1 exit /b 1 echo. echo.The overview file is in %BUILDDIR%/changes. goto end ) if "%1" == "linkcheck" ( %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck if errorlevel 1 exit /b 1 echo. echo.Link check complete; look for any errors in the above output ^ or in %BUILDDIR%/linkcheck/output.txt. goto end ) if "%1" == "doctest" ( %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest if errorlevel 1 exit /b 1 echo. echo.Testing of doctests in the sources finished, look at the ^ results in %BUILDDIR%/doctest/output.txt. goto end ) if "%1" == "xml" ( %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml if errorlevel 1 exit /b 1 echo. echo.Build finished. The XML files are in %BUILDDIR%/xml. goto end ) if "%1" == "pseudoxml" ( %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml if errorlevel 1 exit /b 1 echo. echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. goto end ) :end python-nxs/doc/source/000077500000000000000000000000001306222103500152445ustar00rootroot00000000000000python-nxs/doc/source/conf.py000066400000000000000000000203201306222103500165400ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # Python-nxs documentation build configuration file, created by # sphinx-quickstart on Wed Dec 16 15:17:43 2015. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os import sphinx_rtd_theme # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath('../..')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.coverage', 'sphinx.ext.mathjax', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'Python-nxs' copyright = u'2015, Eugen Wintersberger' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '4.4.1' # The full version, including alpha/beta/rc tags. release = '4.4.1' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. #html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'Python-nxsdoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ ('index', 'Python-nxs.tex', u'Python-nxs Documentation', u'Eugen Wintersberger', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'python-nxs', u'Python-nxs Documentation', [u'Eugen Wintersberger'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'Python-nxs', u'Python-nxs Documentation', u'Eugen Wintersberger', 'Python-nxs', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False python-nxs/doc/source/index.rst000066400000000000000000000013221306222103500171030ustar00rootroot00000000000000.. Python-nxs documentation master file, created by sphinx-quickstart on Wed Dec 16 15:17:43 2015. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to Python-nxs's documentation! ====================================== The :py:mod:`python-nxs` package provides two bindings to the NeXus C-API * :py:mod:`nxs.napi` which is only a very simple wrapper create by means of the :py:mod:`ctypes` module * :py:mod:`nxs.tree` which provides a higher level interface to the C-API Contents: .. toctree:: :maxdepth: 1 napi_api_documentation Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` python-nxs/doc/source/napi_api_documentation.rst000066400000000000000000000002011306222103500225000ustar00rootroot00000000000000====================== NAPI API documentation ====================== .. automodule:: nxs.napi :members: :undoc-members: python-nxs/nxs/000077500000000000000000000000001306222103500140075ustar00rootroot00000000000000python-nxs/nxs/__init__.py000066400000000000000000000013531306222103500161220ustar00rootroot00000000000000# This program is public domain """@package nxs Python NeXus interface. NeXus_ is a common data format for neutron, Xray and muon science. The files contain multidimensional data elements grouped into a hierarchical structure. The data sets are self-describing, with a description of the instrument configuration including the units used as well as the data measured. The NeXus file interface requires compiled libraries to read the underlying HDF_ files. Binary packages are available for some platforms from the NeXus site. Details of where the nxs package searches for the libraries are recorded in `nxs.napi`. """ ## @mainpage NeXus Python Documentation ## See nxs.napi on the Packages tab from nxs.napi import * from nxs.tree import * python-nxs/nxs/napi.py000077500000000000000000001565111306222103500153240ustar00rootroot00000000000000# This program is public domain # Author: Paul Kienzle """ Wrapper for the NeXus shared library. Use this interface when converting code from other languages which do not support the natural view of the hierarchy. Library Location ================ :py:mod:`nxs.napi` needs to know the location of the `libNeXus` C-library in order to load it via :py:mod:`ctypes`. It looks in the following places in order: +-------------------------------------+-----------------+ | Environment | Platforms | +=====================================+=================+ | ``os.environ['NEXUSLIB']`` | All | +-------------------------------------+-----------------+ | directory containing nxs.py | All | +-------------------------------------+-----------------+ | ``os.environ['NEXUSDIR']\\bin`` | Windows | +-------------------------------------+-----------------+ | ``os.environ['LD_LIBRARY_PATH']`` | Unix | +-------------------------------------+-----------------+ | ``os.environ['DYLD_LIBRARY_PATH']`` | Darwin | +-------------------------------------+-----------------+ | ``LIBDIR`` | Unix and Darwin | +-------------------------------------+-----------------+ * On Windows it looks for one of ``libNeXus.dll`` or ``libNeXus-0.dll``. * On OS X it looks for ``libNeXus.0.dylib`` * On Unix it looks for ``libNeXus.so.1`` * ``NEXUSDIR`` defaults to ``C:\\Program Files\\NeXus Data Format``. * ``LIBDIR`` defaults to ``/usr/local/lib``, but is replaced by the value of ``--libdir`` during configure. The import will raise an :py:exc:`OSError` exception if the library wasn't found or couldn't be loaded. Note that on Windows in particular this may be because the supporting HDF5 dlls were not available in the usual places. If you are extracting the nexus library from a bundle at runtime, set `os.environ['NEXUSLIB']` to the path where it is extracted before the first import of nxs. Example ======= .. code-block:: python import nxs file = nxs.open('filename.nxs','rw') file.opengroup('entry1') file.opendata('definition') print file.getdata() file.close() See @see nxstest.py for a more complete example. Interface ========= When converting code to python from other languages you do not necessarily want to redo the file handling code. The nxs provides an interface which more closely follows the NeXus application programming interface (NAPI_). This wrapper differs from NAPI in several respects: * Data values are loaded/stored directly from numpy arrays. * Return codes are turned into exceptions. * The file handle is stored in a file object * Constants are handled somewhat differently (see below) * Type checking on data/parameter storage * Adds iterators file.entries() and file.attrs() * Adds link() function to return the name of the linked to group, if any * NXmalloc/NXfree are not needed. File open modes can be constants or strings: +--------------------------+--------------------------+ | Flag | Character representation | +==========================+==========================+ | `nxs.napi.ACC_READ` | 'r' | +--------------------------+--------------------------+ | `nxs.napi.ACC_RDWR` | 'rw' | +--------------------------+--------------------------+ | `nxs.napi.ACC_CREATE` | 'w' | +--------------------------+--------------------------+ | `nxs.napi.ACC_CREATE4` | 'w4' | +--------------------------+--------------------------+ | `nxs.napi.ACC_CREATE5` | 'w5' | +--------------------------+--------------------------+ | `nxs.napi.ACC_CREATEXML` | 'wx' | +--------------------------+--------------------------+ Dimension constants: +---------------+---------------------------------------+ | Constant | Description | +===============+=======================================+ | nxs.UNLIMITED | for the extensible data dimension | +---------------+---------------------------------------+ | nxs.MAXRANK | for the number of possible dimensions | +---------------+---------------------------------------+ Data types are strings corresponding to the numpy data types: +---------------+-----------------------------+ | Type string | Description | +===============+=============================+ | ``'float32'`` | 32Bit floating point number | +---------------+-----------------------------+ | ``'float64'`` | 64Bit floating point number | +---------------+-----------------------------+ | ``'int8'`` | 8Bit singed integer | +---------------+-----------------------------+ | ``'int16'`` | 16Bit signed integer | +---------------+-----------------------------+ | ``'int32'`` | 32Bit signed integer | +---------------+-----------------------------+ | ``'int64'`` | 64Bit signed integer | +---------------+-----------------------------+ | ``'uint8'`` | 8Bit unsigned integer | +---------------+-----------------------------+ | ``'uint16'`` | 16Bit unsigned integer | +---------------+-----------------------------+ | ``'uint32'`` | 32Bit unsigned integer | +---------------+-----------------------------+ | ``'uint64'`` | 64Bit unsigned integer | +---------------+-----------------------------+ | ``'char'`` | used for string data | +---------------+-----------------------------+ To retrieve the type of a :py:mod:`numpy` array one can use its :py:attr:`dtype` attribute. Dimensions are lists of integers or numpy arrays. You can use the numpy A.shape attribute for the dimensions of array A. Compression codes are:: 'none' 'lzw' 'rle' 'huffman' As of this writing NeXus only supports 'none' and 'lzw'. Miscellaneous constants: +----------------+--------------------------------------------------------------+ | Constant | Description | +================+==============================================================+ | :py:const:`nxs | names must be shorter than this | | .napi.MAXNAMEL | | | EN` | | +----------------+--------------------------------------------------------------+ | nxs.napi.MAXPA | total path length must be shorter than this | | THLEN | | +----------------+--------------------------------------------------------------+ | nxs.napi.H4SKI | class names that may appear in HDF4 files but can be ignored | | P | | +----------------+--------------------------------------------------------------+ Caveats ======= @todo NOSTRIP constant is probably not handled properly, @todo Embedded nulls in strings is not supported @warning We have a memory leak. Calling open/close costs about 90k a pair. This is an eigenbug: - if I test ctypes on a simple library it does not leak - if I use the leak_test1 code in the nexus distribution it doesn't leak - if I remove the open/close call in the wrapper it doesn't leak. """ from __future__ import print_function ## @example nxstest.py # Test program for NeXus python interface __all__ = ['UNLIMITED', 'MAXRANK', 'MAXNAMELEN','MAXPATHLEN','H4SKIP', 'NeXus','NeXusError','open'] import sys, os, numpy, ctypes import six # Defined ctypes from ctypes import c_void_p, c_int, c_int64, c_long, c_char, c_char_p from ctypes import byref as _ref c_void_pp = ctypes.POINTER(c_void_p) c_int_p = ctypes.POINTER(c_int) c_int64_p = ctypes.POINTER(c_int64) class _NXlink(ctypes.Structure): _fields_ = [("iTag", c_long), ("iRef", c_long), ("targetPath", c_char*1024), ("linktype", c_int)] _pack_ = False c_NXlink_p = ctypes.POINTER(_NXlink) # Open modes: ACC_READ,ACC_RDWR,ACC_CREATE=1,2,3 ACC_CREATE4,ACC_CREATE5,ACC_CREATEXML=4,5,6 _nxopen_mode=dict(r=1,rw=2,w=3,w4=4,w5=5,wx=6) NOSTRIP=128 # Status codes OK,ERROR,EOD=1,0,-1 # Other constants UNLIMITED=-1 MAXRANK=32 MAXNAMELEN=64 MAXPATHLEN=1024 # inferred from code # bogus groups; these groups are ignored in HDFView from NCSA. H4SKIP = ['CDF0.0','_HDF_CHK_TBL_','Attr0.0', 'RIG0.0','RI0.0', 'RIATTR0.0N','RIATTR0.0C'] # HDF data types from numpy types _nxtype_code=dict( char=4, float32=5,float64=6, int8=20,uint8=21, int16=22,uint16=23, int32=24,uint32=25, int64=26,uint64=27, ) # Python types from HDF data types # Other than 'char' for the string type, the python types correspond to # the numpy data types, and can be used directly to create numpy arrays. # Note: put this in a lambda to hide v,k from the local namespace _pytype_code=(lambda : dict([(v,k) for (k,v) in six.iteritems(_nxtype_code)]))() # Compression to use when creating data blocks _compression_code=dict( none=100, lzw=200, rle=300, huffman=400) def _is_string_like(obj): """ Return True if object acts like a string. """ # From matplotlib cbook.py John D. Hunter # Python 2.2 style licence. See license.py in matplotlib for details. if hasattr(obj, 'shape'): return False try: obj + '' except (TypeError, ValueError): return False return True def _is_list_like(obj): """ Return True if object acts like a list """ try: obj + [] except TypeError: return False return True def _libnexus(): """ Load the NeXus library. """ # this will get changed as part of the install process # it should correspond to --libdir specified to ./configure # NEXUSLIB takes precedence if 'NEXUSLIB' in os.environ: file = os.environ['NEXUSLIB'] if not os.path.isfile(file): raise OSError("File %s from environment variable NEXUSLIB does " "exist" % file) files = [file] else: files = [] # Default names and locations to look for the library are system dependent filedir = os.path.dirname(__file__) if sys.platform in ('win32','cygwin'): # NEXUSDIR is set by the Windows installer for NeXus if 'NEXUSDIR' in os.environ: winnxdir = os.environ['NEXUSDIR'] else: winnxdir = 'C:/Program Files/NeXus Data Format' files += [filedir+"/libNeXus-0.dll", winnxdir + '/bin/libNeXus-0.dll', filedir+"/libNeXus.dll"] else: if sys.platform in ('darwin'): lib = 'libNeXus.0.dylib' ldenv = 'DYLD_LIBRARY_PATH' else: lib = 'libNeXus.so.1' ldenv = 'LD_LIBRARY_PATH' # Search the load library path as well as the standard locations ldpath = [p for p in os.environ.get(ldenv,'').split(':') if p != ''] stdpath = [ '/usr/local/lib', '/usr/local/lib64', '/usr/lib', '/usr/lib64'] files += [os.path.join(p,lib) for p in [filedir]+ldpath+stdpath] # Given a list of files, try loading the first one that is available. for file in files: if not os.path.isfile(file): continue try: return ctypes.cdll[file] except: raise OSError("NeXus library %s could not be loaded: %s" % (file,sys.exc_info())) raise OSError("Set NEXUSLIB or move NeXus to one of: %s" % ", ".join(files)) def _init(): lib = _libnexus() lib.NXMDisableErrorReporting() return lib # Define the interface to the dll nxlib = _init() def open(filename, mode='r'): """ Returns a NeXus file object. """ return NeXus(filename, mode) class NeXusError(Exception): """NeXus Error""" pass class NeXus(object): # ==== File ==== nxlib.nxiopen_.restype = c_int nxlib.nxiopen_.argtypes = [c_char_p, c_int, c_void_pp] def __init__(self, filename, mode='r'): """ Open the NeXus file returning a handle. mode can be one of the following: nxs.ACC_READ 'r' open a file read-only nxs.ACC_RDWR 'rw' open a file read/write nxs.ACC_CREATE 'w' open a file write nxs.ACC_CREATE4 'w4' open a Nexus file with HDF4 nxs.ACC_CREATE5 'w5' open a Nexus file with HDF5 nxs.ACC_CREATEXML 'wx' open a Nexus file with XML Raises ValueError if the open mode is invalid. Raises NeXusError if the file could not be opened, with the filename as part of the error message. Corresponds to NXopen(filename,mode,&handle) """ self.isopen = False # Convert open mode from string to integer and check it is valid if mode in _nxopen_mode: mode = _nxopen_mode[mode] if mode not in _nxopen_mode.values(): raise ValueError("Invalid open mode %s" % str(mode)) self.filename, self.mode = filename, mode self.handle = c_void_p(None) self._path = [] self._indata = False status = nxlib.nxiopen_(filename,mode,_ref(self.handle)) if status == ERROR: if mode in [ACC_READ, ACC_RDWR]: op = 'open' else: op = 'create' raise NeXusError("Could not %s %s" % (op,filename)) self.isopen = True def _getpath(self): mypath = [level[0] for level in self._path] return '/'+'/'.join(mypath) path = property(_getpath,doc="Unix-style path to node") def _getlongpath(self): mypath = [':'.join(level) for level in self._path] return '/' + '/'.join(mypath) longpath = property(_getlongpath, doc="Unix-style path including " \ + "nxclass to the node") def __del__(self): """ Be sure to close the file before deleting the last reference. """ if self.isopen: self.close() def __str__(self): """ Return a string representation of the NeXus file handle. """ return "NeXus('%s')"%self.filename def open(self): """ Opens the NeXus file handle if it is not already open. Raises NeXusError if the file could not be opened. Corresponds to NXopen(filename,mode,&handle) """ if self.isopen: return if self.mode==ACC_READ: mode = ACC_READ else: mode = ACC_RDWR status = nxlib.nxiopen_(self.filename,mode,_ref(self.handle)) if status == ERROR: raise NeXusError("Could not open %s" % (self.filename)) self._path = [] self._indata = False nxlib.nxiclose_.restype = c_int nxlib.nxiclose_.argtypes = [c_void_pp] def close(self): """ Close the NeXus file associated with handle. Raises NeXusError if file could not be closed. Corresponds to NXclose(&handle) """ if self.isopen: self.isopen = False status = nxlib.nxiclose_(_ref(self.handle)) if status == ERROR: raise NeXusError("Could not close NeXus file %s" % (self.filename)) self._path = [] self._indata = False nxlib.nxiflush_.restype = c_int nxlib.nxiflush_.argtypes = [c_void_pp] def flush(self): """ Flush all data to the NeXus file. Raises NeXusError if this fails. Corresponds to NXflush(&handle) """ status = nxlib.nxiflush_(_ref(self.handle)) if status == ERROR: raise NeXusError("Could not flush NeXus file %s" % (self.filename)) nxlib.nxisetnumberformat_.restype = c_int nxlib.nxisetnumberformat_.argtypes = [c_void_p, c_int, c_char_p] def setnumberformat(self,type,format): """ Set the output format for the numbers of the given type (only applies to XML). Raises ValueError if the number format is incorrect. Corresponds to NXsetnumberformat(&handle,type,format) """ type = _nxtype_code[type] status = nxlib.nxisetnumberformat_(self.handle,type,format) if status == ERROR: raise ValueError("Could not set %s to %s in %s" % (type, format, self.filename)) # ==== Group ==== nxlib.nximakegroup_.restype = c_int nxlib.nximakegroup_.argtypes = [c_void_p, c_char_p, c_char_p] def makegroup(self, name, nxclass): """ Create the group nxclass:name. Raises NeXusError if the group could not be created. Corresponds to NXmakegroup(handle, name, nxclass) """ # print("makegroup", self._loc(), name, nxclass) status = nxlib.nximakegroup_(self.handle, name, nxclass) if status == ERROR: raise NeXusError("Could not create %s:%s in %s" % (nxclass, name, self._loc())) nxlib.nxiopenpath_.restype = c_int nxlib.nxiopenpath_.argtypes = [c_void_p, c_char_p] def openpath(self, path): """ Open a particular group '/path/to/group'. Paths can be absolute or relative to the currently open group. If openpath fails, then currently open path may not be different from the starting path. For better performation the types can be specified as well using '/path:type1/to:type2/group:type3' which will prevent searching the file for the types associated with the supplied names. Raises ValueError. Corresponds to NXopenpath(handle, path) """ self._openpath(path, opendata=True) def _openpath(self, path, opendata=True): """helper function: open relative path and maybe data""" # Determine target node as sequence of group names if path == '/': target = [] else: if path.endswith("/"): path = path[:-1] if path.startswith('/'): target = path[1:].split('/') else: target = self._path + path.split('/') # Remove relative path indicators from target L = [] for t in target: if t == '.': # Skip current node pass elif t == '..': if L == []: raise ValueError("too many '..' in path") L.pop() else: L.append(t) target = L # split out nxclass from each level if available L = [] for t in target: try: item = t.split(":") if len(item) == 1: L.append((item[0], None)) else: L.append(tuple(item)) except AttributeError: L.append(t) target = L # print("current path", self._path) # print("%s" % path, target) # Find which groups need to be closed and opened up = [] down = [] for (i, (name, nxclass)) in enumerate(target): if i == len(self._path): # print("target longer than current") up = [] down = target[i:] break elif self._path[i] != name: # print("target and current differ at", name) up = self._path[i:] down = target[i:] break else: # print("target shorter than current") up = self._path[len(target):] down = [] # add more information to the down path for i in xrange(len(down)): try: (name, nxclass) = down[i] except ValueError: down[i] = (down[i], None) # print("close,open", up, down) # Close groups on the way up if self._indata and up != []: self.closedata() up.pop() for target in up: self.closegroup() # Open groups on the way down for target in down: (name, nxclass) = target if nxclass is None: nxclass = self.__getnxclass(name) if nxclass != "SDS": self.opengroup(name, nxclass) elif opendata: self.opendata(name) else: raise ValueError("node %s not in %s" % (name, self.path)) nxlib.nxiopengrouppath_.restype = c_int nxlib.nxiopengrouppath_.argtypes = [c_void_p, c_char_p] def opengrouppath(self, path): """ Open a particular group '/path/to/group', or the dataset containing the group if the path refers to a dataset. Paths can be relative to the currently open group. Raises ValueError. Corresponds to NXopengrouppath(handle, path) """ self._openpath(path,opendata=False) nxlib.nxiopengroup_.restype = c_int nxlib.nxiopengroup_.argtypes = [c_void_p, c_char_p, c_char_p] def opengroup(self, name, nxclass=None): """ Open the group nxclass:name. If the nxclass is not specified this will search for it. Raises NeXusError if the group could not be opened. Corresponds to NXopengroup(handle, name, nxclass) """ # print("opengroup", self._loc(), name, nxclass) if nxclass is None: nxclass = self.__getnxclass(name) status = nxlib.nxiopengroup_(self.handle, name, nxclass) if status == ERROR: raise ValueError("Could not open %s:%s in %s" % (nxclass, name, self._loc())) self._path.append((name,nxclass)) nxlib.nxiclosegroup_.restype = c_int nxlib.nxiclosegroup_.argtypes = [c_void_p] def closegroup(self): """ Close the currently open group. Raises NeXusError if the group could not be closed. Corresponds to NXclosegroup(handle) """ # print("closegroup") if self._indata: raise NeXusError("Close data before group at %s" % (self._loc())) status = nxlib.nxiclosegroup_(self.handle) if status == ERROR: raise NeXusError("Could not close group at %s" % (self._loc())) self._path.pop() nxlib.nxigetgroupinfo_.restype = c_int nxlib.nxigetgroupinfo_.argtypes = [c_void_p, c_int_p, c_char_p, c_char_p] def getgroupinfo(self): """ Query the currently open group returning the tuple numentries, name, nxclass. Raises ValueError if the group could not be opened. Corresponds to NXgetgroupinfo(handle) Note: corrects error in HDF5 where getgroupinfo returns the entire path rather than the group name. Use the path attribute to get a sensible value of path. """ # Space for the returned strings path = ctypes.create_string_buffer(MAXPATHLEN) nxclass = ctypes.create_string_buffer(MAXNAMELEN) n = c_int(0) status = nxlib.nxigetgroupinfo_(self.handle,_ref(n),path,nxclass) if status == ERROR: raise ValueError("Could not get group info: %s" % (self._loc())) # print("getgroupinfo", self._loc(), nxclass.value, name.value, n.value) name = path.value.split('/')[-1] # Protect against HDF5 returning path return n.value,name,nxclass.value nxlib.nxiinitgroupdir_.restype = c_int nxlib.nxiinitgroupdir_.argtypes = [c_void_p] def initgroupdir(self): """ Reset getnextentry to return the first entry in the group. Raises NeXusError if this fails. Corresponds to NXinitgroupdir(handle) """ status = nxlib.nxiinitgroupdir_(self.handle) if status == ERROR: raise NeXusError("Could not reset group scan: %s" % (self._loc())) nxlib.nxigetnextentry_.restype = c_int nxlib.nxigetnextentry_.argtypes = [c_void_p, c_char_p, c_char_p, c_int_p] def getnextentry(self): """ Return the next entry in the group as name,nxclass tuple. If end of data is reached this returns the tuple (None, None) Raises NeXusError if this fails. Corresponds to NXgetnextentry(handle,name,nxclass,&storage). This function doesn't return the storage class for data entries since getinfo returns shape and storage, both of which are required to read the data. Note that HDF4 files can have entries in the file with classes that don't need to be processed. If the file follows the standard NeXus DTDs then skip any entry for which nxclass.startswith('NX') is False. For non-conforming files, skip those entries with nxclass in nxs.H4SKIP. """ name = ctypes.create_string_buffer(MAXNAMELEN) nxclass = ctypes.create_string_buffer(MAXNAMELEN) storage = c_int(0) status = nxlib.nxigetnextentry_(self.handle,name,nxclass,_ref(storage)) if status == EOD: return (None, None) if status == ERROR: raise NeXusError("Could not get next entry: %s" % (self._loc())) ## Note: ignoring storage --- it is useless without dimensions # if nxclass == 'SDS': # dtype = _pytype_code(storage.value) # print("nextentry", nxclass.value, name.value, storage.value) return name.value,nxclass.value def getentries(self): """ Return a dictionary of the groups[name]=type below the existing open one. Raises NeXusError if this fails. """ self.initgroupdir() result = {} (name, nxclass) = self.getnextentry() if (name, nxclass) != (None, None): result[name] = nxclass while (name, nxclass) != (None, None): result[name] = nxclass (name, nxclass) = self.getnextentry() return result def __getnxclass(self, target): """ Return the nxclass of the supplied name. """ self.initgroupdir() while True: (nxname, nxclass) = self.getnextentry() if nxname == target: return nxclass if nxname is None: break raise NeXusError("Failed to find entry with name \"%s\" at %s" % (target, self.path)) def entries(self): """ Iterator of entries. for name,nxclass in nxs.entries(): process(name,nxclass) This automatically opens the corresponding group/data for you, and closes it when you are done. Do not rely on any paths remaining open between entries as we restore the current path each time. This does not correspond to an existing NeXus API function, but instead combines the work of initgroupdir/getnextentry and open/close on data and group. Entries in nxs.H4SKIP are ignored. """ # To preserve the semantics we must read in the whole list # first, then process the entries one by one. Keep track # of the path so we can restore it between entries. path = self.path # Read list of entries self.initgroupdir() n,_,_ = self.getgroupinfo() L = [] for dummy in range(n): name,nxclass = self.getnextentry() if nxclass not in H4SKIP: L.append((name,nxclass)) for name,nxclass in L: self.openpath(path) # Reset the file cursor if nxclass == "SDS": self.opendata(name) else: self.opengroup(name,nxclass) yield name,nxclass # ==== Data ==== nxlib.nxigetrawinfo64_.restype = c_int nxlib.nxigetrawinfo64_.argtypes = [c_void_p, c_int_p, c_void_p, c_int_p] def getrawinfo(self): """ Returns the tuple dimensions,type for the currently open dataset. Dimensions is an integer array whose length corresponds to the rank of the dataset and whose elements are the size of the individual dimensions. Storage type is returned as a string, with 'char' for a stored string, '[u]int[8|16|32]' for various integer values or 'float[32|64]' for floating point values. No support for complex values. Unlike getinfo(), the size of the string storage area is returned rather than the length of the stored string. Raises NeXusError if this fails. Corresponds to NXgetrawinfo(handle, &rank, dims, &storage), but with storage converted from HDF values to numpy compatible strings, and rank implicit in the length of the returned dimensions. """ rank = c_int(0) shape = numpy.zeros(MAXRANK, 'int64') storage = c_int(0) status = nxlib.nxigetrawinfo64_(self.handle, _ref(rank), shape.ctypes.data, _ref(storage)) if status == ERROR: raise NeXusError("Could not get data info: %s" % (self._loc())) shape = shape[:rank.value]+0 dtype = _pytype_code[storage.value] # print("getrawinfo", self._loc(), "->", shape, dtype) return shape,dtype nxlib.nxigetinfo64_.restype = c_int nxlib.nxigetinfo64_.argtypes = [c_void_p, c_int_p, c_void_p, c_int_p] def getinfo(self): """ Returns the tuple dimensions,type for the currently open dataset. Dimensions is an integer array whose length corresponds to the rank of the dataset and whose elements are the size of the individual dimensions. Storage type is returned as a string, with 'char' for a stored string, '[u]int[8|16|32]' for various integer values or 'float[32|64]' for floating point values. No support for complex values. Unlike getrawinfo(), the length of the stored string is returned rather than the size of the string storage area. Raises NeXusError if this fails. Note that this is the recommended way to establish if you have a dataset open. Corresponds to NXgetinfo(handle, &rank, dims, &storage), but with storage converted from HDF values to numpy compatible strings, and rank implicit in the length of the returned dimensions. """ rank = c_int(0) shape = numpy.zeros(MAXRANK, 'int64') storage = c_int(0) status = nxlib.nxigetinfo64_(self.handle, _ref(rank), shape.ctypes.data, _ref(storage)) if status == ERROR: raise NeXusError("Could not get data info: %s" % (self._loc())) shape = shape[:rank.value]+0 dtype = _pytype_code[storage.value] # print("getinfo", self._loc(), "->", shape, dtype) return shape,dtype nxlib.nxiopendata_.restype = c_int nxlib.nxiopendata_.argtypes = [c_void_p, c_char_p] def opendata(self, name): """ Open the named data set within the current group. Raises ValueError if could not open the dataset. Corresponds to NXopendata(handle, name) """ # print("opendata", self._loc(), name) if self._indata: status = ERROR else: status = nxlib.nxiopendata_(self.handle, name) if status == ERROR: raise ValueError("Could not open data %s: %s" % (name, self._loc())) self._path.append((name,"SDS")) self._indata = True nxlib.nxiclosedata_.restype = c_int nxlib.nxiclosedata_.argtypes = [c_void_p] def closedata(self): """ Close the currently open data set. Raises NeXusError if this fails (e.g., because no dataset is open). Corresponds to NXclosedata(handle) """ # print("closedata") status = nxlib.nxiclosedata_(self.handle) if status == ERROR: raise NeXusError("Could not close data at %s" % (self._loc())) self._path.pop() self._indata = False nxlib.nximakedata64_.restype = c_int nxlib.nximakedata64_.argtypes = [c_void_p, c_char_p, c_int, c_int, c_int64_p] def makedata(self, name, dtype=None, shape=None): """ Create a data element of the given type and shape. See getinfo for details on types. This does not open the data for writing. Set the first dimension to nxs.UNLIMITED, for extensible data sets, and use putslab to write individual slabs. Raises ValueError if it fails. Corresponds to NXmakedata(handle,name,type,rank,dims) """ # TODO: With keywords for compression and chunks, this can act as # TODO: compmakedata. # TODO: With keywords for value and attr, this can be used for # TODO: makedata, opendata, putdata, putattr, putattr, ..., closedata # print("makedata", self._loc(), name, shape, dtype) storage = _nxtype_code[str(dtype)] shape = numpy.asarray(shape,'int64') status = nxlib.nximakedata64_(self.handle,name,storage,len(shape), shape.ctypes.data_as(c_int64_p)) if status == ERROR: raise ValueError("Could not create data %s: %s" % (name,self._loc())) nxlib.nxicompmakedata64_.restype = c_int nxlib.nxicompmakedata64_.argtypes = [c_void_p, c_char_p, c_int, c_int, c_int64_p, c_int, c_int64_p] def compmakedata(self, name, dtype=None, shape=None, mode='lzw', chunks=None): """ Create a data element of the given dimensions and type. See getinfo for details on types. Compression mode is one of 'none', 'lzw', 'rle' or 'huffman'. chunks gives the alignment of the compressed chunks in the data file. There should be one chunk size for each dimension in the data. Defaults to mode='lzw' with chunk size set to the length of the fastest varying dimension. Raises ValueError if it fails. Corresponds to NXmakedata(handle,name,type,rank,dims). """ storage = _nxtype_code[str(dtype)] # Make sure shape/chunk_shape are integers; hope that 32/64 bit issues # with the c int type sort themselves out. dims = numpy.asarray(shape,'int64') if chunks is None: chunks = numpy.ones(dims.shape,'int64') chunks[-1] = shape[-1] else: chunks = numpy.array(chunks,'int64') status = nxlib.nxicompmakedata64_(self.handle,name,storage,len(dims), dims.ctypes.data_as(c_int64_p), _compression_code[mode], chunks.ctypes.data_as(c_int64_p)) if status == ERROR: raise ValueError("Could not create compressed data %s: %s" % (name, self._loc())) nxlib.nxigetdata_.restype = c_int nxlib.nxigetdata_.argtypes = [c_void_p, c_void_p] def getdata(self): """ Return the data. If data is a string (1-D char array), a python string is returned. If data is a scalar (1-D numeric array of length 1), a python scalar is returned. If data is a string array, a numpy array of type 'S#' where # is the maximum string length is returned. If data is a numeric array, a numpy array is returned. Raises ValueError if this fails. Corresponds to NXgetdata(handle, data) """ # TODO: consider accepting preallocated data so we don't thrash memory shape,dtype = self.getinfo() dummy_data,pdata,dummy_size,datafn = self._poutput(dtype,shape) status = nxlib.nxigetdata_(self.handle,pdata) if status == ERROR: raise ValueError("Could not read data: %s" % (self._loc())) # print("getdata", self._loc(), shape, dtype) return datafn() nxlib.nxigetslab64_.restype = c_int nxlib.nxigetslab64_.argtypes = [c_void_p, c_void_p, c_int64_p, c_int64_p] def getslab(self, slab_offset, slab_shape): """ Get a slab from the data array. Offsets are 0-origin. Shape can be inferred from the data. Offset and shape must each have one entry per dimension. Raises ValueError if this fails. Corresponds to NXgetslab(handle,data,offset,shape) """ # TODO: consider accepting preallocated data so we don't thrash memory dummy_shape,dtype = self.getrawinfo() dummy_data,pdata,dummy_size,datafn = self._poutput(dtype,slab_shape) slab_offset = numpy.asarray(slab_offset,'int64') slab_shape = numpy.asarray(slab_shape,'int64') status = nxlib.nxigetslab64_(self.handle,pdata, slab_offset.ctypes.data_as(c_int64_p), slab_shape.ctypes.data_as(c_int64_p)) # print("slab", offset, size, data) if status == ERROR: raise ValueError("Could not read slab: %s" % (self._loc())) return datafn() nxlib.nxiputdata_.restype = c_int nxlib.nxiputdata_.argtypes = [c_void_p, c_void_p] def putdata(self, data): """ Write data into the currently open data block. Raises ValueError if this fails. Corresponds to NXputdata(handle, data) """ shape,dtype = self.getrawinfo() # print("putdata", self._loc(), shape, dtype) data,pdata = self._pinput(data,dtype,shape) status = nxlib.nxiputdata_(self.handle,pdata) if status == ERROR: raise ValueError("Could not write data: %s" % (self._loc())) nxlib.nxiputslab64_.restype = c_int nxlib.nxiputslab64_.argtypes = [c_void_p, c_void_p, c_int64_p, c_int64_p] def putslab(self, data, slab_offset, slab_shape): """ Put a slab into the data array. Offsets are 0-origin. Shape can be inferred from the data. Offset and shape must each have one entry per dimension. Raises ValueError if this fails. Corresponds to NXputslab(handle,data,offset,shape) """ dummy_shape,dtype = self.getrawinfo() data,pdata = self._pinput(data,dtype,slab_shape) slab_offset = numpy.asarray(slab_offset,'int64') slab_shape = numpy.asarray(slab_shape,'int64') # print("slab", offset, size, data) status = nxlib.nxiputslab64_(self.handle,pdata, slab_offset.ctypes.data_as(c_int64_p), slab_shape.ctypes.data_as(c_int64_p)) if status == ERROR: raise ValueError("Could not write slab: %s" % (self._loc())) # ==== Attributes ==== nxlib.nxiinitattrdir_.restype = c_int nxlib.nxiinitattrdir_.argtypes = [c_void_p] def initattrdir(self): """ Reset the getnextattr list to the first attribute. Raises NeXusError if this fails. Corresponds to NXinitattrdir(handle) """ status = nxlib.nxiinitattrdir_(self.handle) if status == ERROR: raise NeXusError("Could not reset attribute list: %s" % (self._loc())) nxlib.nxigetattrinfo_.restype = c_int nxlib.nxigetattrinfo_.argtypes = [c_void_p, c_int_p] def getattrinfo(self): """ Returns the number of attributes for the currently open group/data object. Do not call getnextattr() more than this number of times. Raises NeXusError if this fails. Corresponds to NXgetattrinfo(handl, &n) """ n = c_int(0) status = nxlib.nxigetattrinfo_(self.handle,_ref(n)) if status == ERROR: raise NeXusError("Could not get attr info: %s" % (self._loc())) # print("num attrs", n.value) return n.value nxlib.nxigetnextattr_.restype = c_int nxlib.nxigetnextattr_.argtypes = [c_void_p, c_char_p, c_int_p, c_int_p] def getnextattr(self): """ Returns the name, length, and data type for the next attribute. Call getattrinfo to determine the number of attributes before calling getnextattr. Data type is returned as a string. See getinfo for details. Length is the number of elements in the attribute. Raises NeXusError if NeXus returns ERROR or EOD. Corresponds to NXgetnextattr(handle,name,&length,&storage) but with storage converted from HDF values to numpy compatible strings. Note: NeXus API documentation seems to say that length is the number of bytes required to store the entire attribute. """ name = ctypes.create_string_buffer(MAXNAMELEN) length = c_int(0) storage = c_int(0) status = nxlib.nxigetnextattr_(self.handle,name,_ref(length),_ref(storage)) if status == EOD: return (None, None, None) if status == ERROR or status == EOD: raise NeXusError("Could not get next attr: %s" % (self._loc())) dtype = _pytype_code[storage.value] # print("getnextattr", name.value, length.value, dtype) return name.value, length.value, dtype # TODO: Resolve discrepency between NeXus API documentation and # TODO: apparent behaviour for getattr/putattr length. nxlib.nxigetattr_.restype = c_int nxlib.nxigetattr_.argtypes = [c_void_p, c_char_p, c_void_p, c_int_p, c_int_p] def getattr(self, name, length, dtype): """ Returns the value of the named attribute. Requires length and data type from getnextattr to allocate the appropriate amount of space for the attribute. Corresponds to NXgetattr(handle,name,data,&length,&storage) """ if dtype is 'char': length += 1 # HDF4 needs zero-terminator dummy_data,pdata,size,datafn = self._poutput(str(dtype),[length]) storage = c_int(_nxtype_code[str(dtype)]) # print("getattr", self._loc(), name, length, size, dtype) size = c_int(size) status = nxlib.nxigetattr_(self.handle,name,pdata,_ref(size),_ref(storage)) if status == ERROR: raise ValueError("Could not read attr %s: %s" % (name, self._loc())) # print("getattr", self._loc(), name, datafn()) return datafn() nxlib.nxiputattr_.restype = c_int nxlib.nxiputattr_.argtypes = [c_void_p, c_char_p, c_void_p, c_int, c_int] def putattr(self, name, value, dtype = None): """ Saves the named attribute. The attribute value is a string or a scalar. Raises TypeError if the value type is incorrect. Raises NeXusError if the attribute could not be saved. Corresponds to NXputattr(handle,name,data,length,storage) Note length is the number of elements to write rather than the number of bytes to write. """ # Establish attribute type if dtype == None: # Type is inferred from value if hasattr(value,'dtype'): dtype = str(value.dtype) elif _is_string_like(value): dtype = 'char' else: value = numpy.array(value) dtype = str(value.dtype) else: # Set value to type dtype = str(dtype) if dtype == 'char' and not _is_string_like(value): raise TypeError("Expected string for 'char' attribute value") if dtype != 'char': value = numpy.array(value,dtype=dtype) # Determine shape if dtype == 'char': length = len(value) data = value elif numpy.prod(value.shape) != 1: # NAPI silently ignores attribute arrays raise TypeError("Attribute value must be scalar or string") else: length = 1 data = value.ctypes.data # Perform the call storage = c_int(_nxtype_code[dtype]) status = nxlib.nxiputattr_(self.handle,name,data,length,storage) if status == ERROR: raise NeXusError("Could not write attr %s: %s" % (name, self._loc())) def getattrs(self): """ Returns a dicitonary of the attributes on the current node. This is a second form of attrs(self). """ result = {} for (name, value) in self.attrs(): result[name] = value return result def attrs(self): """ Iterate over attributes. for name,value in file.attrs(): process(name,value) This automatically reads the attributes of the group/data. Do not change the active group/data while processing the list. This does not correspond to an existing NeXus API function, but combines the work of attrinfo/initattrdir/getnextattr/getattr. """ self.initattrdir() n = self.getattrinfo() for dummy in range(n): name,length,dtype = self.getnextattr() value = self.getattr(name,length,dtype) yield name,value # ==== Linking ==== nxlib.nxigetgroupid_.restype = c_int nxlib.nxigetgroupid_.argtypes = [c_void_p, c_NXlink_p] def getgroupID(self): """ Return the id of the current group so we can link to it later. Raises NeXusError Corresponds to NXgetgroupID(handle, &ID) """ ID = _NXlink() status = nxlib.nxigetgroupid_(self.handle,_ref(ID)) if status == ERROR: raise NeXusError("Could not link to group: %s" % (self._loc())) return ID nxlib.nxigetdataid_.restype = c_int nxlib.nxigetdataid_.argtypes = [c_void_p, c_NXlink_p] def getdataID(self): """ Return the id of the current data so we can link to it later. Raises NeXusError Corresponds to NXgetdataID(handle, &ID) """ ID = _NXlink() status = nxlib.nxigetdataid_(self.handle,_ref(ID)) if status == ERROR: raise NeXusError("Could not link to data: %s" % (self._loc())) return ID nxlib.nximakelink_.restype = c_int nxlib.nximakelink_.argtypes = [c_void_p, c_NXlink_p] def makelink(self, ID): """ Link the previously captured group/data ID into the currently open group. Raises NeXusError Corresponds to NXmakelink(handle, &ID) """ status = nxlib.nximakelink_(self.handle,_ref(ID)) if status == ERROR: raise NeXusError("Could not make link: %s" % (self._loc())) nxlib.nximakenamedlink_.restype = c_int nxlib.nximakenamedlink_.argtypes = [c_void_p, c_char_p, c_NXlink_p] def makenamedlink(self,name,ID): """ Link the previously captured group/data ID into the currently open group, but under a different name. Raises NeXusError Corresponds to NXmakenamedlink(handle,name,&ID) """ status = nxlib.nximakenamedlink_(self.handle,name,_ref(ID)) if status == ERROR: raise NeXusError("Could not make link %s: %s" % (name, self._loc())) nxlib.nxisameid_.restype = c_int nxlib.nxisameid_.argtypes = [c_void_p, c_NXlink_p, c_NXlink_p] def sameID(self, ID1, ID2): """ Return True of ID1 and ID2 point to the same group/data. This should not raise any errors. Corresponds to NXsameID(handle,&ID1,&ID2) """ status = nxlib.nxisameid_(self.handle, _ref(ID1), _ref(ID2)) return status == OK nxlib.nxiopensourcegroup_.restype = c_int nxlib.nxiopensourcegroup_.argtyps = [c_void_p] def opensourcegroup(self): """ If the current node is a linked to another group or data, then open the group or data that it is linked to. Note: it is unclear how can we tell if we are linked, other than perhaps the existence of a 'target' attribute in the current item. Raises NeXusError. Corresponds to NXopensourcegroup(handle) """ status = nxlib.nxiopensourcegroup_(self.handle) if status == ERROR: raise NeXusError("Could not open source group: %s" % (self._loc())) def link(self): """ Returns the item which the current item links to, or None if the current item is not linked. This is equivalent to scanning the attributes for target and returning it if target is not equal to self. This does not correspond to an existing NeXus API function, but combines the work of attrinfo/initattrdir/getnextattr/getattr. """ n = self.getattrinfo() self.initattrdir() for dummy in range(n): name,length,dtype = self.getnextattr() if name == "target": target = self.getattr(name,length,dtype) # print("target %s, path %s" % (target,self.path)) if target != self.path: return target else: return None return None # ==== External linking ==== nxlib.nxiinquirefile_.restype = c_int nxlib.nxiinquirefile_.argtypes = [c_void_p, c_char_p, c_int] def inquirefile(self, maxnamelen=MAXPATHLEN): """ Return the filename for the current file. This may be different from the file that was opened (file.filename) if the current group is an external link to another file. Raises NeXusError if this fails. Corresponds to NXinquirefile(&handle,file,len) """ filename = ctypes.create_string_buffer(maxnamelen) status = nxlib.nxiinquirefile_(self.handle,filename,maxnamelen) if status == ERROR: raise NeXusError("Could not determine filename: %s" % (self._loc())) return filename.value nxlib.nxilinkexternal_.restype = c_int nxlib.nxilinkexternal_.argtyps = [c_void_p, c_char_p, c_char_p, c_char_p] def linkexternal(self, name, nxclass, url): """ Return the filename for the external link if there is one, otherwise return None. Raises NeXusError if link fails. Corresponds to NXisexternalgroup(&handle,name,nxclass,file,len) """ status = nxlib.nxilinkexternal_(self.handle,name,nxclass,url) if status == ERROR: raise NeXusError("Could not link %s to %s: %s" % (name, url, self._loc())) nxlib.nxiisexternalgroup_.restype = c_int nxlib.nxiisexternalgroup_.argtyps = [c_void_p, c_char_p, c_char_p, c_char_p, c_int] def isexternalgroup(self, name, nxclass, maxnamelen=MAXPATHLEN): """ Return the filename for the external link if there is one, otherwise return None. Corresponds to NXisexternalgroup(&handle,name,nxclass,file,len) """ url = ctypes.create_string_buffer(maxnamelen) status = nxlib.nxiisexternalgroup_(self.handle,name,nxclass, url,maxnamelen) if status == ERROR: return None else: return url.value # ==== Utility functions ==== def _loc(self): """ Return file location as string filename(path) This is an extension to the NeXus API. """ return "%s(%s)"%(self.filename,self.path) def _poutput(self, dtype, shape): """ Build space to collect a nexus data element. Returns data,pdata,size,datafn where - data is a python type to hold the returned data - pdata is the pointer to the start of the data - size is the number of bytes in the data block - datafn is a lamba expression to extract the return value from data Note that datafn can return a string, a scalar or an array depending on the data type and shape of the data group. """ if isinstance(shape, int): shape = [shape] if len(shape) == 1 and dtype == 'char': # string - use ctypes allocator size = int(shape[0]) data = ctypes.create_string_buffer(size) pdata = data datafn = lambda: data.value else: # scalar, array or string list - use numpy array if dtype=='char': data = numpy.zeros(shape[:-1], dtype='S%i'%shape[-1]) else: data = numpy.zeros(shape, dtype) if len(shape) == 1 and shape[0] == 1: datafn = lambda: data[0] else: datafn = lambda: data pdata = data.ctypes.data size = data.nbytes return data,pdata,size,datafn def _pinput(self, data, dtype, shape): """ Convert an input array to a C pointer to a dense array. Returns data, pdata where - data is a possibly new copy of the array - pdata is a pointer to the beginning of the array. Note that you must hold a reference to data for as long as you need pdata to keep the memory from being released to the heap. """ if isinstance(shape, int): shape = [shape] if dtype == "char": data = numpy.asarray(data, dtype='S%d'%(shape[-1])) else: # Convert scalars to vectors of length one if numpy.prod(shape) == 1 and not hasattr(data,'shape'): data = numpy.array([data], dtype=dtype) # Check that dimensions match # Ick! need to exclude dimensions of length 1 in order to catch # array slices such as a[:,1], which only report one dimension input_shape = numpy.array([i for i in data.shape if i != 1]) target_shape = numpy.array([i for i in shape if i != 1]) if len(input_shape) != len(target_shape) \ or (input_shape != target_shape).any(): raise ValueError("Shape mismatch %s!=%s: %s" % (data.shape, shape, self._loc())) # Check data type if str(data.dtype) != dtype: raise ValueError("Type mismatch %s!=%s: %s" % (dtype, data.dtype, self._loc())) data = numpy.ascontiguousarray(data) pdata = data.ctypes.data return data,pdata def show(self, path=None, indent=0): """ Print the structure of a NeXus file from the current node. TODO: Break this into a tree walker and a visitor. """ oldpath = self.path self.openpath(path) print("=== File", self.inquirefile(), path) self._show(indent=indent) self.openpath(oldpath) def _show(self, indent=0): """ Print the structure of a NeXus file from the current node. TODO: Break this into a tree walker and a visitor. """ prefix = ' '*indent link = self.link() if link: print("%(prefix)s-> %(link)s" % locals()) return for attr,value in self.attrs(): print("%(prefix)s@%(attr)s: %(value)s" % locals()) for name,nxclass in self.entries(): if nxclass == "SDS": shape,dtype = self.getinfo() dims = "x".join([str(x) for x in shape]) print("%(prefix)s%(name)s %(dtype)s %(dims)s" % locals()) link = self.link() if link: print(" %(prefix)s-> %(link)s" % locals()) else: for attr,value in self.attrs(): print(" %(prefix)s@%(attr)s: %(value)s" % locals()) if numpy.prod(shape) < 8: value = self.getdata() print(" %s%s"%(prefix,str(value))) else: print("%(prefix)s%(name)s %(nxclass)s" % locals()) self._show(indent=indent+2) __id__ = "$ID$" python-nxs/nxs/test/000077500000000000000000000000001306222103500147665ustar00rootroot00000000000000python-nxs/nxs/test/__init__.py000066400000000000000000000000241306222103500170730ustar00rootroot00000000000000from .napi import * python-nxs/nxs/test/napi/000077500000000000000000000000001306222103500157155ustar00rootroot00000000000000python-nxs/nxs/test/napi/__init__.py000066400000000000000000000013211306222103500200230ustar00rootroot00000000000000"""@package nxs Python NeXus interface. NeXus_ is a common data format for neutron, Xray and muon science. The files contain multidimensional data elements grouped into a hierarchical structure. The data sets are self-describing, with a description of the instrument configuration including the units used as well as the data measured. The NeXus file interface requires compiled libraries to read the underlying HDF_ files. Binary packages are available for some platforms from the NeXus site. Details of where the nxs package searches for the libraries are recorded in `nxs.napi`. """ from .test_constants import test_constants from .test_file_creation import test_file_creation from .test_field_creation import * python-nxs/nxs/test/napi/nxs_napi_lowlevel.py000066400000000000000000000127531306222103500220270ustar00rootroot00000000000000"""@package nxs Python NeXus interface. NeXus_ is a common data format for neutron, Xray and muon science. The files contain multidimensional data elements grouped into a hierarchical structure. The data sets are self-describing, with a description of the instrument configuration including the units used as well as the data measured. The NeXus file interface requires compiled libraries to read the underlying HDF_ files. Binary packages are available for some platforms from the NeXus site. Details of where the nxs package searches for the libraries are recorded in `nxs.napi`. """ #testing low level functionality import unittest import sys import os import numpy sys.path.append("../") #load the testing module import nxs.napi class NAPILowLevelTestCase(unittest.TestCase): #static members _fname = "NAPI_LL_TEST.nxs" _shape1 = [20,30] _shape2 = [10,20,30] _dshape = [nxs.UNLIMITED,20,30] def setUp(self): self._file = nxs.napi.open(self._fname,"w4") self._file.makegroup("scan_1","NXentry") pass def tearDown(self): self._file.close() try: os.remove(self._fname) except: pass def test_file_open(self): self.assertRaises(nxs.napi.NeXusError,nxs.napi.open,"bla.nxs","r") #should produce no exception f = nxs.napi.open("bla.nxs","w") f = nxs.napi.open("bla.nxs","rw") f.close() os.remove("bla.nxs") def test_file_attributes(self): self._file.makegroup("/data","NXentry") self._file.opengroup("/data") self._file.closegroup() def test_file_groups(self): #should not work - group does not exist self.assertRaises(nxs.napi.NeXusError,self._file.opengroup,"data") self.assertRaises(nxs.napi.NeXusError,self._file.opengroup,"/data/test") #this should work self._file.makegroup("data","NXentry") #try to open the group in various ways self.assertRaises(nxs.napi.NeXusError,self._file.opengrouppath,"/data/test/dir") self.assertRaises(ValueError,self._file.opengroup,"data","NXinstrument") #the documentation and the code is not very consistent with #its exceptions - NeXusError sometimes ValueError self._file.opengrouppath("/data") #should work self._file.openpath("/data") #works too self.assertRaises(nxs.napi.NeXusError,self._file.openpath,"/data/test/dir") def test_file_compdata(self): pass def test_file_simpledata(self): pass def __test_slab_data(self,typecode): self._file.opengroup("scan_1") #--------------write data to disk--------------------------- def write_slab(np,d,offset,shape): for i in range(np): offset[0] = i*shape[0] d[...] = i self._file.putslab(d,offset,shape) #-------------read data from disk---------------------------- def read_slab(np,d,offset,shape): for i in range(np): d[...] = i offset[0] = i*shape[0] o = numpy.squeeze(self._file.getslab(offset,shape)) self.assertListEqual(d.flatten().tolist(),o.flatten().tolist()) #--------------------IO per frame----------------------------- #create and open dataset self._file.makedata("data",dtype=typecode,shape=self._dshape) self._file.opendata("data") #create data d = numpy.ones(self._shape1,dtype=typecode) offset = [0,0,0] shape = [1]+self._shape1 #write data write_slab(10,d,offset,shape) #read data back d[...] = 1 read_slab(10,d,offset,shape) #close dataset self._file.closedata() #--------------------IO per frame with compression----------------- #create and open dataset self._file.compmakedata("data3",dtype=typecode,shape=self._dshape, mode="lzw", chunks=[1]+self._shape1) self._file.opendata("data") #create data d = numpy.ones(self._shape1,dtype=typecode) offset = [0,0,0] shape = [1]+self._shape1 #write data write_slab(10,d,offset,shape) #read data back d[...] = 1 read_slab(10,d,offset,shape) #close dataset self._file.closedata() #------------------volume IO---------------------------------- #create new dataset self._file.makedata("data2",dtype=typecode,shape=self._dshape) self._file.opendata("data2") #create data d = numpy.ones(self._shape2,dtype=typecode) offset = [0,0,0] shape = self._shape2 #write data write_slab(10,d,offset,shape) #read data back d[...] = 1 read_slab(10,d,offset,shape) #close the dataset self._file.closedata() self._file.closegroup() def test_file_slabdata_uint16(self): self.__test_slab_data("uint16") def test_file_slabdata_int16(self): self.__test_slab_data("int16") def test_file_slabdata_uint32(self): self.__test_slab_data("uint32") def test_file_slabdata_uint32(self): self.__test_slab_data("int32") def test_file_slabdata_float32(self): self.__test_slab_data("float32") def test_file_slabdata_float64(self): self.__test_slab_data("float64") python-nxs/nxs/test/napi/test_constants.py000066400000000000000000000034351306222103500213470ustar00rootroot00000000000000"""@package nxs Python NeXus interface. NeXus_ is a common data format for neutron, Xray and muon science. The files contain multidimensional data elements grouped into a hierarchical structure. The data sets are self-describing, with a description of the instrument configuration including the units used as well as the data measured. The NeXus file interface requires compiled libraries to read the underlying HDF_ files. Binary packages are available for some platforms from the NeXus site. Details of where the nxs package searches for the libraries are recorded in `nxs.napi`. """ import unittest import nxs.napi as napi class test_constants(unittest.TestCase): """ Testing constants exported by nxs. """ def test_scalar_constants(self): self.assertEqual(napi.NOSTRIP,128) self.assertEqual(napi.UNLIMITED,-1) self.assertEqual(napi.MAXRANK,32) self.assertEqual(napi.MAXNAMELEN,64) self.assertEqual(napi.MAXPATHLEN,1024) def test_type_codes(self): d_expected = {"char":4,"float32":5,"float64":6, "int8":20,"uint8":21,"int16":22,"uint16":23, "int32":24,"uint32":25,"int64":26,"uint64":27} self.assertDictEqual(napi._nxtype_code,d_expected) def test_open_modes(self): dr = {"r":1,"rw":2,"w":3,"w4":4,"w5":5,"wx":6} self.assertDictEqual(napi._nxopen_mode,dr) def test_compression_code(self): d_expected = {"none":100,"lzw":200,"rle":300,"huffman":400} self.assertDictEqual(napi._compression_code,d_expected) def test_h4skip(self): l_expected = ['CDF0.0','_HDF_CHK_TBL_','Attr0.0','RIG0.0','RI0.0', 'RIATTR0.0N','RIATTR0.0C'] self.assertListEqual(napi.H4SKIP,l_expected) python-nxs/nxs/test/napi/test_field_creation.py000066400000000000000000000010401306222103500222700ustar00rootroot00000000000000import nxs.napi as napi import unittest class test_field_creation_hdf5(unittest.TestCase): mode = "w5" filename = "test_file_creation_hdf5.nxs" def setUp(self): self._file = napi.NeXus(self.filename,self.mode) self._file.makegroup("entry","NXentry") self._file.opengroup("entry") def tearDown(self): self._file.close() def test_field_without_compression(self): self._file.makedata("data1",'uint32',(3,4)) self._file.opendata("data1") python-nxs/nxs/test/napi/test_file_creation.py000066400000000000000000000017421306222103500221350ustar00rootroot00000000000000"""@package nxs Python NeXus interface. NeXus_ is a common data format for neutron, Xray and muon science. The files contain multidimensional data elements grouped into a hierarchical structure. The data sets are self-describing, with a description of the instrument configuration including the units used as well as the data measured. The NeXus file interface requires compiled libraries to read the underlying HDF_ files. Binary packages are available for some platforms from the NeXus site. Details of where the nxs package searches for the libraries are recorded in `nxs.napi`. """ import unittest import nxs.napi as napi import os class test_file_creation(unittest.TestCase): def test_hdf5(self): f = napi.open("test_hdf5.nxs","w5") os.remove("test_hdf5.nxs") def test_hdf4(self): f = napi.open("test_hdf4.nxs","w4") os.remove("test_hdf4.nxs") def test_mxml(self): f = napi.open("test_mxml.nxs","wx") python-nxs/nxs/tree.py000066400000000000000000003460501306222103500153300ustar00rootroot00000000000000#!/usr/bin/env python # This program is public domain # Author: Paul Kienzle, Ray Osborn """ NeXus data as Python trees ========================== The `nexus.tree` modules are designed to accomplish two goals: 1. To provide convenient access to existing data contained in NeXus files. 2. To enable new NeXus data to be created and manipulated interactively. These goals are achieved by mapping hierarchical NeXus data structures directly into python objects, which either represent NeXus groups or NeXus fields. Entries in a group are referenced much like fields in a class are referenced in python. The entire data hierarchy can be referenced at any time, whether the NeXus data has been loaded in from an existing NeXus file or created dynamically within the python session. This provides a much more natural scripting interface to NeXus data than the directory model of the `nexus.napi` interface. Example 1: Loading a NeXus file ------------------------------- The following commands loads NeXus data from a file, displays (some of) the contents as a tree, and then accesses individual data items. >>> from nexpy.api import nexus as nx >>> a=nx.load('sns/data/ARCS_7326.nxs') >>> print a.tree root:NXroot @HDF5_Version = 1.8.2 @NeXus_version = 4.2.1 @file_name = ARCS_7326.nxs @file_time = 2010-05-05T01:59:25-05:00 entry:NXentry data:NXdata data = float32(631x461x4x825) @axes = rotation_angle:tilt_angle:sample_angle:time_of_flight @signal = 1 rotation_angle = float32(632) @units = degree sample_angle = [ 210. 215. 220. 225. 230.] @units = degree tilt_angle = float32(462) @units = degree time_of_flight = float32(826) @units = microsecond run_number = 7326 sample:NXsample pulse_time = 2854.94747365 @units = microsecond . . . >>> a.entry.run_number NXfield(7326) So the tree returned from load() has an entry for each group, field and attribute. You can traverse the hierarchy using the names of the groups. For example, tree.entry.instrument.detector.distance is an example of a field containing the distance to each pixel in the detector. Entries can also be referenced by NXclass name, such as tree.NXentry[0].instrument. Since there may be multiple entries of the same NeXus class, the NXclass attribute returns a (possibly empty) list. The load() and save() functions are implemented using the class `nexus.tree.NeXusTree`, a subclass of `nexus.napi.NeXus` which allows all the usual API functions. Example 2: Creating a NeXus file dynamically -------------------------------------------- The second example shows how to create NeXus data dynamically and saves it to a file. The data are first created as Numpy arrays >>> import numpy as np >>> x=y=np.linspace(0,2*np.pi,101) >>> X,Y=np.meshgrid(y,x) >>> z=np.sin(X)*np.sin(Y) Then a NeXus data groups are created and the data inserted to produce a NeXus-compliant structure that can be saved to a file. >>> root=nx.NXroot(NXentry()) >>> print root.tree root:NXroot entry:NXentry >>> root.entry.data=nx.NXdata(z,[x,y]) Additional metadata can be inserted before saving the data to a file. >>> root.entry.sample=nx.NXsample() >>> root.entry.sample.temperature = 40.0 >>> root.entry.sample.temperature.units = 'K' >>> root.save('example.nxs') NXfield objects have much of the functionality of Numpy arrays. They may be used in simple arithmetic expressions with other NXfields, Numpy arrays or scalar values and will be cast as ndarray objects if used as arguments in Numpy modules. >>> x=nx.NXfield(np.linspace(0,10.0,11)) >>> x NXfield([ 0. 1. 2. ..., 8. 9. 10.]) >>> x + 10 NXfield([ 10. 11. 12. ..., 18. 19. 20.]) >>> sin(x) array([ 0. , 0.84147098, 0.90929743, ..., 0.98935825, 0.41211849, -0.54402111]) If the arithmetic operation is assigned to a NeXus group attribute, it will be automatically cast as a valid NXfield object with the type and shape determined by the Numpy array type and shape. >>> entry.data.result = sin(x) >>> entry.data.result NXfield([ 0. 0.84147098 0.90929743 ..., 0.98935825 0.41211849 -0.54402111]) >>> entry.data.result.dtype, entry.data.result.shape (dtype('float64'), (11,)) NeXus Objects ------------- Properties of the entry in the tree are referenced by attributes that depend on the object type, different nx attributes may be available. Objects (class NXobject) have attributes shared by both groups and fields:: * nxname object name * nxclass object class for groups, 'NXfield' for fields * nxgroup group containing the entry, or None for the root * attrs dictionary of NeXus attributes for the object Groups (class NXgroup) have attributes for accessing children:: * entries dictionary of entries within the group * component('nxclass') return group entries of a particular class * dir() print the list of entries in the group * tree return the list of entries and subentries in the group * plot() plot signal and axes for the group, if available Fields (class NXfield) have attributes for accessing data: * shape dimensions of data in the field * dtype data type * nxdata data in the field Linked fields or groups (class NXlink) have attributes for accessing the link:: * nxlink reference to the linked field or group NeXus attributes (class NXattr) have a type and a value only:: * dtype attribute type * nxdata attribute data There is a subclass of NXgroup for each group class defined by the NeXus standard, so it is possible to create an NXgroup of NeXus class NXsample directly using: >>> sample = NXsample() The default group name will be the class name following the 'NX', so the above group will have an nxname of 'sample'. However, this is overridden by the attribute name when it is assigned as a group attribute, e.g., >>> entry.sample1 = NXsample() >>> entry.sample1.nxname sample1 You can traverse the tree by component class instead of component name. Since there may be multiple components of the same class in one group you will need to specify which one to use. For example, tree.NXentry[0].NXinstrument[0].NXdetector[0].distance references the first detector of the first instrument of the first entry. Unfortunately, there is no guarantee regarding the order of the entries, and it may vary from call to call, so this is mainly useful in iterative searches. Unit Conversion --------------- Data can be stored in the NeXus file in a variety of units, depending on which facility is storing the file. This makes life difficult for reduction and analysis programs which must know the units they are working with. Our solution to this problem is to allow the reader to retrieve data from the file in particular units. For example, if detector distance is stored in the file using millimeters you can retrieve them in meters using:: entry.instrument.detector.distance.convert('m') See `nexus.unit` for more details on the unit formats supported. Reading and Writing Slabs ------------------------- The slab interface to field data works by opening the file handle and keeping it open as long as the slab interface is needed. This is done in python 2.5 using the with statement. Once the context is entered, get() and put() methods on the object allow you to read and write data a slab at a time. For example:: # Read a Ni x Nj x Nk array one vector at a time with root.NXentry[0].data.data as slab: Ni,Nj,Nk = slab.shape size = [1,1,Nk] for i in range(Ni): for j in range(Nj): value = slab.get([i,j,0],size) The equivalent can be done in Python 2.4 and lower using the context functions __enter__ and __exit__:: slab = data.slab.__enter__() ... do the slab functions ... data.slab.__exit__() Plotting NeXus data ------------------- There is a plot() method for groups that automatically looks for 'signal' and 'axes' attributes within the group in order to determine what to plot. These are defined by the 'nxsignal' and 'nxaxes' properties of the group. This means that the method will determine whether the plot should be one- or two- dimensional. For higher than two dimensions, only the top slice is plotted by default. The plot method accepts as arguments the standard matplotlib.pyplot.plot format strings to customize one-dimensional plots, axis and scale limits, and will transmit keyword arguments to the matplotlib plotting methods. >>> a=nx.load('chopper.nxs') >>> a.entry.monitor1.plot() >>> a.entry.monitor2.plot('r+', xmax=2600) It is possible to plot over the existing figure with the oplot() method and to plot with logarithmic intensity scales with the logplot() method. The x- and y-axes can also be rendered logarithmically using the logx and logy keywards. Although the plot() method uses matplotlib by default to plot the data, you can replace this with your own plotter by setting nexus.NXgroup._plotter to your own plotter class. The plotter class has one method:: plot(signal, axes, entry, title, format, **opts) where signal is the field containing the data, axes are the fields listing the signal sample points, entry is file/path within the file to the data group and title is the title of the group or the parent NXentry, if available. """ from __future__ import with_statement from copy import copy, deepcopy import numpy as np import napi from napi import NeXusError #Memory in MB NX_MEMORY = 500 __all__ = ['NeXusTree', 'NXobject', 'NXfield', 'NXgroup', 'NXattr', 'NX_MEMORY', 'setmemory', 'load', 'save', 'tree', 'centers', 'NXlink', 'NXlinkfield', 'NXlinkgroup', 'SDS', 'NXlinkdata'] #List of defined base classes (later added to __all__) _nxclasses = ['NXroot', 'NXentry', 'NXsubentry', 'NXdata', 'NXmonitor', 'NXlog', 'NXsample', 'NXinstrument', 'NXaperture', 'NXattenuator', 'NXbeam', 'NXbeam_stop', 'NXbending_magnet', 'NXcharacterization', 'NXcollection', 'NXcollimator', 'NXcrystal', 'NXdetector', 'NXdisk_chopper', 'NXenvironment', 'NXevent_data', 'NXfermi_chopper', 'NXfilter', 'NXflipper', 'NXgeometry', 'NXguide', 'NXinsertion_device', 'NXmirror', 'NXmoderator', 'NXmonochromator', 'NXnote', 'NXorientation', 'NXparameter', 'NXpolarizer', 'NXpositioner', 'NXprocess', 'NXsensor', 'NXshape', 'NXsource', 'NXtranslation', 'NXuser', 'NXvelocity_selector', 'NXtree'] np.set_printoptions(threshold=5) class NeXusTree(napi.NeXus): """ Structure-based interface to the NeXus file API. Usage:: file = NeXusTree(filename, ['r','rw','w']) - open the NeXus file root = file.readfile() - read the structure of the NeXus file. This returns a NeXus tree. file.writefile(root) - write a NeXus tree to the file. data = file.readpath(path) - read data from a particular path Example:: nx = NeXusTree('REF_L_1346.nxs','r') tree = nx.readfile() for entry in tree.NXentry: process(entry) copy = NeXusTree('modified.nxs','w') copy.writefile(tree) Note that the large datasets are not loaded immediately. Instead, the when the data set is requested, the file is reopened, the data read, and the file closed again. open/close are available for when we want to read/write slabs without the overhead of moving the file cursor each time. The NXdata objects in the returned tree hold the object values. """ def readfile(self): """ Read the NeXus file structure from the file and return a tree of NXobjects. Large datasets are not read until they are needed. """ self.open() self.openpath("/") root = self._readgroup() self.close() root._group = None # Resolve links (not necessary now that link is set as a property) #self._readlinks(root, root) root._file = self return root def writefile(self, tree): """ Write the NeXus file structure to a file. The file is assumed to start empty. Updating individual objects can be done using the napi interface, with nx.handle as the nexus file handle. """ self.open() links = [] for entry in tree.entries.values(): links += self._writegroup(entry, path="") self._writelinks(links) self.close() def readpath(self, path): """ Return the data on a particular file path. Returns a numpy array containing the data, a python scalar, or a string depending on the shape and storage class. """ self.open() self.openpath(path) try: return self.getdata() except ValueError: return None def _readdata(self, name): """ Read a data object and return it as an NXfield or NXlink. """ # Finally some data, but don't read it if it is big # Instead record the location, type and size self.opendata(name) attrs={} attrs = self.getattrs() if 'target' in attrs and attrs['target'] != self.path: # This is a linked dataset; don't try to load it. data = NXlinkfield(target=attrs['target'], name=name) else: dims,type = self.getinfo() #Read in the data if it's not too large if np.prod(dims) < 1000:# i.e., less than 1k dims try: value = self.getdata() except ValueError: value = None else: value = None data = NXfield(value=value,name=name,dtype=type,shape=dims,attrs=attrs) self.closedata() data._infile = data._saved = data._changed = True return data # These are groups that HDFView explicitly skips _skipgroups = ['CDF0.0','_HDF_CHK_TBL_','Attr0.0','RIG0.0','RI0.0', 'RIATTR0.0N','RIATTR0.0C'] def _readchildren(self,n): children = {} for _item in range(n): name,nxclass = self.getnextentry() if nxclass in self._skipgroups: pass # Skip known bogus classes elif nxclass == 'SDS': # NXgetnextentry returns 'SDS' as the class for NXfields children[name] = self._readdata(name) else: self.opengroup(name,nxclass) children[name] = self._readgroup() self.closegroup() return children def _readgroup(self): """ Read the currently open group and return it as an NXgroup. """ n,name,nxclass = self.getgroupinfo() attrs = {} attrs = self.getattrs() if 'target' in attrs and attrs['target'] != self.path: # This is a linked group; don't try to load it. group = NXlinkgroup(target=attrs['target'], name=name) else: children = self._readchildren(n) # If we are subclassed with a handler for the particular # NXentry class name use that constructor for the group # rather than the generic NXgroup class. group = NXgroup(nxclass=nxclass,name=name,attrs=attrs,entries=children) # Build chain back structure for obj in children.values(): obj._group = group group._infile = group._saved = group._changed = True return group def _readlinks(self, root, group): """ Convert linked objects into direct references. """ for entry in group.entries.values(): if isinstance(entry, NXlink): link = root try: for level in entry._target[1:].split('/'): link = getattr(link,level) entry.nxlink = link except AttributeError: pass elif isinstance(entry, NXgroup): self._readlinks(root, entry) def _writeattrs(self, attrs): """ Return the attributes for the currently open group/data. If no group or data object is open, the file attributes are returned. """ for name,pair in attrs.iteritems(): self.putattr(name,pair.nxdata,pair.dtype) def _writedata(self, data, path): """ Write the given data to a file. NXlinks cannot be written until the linked group is created, so this routine returns the set of links that need to be written. Call writelinks on the list. """ path = path + "/" + data.nxname # If the data is linked then if hasattr(data,'_target'): return [(path, data._target)] shape = data.shape if shape == (): shape = (1,) #If the array size is too large, their product needs a long integer if np.prod(shape) > 10000: # Compress the fastest moving dimension of large datasets slab_dims = np.ones(len(shape),'i') if shape[-1] < 100000: slab_dims[-1] = shape[-1] else: slab_dims[-1] = 100000 self.compmakedata(data.nxname, data.dtype, shape, 'lzw', slab_dims) else: # Don't use compression for small datasets try: self.makedata(data.nxname, data.dtype, shape) except StandardError as errortype: print("Error in tree, makedata: ", errortype) self.opendata(data.nxname) self._writeattrs(data.attrs) value = data.nxdata if value is not None: self.putdata(data.nxdata) self.closedata() return [] def _writegroup(self, group, path): """ Write the given group structure, including the data. NXlinks cannot be written until the linked group is created, so this routine returns the set of links that need to be written. Call writelinks on the list. """ path = path + "/" + group.nxname links = [] self.makegroup(group.nxname, group.nxclass) self.opengroup(group.nxname, group.nxclass) self._writeattrs(group.attrs) if hasattr(group, '_target'): links += [(path, group._target)] for child in group.entries.values(): if child.nxclass == 'NXfield': links += self._writedata(child,path) elif hasattr(child,'_target'): links += [(path+"/"+child.nxname,child._target)] else: links += self._writegroup(child,path) self.closegroup() return links def _writelinks(self, links): """ Create links within the NeXus file. THese are defined by the set of pairs returned by _writegroup. """ gid = {} # identify targets for path,target in links: gid[target] = None # find gids for targets for target in gid.iterkeys(): self.openpath(target) # Can't tell from the name if we are linking to a group or # to a dataset, so cheat and rely on getdataID to signal # an error if we are not within a group. try: gid[target] = self.getdataID() except NeXusError: gid[target] = self.getgroupID() # link sources to targets for path,target in links: if path != target: # ignore self-links parent = "/".join(path.split("/")[:-1]) self.openpath(parent) self.makelink(gid[target]) def _readaxes(axes): """ Return a list of axis names stored in the 'axes' attribute. The delimiter separating each axis can be white space, a comma, or a colon. """ import re sep=re.compile('[\[]*(\s*,*:*)+[\]]*') return filter(lambda x: len(x)>0, sep.split(axes)) class AttrDict(dict): """ A dictionary class to assign all attributes to the NXattr class. """ def __setitem__(self, key, value): if isinstance(value, NXattr): dict.__setitem__(self, key, value) else: dict.__setitem__(self, key, NXattr(value)) class NXattr(object): """ Class for NeXus attributes of a NXfield or NXgroup object. This class is only used for NeXus attributes that are stored in a NeXus file and helps to distinguish them from Python attributes. There are two Python attributes for each NeXus attribute. Python Attributes ----------------- nxdata : string, Numpy scalar, or Numpy ndarray The value of the NeXus attribute. dtype : string The data type of the NeXus attribute. This is set to 'char' for a string attribute or the string of the corresponding Numpy data type for a numeric attribute. NeXus Attributes ---------------- NeXus attributes are stored in the 'attrs' dictionary of the parent object, NXfield or NXgroup, but can often be referenced or assigned using the attribute name as if it were an object attribute. For example, after assigning the NXfield, the following three attribute assignments are all equivalent:: >>> entry.sample.temperature = NXfield(40.0) >>> entry.sample.temperature.attrs['units'] = 'K' >>> entry.sample.temperature.units = NXattr('K') >>> entry.sample.temperature.units = 'K' The third version above is only allowed for NXfield attributes and is not allowed if the attribute has the same name as one of the following internally defined attributes, i.e., ['entries', 'attrs', 'dtype','shape'] or if the attribute name begins with 'nx' or '_'. It is only possible to reference attributes with one of the proscribed names using the 'attrs' dictionary. """ def __init__(self,value=None,dtype=''): if isinstance(value, NXattr): self._data,self._dtype = value.nxdata,value.dtype elif dtype: if dtype in np.typeDict: self._data,self._dtype = np.__dict__[dtype](value),dtype elif dtype == 'char': self._data,self._dtype = str(value),dtype else: raise NeXusError("Invalid data type") else: if isinstance(value, str): self._data,self._dtype = str(value), 'char' elif value is not None: if isinstance(value, NXobject): raise NeXusError("A data attribute cannot be a NXfield or NXgroup") else: self._data = np.array(value) self._dtype = self._data.dtype.name if self._data.size == 1: self._data = np.__dict__[self._dtype](self._data) else: self._data,self._dtype = None, 'char' def __str__(self): return str(self.nxdata) def __repr__(self): if str(self.dtype) == 'char': return "NXattr('%s')"%self.nxdata else: return "NXattr(%s)"%self.nxdata def __eq__(self, other): """ Return true if the value of the attribute is the same as the other. """ if isinstance(other, NXattr): return self.nxdata == other.nxdata else: return self.nxdata == other def _getdata(self): """ Return the attribute value. """ return self._data def _getdtype(self): return self._dtype nxdata = property(_getdata,doc="The attribute values") dtype = property(_getdtype, "Data type of NeXus attribute") _npattrs = filter(lambda x: not x.startswith('_'), np.ndarray.__dict__.keys()) class NXobject(object): """ Abstract base class for elements in NeXus files. The object has a subclass of NXfield, NXgroup, or one of the NXgroup subclasses. Child nodes should be accessible directly as object attributes. Constructors for NXobject objects are defined by either the NXfield or NXgroup classes. Python Attributes ----------------- nxclass : string The class of the NXobject. NXobjects can have class NXfield, NXgroup, or be one of the NXgroup subclasses. nxname : string The name of the NXobject. Since it is possible to reference the same Python object multiple times, this is not necessarily the same as the object name. However, if the object is part of a NeXus tree, this will be the attribute name within the tree. nxgroup : NXgroup The parent group containing this object within a NeXus tree. If the object is not part of any NeXus tree, it will be set to None. nxpath : string The path to this object with respect to the root of the NeXus tree. For NeXus data read from a file, this will be a group of class NXroot, but if the NeXus tree was defined interactively, it can be any valid NXgroup. nxroot : NXgroup The root object of the NeXus tree containing this object. For NeXus data read from a file, this will be a group of class NXroot, but if the NeXus tree was defined interactively, it can be any valid NXgroup. nxfile : NeXusTree The file handle of the root object of the NeXus tree containing this object. filename : string The file name of NeXus object's tree file handle. attrs : dict A dictionary of the NeXus object's attributes. Methods ------- dir(self, attrs=False, recursive=False): Print the group directory. The directory is a list of NeXus objects within this group, either NeXus groups or NXfield data. If 'attrs' is True, NXfield attributes are displayed. If 'recursive' is True, the contents of child groups are also displayed. tree: Return the object's tree as a string. It invokes the 'dir' method with both 'attrs' and 'recursive' set to True. Note that this is defined as a property attribute and does not require parentheses. save(self, filename, format='w5') Save the NeXus group into a file The object is wrapped in an NXroot group (with name 'root') and an NXentry group (with name 'entry'), if necessary, in order to produce a valid NeXus file. """ _class = "unknown" _name = "unknown" _group = None _file = None _infile = False _saved = False _changed = True def __str__(self): return "%s:%s"%(self.nxclass,self.nxname) def __repr__(self): return "NXobject('%s','%s')"%(self.nxclass,self.nxname) def _setattrs(self, attrs): for k,v in attrs.items(): self._attrs[k] = v def _str_name(self,indent=0): if self.nxclass == 'NXfield': return " "*indent+self.nxname else: return " "*indent+self.nxname+':'+self.nxclass def _str_value(self,indent=0): return "" def _str_attrs(self,indent=0): names = self.attrs.keys() names.sort() result = [] for k in names: result.append(" "*indent+"@%s = %s"%(k,self.attrs[k].nxdata)) return "\n".join(result) def _str_tree(self,indent=0,attrs=False,recursive=False): """ Print current object and possibly children. """ result = [self._str_name(indent=indent)] if attrs and self.attrs: result.append(self._str_attrs(indent=indent+2)) # Print children entries = self.entries if entries: names = entries.keys() names.sort() if recursive: for k in names: result.append(entries[k]._str_tree(indent=indent+2, attrs=attrs, recursive=True)) else: for k in names: result.append(entries[k]._str_name(indent=indent+2)) result return "\n".join(result) def walk(self): if False: yield def dir(self,attrs=False,recursive=False): """ Print the object directory. The directory is a list of NeXus objects within this object, either NeXus groups or NXfields. If 'attrs' is True, NXfield attributes are displayed. If 'recursive' is True, the contents of child groups are also displayed. """ print self._str_tree(attrs=attrs,recursive=recursive) @property def tree(self): """ Return the directory tree as a string. The tree contains all child objects of this object and their children. It invokes the 'dir' method with both 'attrs' and 'recursive' set to True. """ return self._str_tree(attrs=True,recursive=True) def __enter__(self): """ Open the datapath for reading or writing. Note: the results are undefined if you try accessing more than one slab at a time. Don't nest your "with data" statements! """ self._close_on_exit = not self.nxfile.isopen self.nxfile.open() # Force file open even if closed self.nxfile.openpath(self.nxpath) self._incontext = True return self.nxfile def __exit__(self, type, value, traceback): """ Close the file associated with the data. """ self._incontext = False if self._close_on_exit: self.nxfile.close() def save(self, filename=None, format='w5'): """ Save the NeXus object to a data file. An error is raised if the object is an NXroot group from an external file that has been opened as readonly and no file name is specified. The object is wrapped in an NXroot group (with name 'root') and an NXentry group (with name 'entry'), if necessary, in order to produce a valid NeXus file. Example ------- >>> data = NXdata(sin(x), x) >>> data.save('file.nxs') >>> print data.nxroot.tree root:NXroot @HDF5_Version = 1.8.2 @NeXus_version = 4.2.1 @file_name = file.nxs @file_time = 2012-01-20T13:14:49-06:00 entry:NXentry data:NXdata axis1 = float64(101) signal = float64(101) @axes = axis1 @signal = 1 >>> root.entry.data.axis1.units = 'meV' >>> root.save() """ if filename: if self.nxclass == "NXroot": root = self elif self.nxclass == "NXentry": root = NXroot(self) else: root = NXroot(NXentry(self)) if root.nxfile: root.nxfile.close() file = NeXusTree(filename, format) file.writefile(root) file.close() root._file = NeXusTree(filename, 'rw') root._setattrs(root._file.getattrs()) for node in root.walk(): node._infile = node._saved = True elif self.nxfile: for entry in self.nxroot.values(): entry.write() else: raise NeXusError("No output file specified") @property def infile(self): """ Returns True if the object has been created in a NeXus file. """ return self._infile @property def saved(self): """ Returns True if the object has been saved to a file. """ return self._saved @property def changed(self): """ Returns True if the object has been changed. This property is for use by external scripts that need to track which NeXus objects have been changed. """ return self._changed def set_unchanged(self, recursive=False): """ Set an object's change status to unchanged. """ if recursive: for node in self.walk(): node._changed = False else: self._changed = False def _getclass(self): return self._class def _getname(self): return self._name def _setname(self, value): self._name = str(value) def _getgroup(self): return self._group def _getpath(self): if self.nxgroup is None: return "" elif isinstance(self.nxgroup, NXroot): return "/" + self.nxname else: return self.nxgroup._getpath()+"/"+self.nxname def _getroot(self): if self.nxgroup is None: return self elif isinstance(self.nxgroup, NXroot): return self.nxgroup else: return self.nxgroup._getroot() def _getfile(self): return self.nxroot._file def _getfilename(self): return self.nxroot._file.filename def _getattrs(self): return self._attrs nxclass = property(_getclass, doc="Class of NeXus object") nxname = property(_getname, _setname, doc="Name of NeXus object") nxgroup = property(_getgroup, doc="Parent group of NeXus object") nxpath = property(_getpath, doc="Path to NeXus object") nxroot = property(_getroot, doc="Root group of NeXus object's tree") nxfile = property(_getfile, doc="File handle of NeXus object's tree") attrs = property(_getattrs, doc="NeXus attributes for an object") class NXfield(NXobject): """ A NeXus data field. This is a subclass of NXobject that contains scalar, array, or string data and associated NeXus attributes. NXfield(value=None, name='unknown', dtype='', shape=[], attrs={}, file=None, path=None, group=None, **attr) Input Parameters ---------------- value : scalar value, Numpy array, or string The numerical or string value of the NXfield, which is directly accessible as the NXfield attribute 'nxdata'. name : string The name of the NXfield, which is directly accessible as the NXfield attribute 'name'. If the NXfield is initialized as the attribute of a parent object, the name is automatically set to the name of this attribute. dtype : string The data type of the NXfield value, which is directly accessible as the NXfield attribute 'dtype'. Valid input types correspond to standard Numpy data types, using names defined by the NeXus API, i.e., 'float32' 'float64' 'int8' 'int16' 'int32' 'int64' 'uint8' 'uint16' 'uint32' 'uint64' 'char' If the data type is not specified, then it is determined automatically by the data type of the 'value' parameter. shape : list of ints The dimensions of the NXfield data, which is accessible as the NXfield attribute 'shape'. This corresponds to the shape of the Numpy array. Scalars (numeric or string) are stored as Numpy zero-rank arrays, for which shape=[]. attrs : dict A dictionary containing NXfield attributes. The dictionary values should all have class NXattr. file : filename The file from which the NXfield has been read. path : string The path to this object with respect to the root of the NeXus tree, using the convention for unix file paths. group : NXgroup or subclass of NXgroup The parent NeXus object. If the NXfield is initialized as the attribute of a parent group, this attribute is automatically set to the parent group. Python Attributes ----------------- nxclass : 'NXfield' The class of the NXobject. nxname : string The name of the NXfield. Since it is possible to reference the same Python object multiple times, this is not necessarily the same as the object name. However, if the field is part of a NeXus tree, this will be the attribute name within the tree. nxgroup : NXgroup The parent group containing this field within a NeXus tree. If the field is not part of any NeXus tree, it will be set to None. dtype : string or Numpy dtype The data type of the NXfield value. If the NXfield has been initialized but the data values have not been read in or defined, this is a string. Otherwise, it is set to the equivalent Numpy dtype. shape : list or tuple of ints The dimensions of the NXfield data. If the NXfield has been initialized but the data values have not been read in or defined, this is a list of ints. Otherwise, it is set to the equivalent Numpy shape, which is a tuple. Scalars (numeric or string) are stored as Numpy zero-rank arrays, for which shape=(). attrs : dict A dictionary of all the NeXus attributes associated with the field. These are objects with class NXattr. nxdata : scalar, Numpy array or string The data value of the NXfield. This is normally initialized using the 'value' parameter (see above). If the NeXus data is contained in a file and the size of the NXfield array is too large to be stored in memory, the value is not read in until this attribute is directly accessed. Even then, if there is insufficient memory, a value of None will be returned. In this case, the NXfield array should be read as a series of smaller slabs using 'get'. nxdata_as('units') : scalar value or Numpy array If the NXfield 'units' attribute has been set, the data values, stored in 'nxdata', are returned after conversion to the specified units. nxpath : string The path to this object with respect to the root of the NeXus tree. For NeXus data read from a file, this will be a group of class NXroot, but if the NeXus tree was defined interactively, it can be any valid NXgroup. nxroot : NXgroup The root object of the NeXus tree containing this object. For NeXus data read from a file, this will be a group of class NXroot, but if the NeXus tree was defined interactively, it can be any valid NXgroup. NeXus Attributes ---------------- NeXus attributes are stored in the 'attrs' dictionary of the NXfield, but can usually be assigned or referenced as if they are Python attributes, as long as the attribute name is not the same as one of those listed above. This is to simplify typing in an interactive session and should not cause any problems because there is no name clash with attributes so far defined within the NeXus standard. When writing modules, it is recommended that the attributes always be referenced using the 'attrs' dictionary if there is any doubt. a) Assigning a NeXus attribute In the example below, after assigning the NXfield, the following three NeXus attribute assignments are all equivalent: >>> entry.sample.temperature = NXfield(40.0) >>> entry.sample.temperature.attrs['units'] = 'K' >>> entry.sample.temperature.units = NXattr('K') >>> entry.sample.temperature.units = 'K' b) Referencing a NeXus attribute If the name of the NeXus attribute is not the same as any of the Python attributes listed above, or one of the methods listed below, or any of the attributes defined for Numpy arrays, they can be referenced as if they were a Python attribute of the NXfield. However, it is only possible to reference attributes with one of the proscribed names using the 'attrs' dictionary. >>> entry.sample.temperature.tree = 10.0 >>> entry.sample.temperature.tree temperature = 40.0 @tree = 10.0 @units = K >>> entry.sample.temperature.attrs['tree'] NXattr(10.0) Numerical Operations on NXfields -------------------------------- NXfields usually consist of arrays of numeric data with associated meta-data, the NeXus attributes. The exception is when they contain character strings. This makes them similar to Numpy arrays, and this module allows the use of NXfields in numerical operations in the same way as Numpy ndarrays. NXfields are technically not a sub-class of the ndarray class, but most Numpy operations work on NXfields, returning either another NXfield or, in some cases, an ndarray that can easily be converted to an NXfield. >>> x = NXfield((1.0,2.0,3.0,4.0)) >>> print x+1 [ 2. 3. 4. 5.] >>> print 2*x [ 2. 4. 6. 8.] >>> print x/2 [ 0.5 1. 1.5 2. ] >>> print x**2 [ 1. 4. 9. 16.] >>> print x.reshape((2,2)) [[ 1. 2.] [ 3. 4.]] >>> y = NXfield((0.5,1.5,2.5,3.5)) >>> x+y NXfield(name=x,value=[ 1.5 3.5 5.5 7.5]) >>> x*y NXfield(name=x,value=[ 0.5 3. 7.5 14. ]) >>> (x+y).shape (4,) >>> (x+y).dtype dtype('float64') All these operations return valid NXfield objects containing the same attributes as the first NXobject in the expression. The 'reshape' and 'transpose' methods also return NXfield objects. It is possible to use the standard slice syntax. >>> x=NXfield(np.linspace(0,10,11)) >>> x NXfield([ 0. 1. 2. ..., 8. 9. 10.]) >>> x[2:5] NXfield([ 2. 3. 4.]) In addition, it is possible to use floating point numbers as the slice indices. If one of the indices is not integer, both indices are used to extract elements in the array with values between the two index values. >>> x=NXfield(np.linspace(0,100.,11)) >>> x NXfield([ 0. 10. 20. ..., 80. 90. 100.]) >>> x[20.:50.] NXfield([ 20. 30. 40. 50.]) The standard Numpy ndarray attributes and methods will also work with NXfields, but will return scalars or Numpy arrays. >>> x.size 4 >>> x.sum() 10.0 >>> x.max() 4.0 >>> x.mean() 2.5 >>> x.var() 1.25 >>> x.reshape((2,2)).sum(1) array([ 3., 7.]) Finally, NXfields are cast as ndarrays for operations that require them. The returned value will be the same as for the equivalent ndarray operation, e.g., >>> np.sin(x) array([ 0.84147098, 0.90929743, 0.14112001, -0.7568025 ]) >>> np.sqrt(x) array([ 1. , 1.41421356, 1.73205081, 2. ]) Methods ------- dir(self, attrs=False): Print the NXfield specification. This outputs the name, dimensions and data type of the NXfield. If 'attrs' is True, NXfield attributes are displayed. tree: Return the NXfield's tree. It invokes the 'dir' method with both 'attrs' and 'recursive' set to True. Note that this is defined as a property attribute and does not require parentheses. save(self, filename, format='w5') Save the NXfield into a file wrapped in a NXroot group and NXentry group with default names. This is equivalent to >>> NXroot(NXentry(NXfield(...))).save(filename) Examples -------- >>> x = NXfield(np.linspace(0,2*np.pi,101), units='degree') >>> phi = x.nxdata_as(units='radian') >>> y = NXfield(np.sin(phi)) # Read a Ni x Nj x Nk array one vector at a time >>> with root.NXentry[0].data.data as slab: Ni,Nj,Nk = slab.shape size = [1,1,Nk] for i in range(Ni): for j in range(Nj): value = slab.get([i,j,0],size) """ def __init__(self, value=None, name='field', dtype=None, shape=(), group=None, attrs={}, **attr): if isinstance(value, list) or isinstance(value, tuple): value = np.array(value) self._value = value self._class = 'NXfield' self._name = name.replace(' ','_') self._group = group self._dtype = dtype if dtype == 'char': self._dtype = 'char' elif dtype in np.typeDict: self._dtype = np.dtype(dtype) elif dtype: raise NeXusError("Invalid data type: %s" % dtype) self._shape = tuple(shape) # Append extra keywords to the attribute list self._attrs = AttrDict() for key in attr.keys(): attrs[key] = attr[key] # Convert NeXus attributes to python attributes self._setattrs(attrs) if 'units' in attrs: units = attrs['units'] else: units = None self._incontext = False del attrs if value is not None and dtype == 'char': value = str(value) self._setdata(value) self._saved = False self._changed = True def __repr__(self): if self._value is not None: if str(self.dtype) == 'char': return "NXfield('%s')" % str(self) else: return "NXfield(%s)" % self._str_value() else: return "NXfield(dtype=%s,shape=%s)" % (self.dtype,self.shape) def __getattr__(self, name): """ Enable standard numpy ndarray attributes if not otherwise defined. """ if name in _npattrs: return self.nxdata.__getattribute__(name) elif name in self.attrs: return self.attrs[name].nxdata raise KeyError(name+" not in "+self.nxname) def __setattr__(self, name, value): """ Add an attribute to the NXfield 'attrs' dictionary unless the attribute name starts with 'nx' or '_', or unless it is one of the standard Python attributes for the NXfield class. """ if name.startswith('_') or name.startswith('nx'): object.__setattr__(self, name, value) elif isinstance(value, NXattr): self._attrs[name] = value self._saved = False self._changed = True else: self._attrs[name] = NXattr(value) self._saved = False self._changed = True def __getitem__(self, index): """ Returns a slice from the NXfield. In most cases, the slice values are applied to the NXfield nxdata array and returned within an NXfield object with the same metadata. However, if the array is one-dimensional and the index start and stop values are real, the nxdata array is returned with values between those limits. This is to allow axis arrays to be limited by their actual value. This real-space slicing should only be used on monotonically increasing (or decreasing) one-dimensional arrays. """ if isinstance(index, slice) and \ (isinstance(index.start, float) or isinstance(index.stop, float)): index = slice(self.index(index.start), self.index(index.stop,max=True)+1) if self._value is not None: result = self.nxdata.__getitem__(index) else: offset = np.zeros(len(self.shape),dtype=int) size = np.array(self.shape) if isinstance(index, int): offset[0] = index size[0] = 1 else: if isinstance(index, slice): index = [index] i = 0 for ind in index: if isinstance(ind, int): offset[i] = ind size[i] = 1 else: if ind.start: offset[i] = ind.start if ind.stop: size[i] = ind.stop - offset[i] i = i + 1 try: result = self.get(offset, size) except ValueError: result = self.nxdata.__getitem__(index) return NXfield(result, name=self.nxname, attrs=self.attrs) def __setitem__(self, index, value): """ Assign a slice to the NXfield. """ if self._value is not None: self.nxdata[index] = value self._saved = False self._changed = True else: raise NeXusError("NXfield dataspace not yet allocated") def __deepcopy__(self, memo): dpcpy = self.__class__() memo[id(self)] = dpcpy dpcpy._value = copy(self._value) dpcpy._name = copy(self.nxname) dpcpy._dtype = copy(self.dtype) dpcpy._shape = copy(self.shape) for k, v in self.attrs.items(): dpcpy.attrs[k] = copy(v) return dpcpy def __len__(self): """ Return the length of the NXfield data. """ return np.prod(self.shape) def index(self, value, max=False): """ Return the index of the NXfield nxdata array that is greater than or equal to the value. If max, then return the index that is less than or equal to the value. This should only be used on one-dimensional monotonically increasing arrays. """ if max: return len(self.nxdata)-len(self.nxdata[self.nxdata>=value]) else: return len(self.nxdata[self.nxdata 10000: # Compress the fastest moving dimension of large datasets slab_dims = np.ones(len(shape),'i') if shape[-1] < 100000: slab_dims[-1] = shape[-1] else: slab_dims[-1] = 100000 path.compmakedata(self.nxname, self.dtype, shape, 'lzw', slab_dims) else: # Don't use compression for small datasets path.makedata(self.nxname, self.dtype, shape) self._infile = True if not self.saved: with self as path: path._writeattrs(self.attrs) value = self.nxdata if value is not None: path.putdata(value) self._saved = True else: raise IOError("Data is not attached to a file") def get(self, offset, size): """ Return a slab from the data array. Offsets are 0-origin. Shape can be inferred from the data. Offset and shape must each have one entry per dimension. Corresponds to NXgetslab(handle,data,offset,shape) """ if self.nxfile: with self as path: value = path.getslab(offset,size) return value else: raise IOError("Data is not attached to a file") def put(self, data, offset, refresh=True): """ Put a slab into the data array. Offsets are 0-origin. Shape can be inferred from the data. Offset and shape must each have one entry per dimension. Corresponds to NXputslab(handle,data,offset,shape) """ if self.nxfile: if self.nxfile.mode == napi.ACC_READ: raise NeXusError("NeXus file is readonly") with self as path: if isinstance(data, NXfield): path.putslab(data.nxdata.astype(self.dtype), offset, data.shape) else: data = np.array(data) path.putslab(data.astype(self.dtype), offset, data.shape) if refresh: self.read() else: raise IOError("Data is not attached to a file") def add(self, data, offset, refresh=True): """ Add a slab into the data array. Calls get to read in existing data before adding the value and calling put. It assumes that the two sets of data have compatible data types. """ if isinstance(data, NXfield): value = self.get(offset, data.shape) self.put(data.nxdata.astype(self.dtype)+value, offset) else: value = self.get(offset, data.shape) self.put(data.astype(self.dtype)+value, offset) if refresh: self.refresh() def refresh(self): """ Rereads the data from the file. If put has been called, then nxdata is no longer synchronized with the file making a refresh necessary. This will only be performed if nxdata already stores the data. """ if self._value is not None: if self.nxfile: self._value = self.nxfile.readpath(self.nxpath) self._infile = self._saved = True else: raise IOError("Data is not attached to a file") def convert(self, units=""): """ Return the data in particular units. """ try: import units except ImportError: raise NeXusError("No conversion utility available") if self._value is not None: return self._converter(self._value,units) else: return None def __str__(self): """ If value is loaded, return the value as a string. If value is not loaded, return the empty string. Only the first view values for large arrays will be printed. """ if self._value is not None: return str(self._value) return "" def _str_value(self,indent=0): v = str(self) if '\n' in v: v = '\n'.join([(" "*indent)+s for s in v.split('\n')]) return v def _str_tree(self,indent=0,attrs=False,recursive=False): dims = 'x'.join([str(n) for n in self.shape]) s = str(self) if '\n' in s or s == "": s = "%s(%s)"%(self.dtype, dims) v=[" "*indent + "%s = %s"%(self.nxname, s)] if attrs and self.attrs: v.append(self._str_attrs(indent=indent+2)) return "\n".join(v) def walk(self): yield self def _getaxes(self): """ Return a list of NXfields containing axes. Only works if the NXfield has the 'axes' attribute """ try: return [getattr(self.nxgroup,name) for name in _readaxes(self.axes)] except KeyError: return None def _getdata(self): """ Return the data if it is not larger than NX_MEMORY. """ if self._value is None: if self.nxfile: if str(self.dtype) == 'char': self._value = self.nxfile.readpath(self.nxpath) elif np.prod(self.shape) * np.dtype(self.dtype).itemsize <= NX_MEMORY*1024*1024: self._value = self.nxfile.readpath(self.nxpath) else: raise MemoryError('Data size larger than NX_MEMORY=%s MB' % NX_MEMORY) self._saved = True else: return None return self._value def _setdata(self, value): if value is not None: if str(self._dtype) == 'char' or isinstance(value,str): self._value = str(value) self._shape = (len(self._value),) self._dtype = 'char' else: if self.dtype in np.typeDict: self._value = np.array(value,self.dtype) else: self._value = np.array(value) self._shape = self._value.shape self._dtype = self._value.dtype self._saved = False self._changed = True def _getdtype(self): return self._dtype def _getshape(self): return self._shape def _getsize(self): return len(self) nxdata = property(_getdata,_setdata,doc="The data values") nxaxes = property(_getaxes,doc="The plotting axes") dtype = property(_getdtype,doc="Data type of NeXus field") shape = property(_getshape,doc="Shape of NeXus field") size = property(_getsize,doc="Size of NeXus field") SDS = NXfield # For backward compatibility def _fixaxes(signal, axes): """ Remove length-one dimensions from plottable data """ shape = list(signal.shape) while 1 in shape: shape.remove(1) newaxes = [] for axis in axes: if axis.size > 1: newaxes.append(axis) return signal.nxdata.view().reshape(shape), newaxes class PylabPlotter(object): """ Matplotlib plotter class for NeXus data. """ def plot(self, signal, axes, title, errors, fmt, xmin, xmax, ymin, ymax, zmin, zmax, **opts): """ Plot the data entry. Raises NeXusError if the data cannot be plotted. """ try: import matplotlib.pyplot as plt except ImportError: raise NeXusError("Default plotting package (matplotlib) not available.") over = False if "over" in opts.keys(): if opts["over"]: over = True del opts["over"] log = logx = logy = False if "log" in opts.keys(): if opts["log"]: log = True del opts["log"] if "logy" in opts.keys(): if opts["logy"]: logy = True del opts["logy"] if "logx" in opts.keys(): if opts["logx"]: logx = True del opts["logx"] if over: plt.autoscale(enable=False) else: plt.autoscale(enable=True) plt.clf() # Provide a new view of the data if there is a dimension of length 1 if 1 in signal.shape: data, axes = _fixaxes(signal, axes) else: data = signal.nxdata # Find the centers of the bins for histogrammed data axis_data = centers(data, axes) #One-dimensional Plot if len(data.shape) == 1: plt.ioff() if hasattr(signal, 'units'): if not errors and signal.units == 'counts': errors = NXfield(np.sqrt(data)) if errors: ebars = errors.nxdata plt.errorbar(axis_data[0], data, ebars, fmt=fmt, **opts) else: plt.plot(axis_data[0], data, fmt, **opts) if not over: ax = plt.gca() xlo, xhi = ax.set_xlim(auto=True) ylo, yhi = ax.set_ylim(auto=True) if xmin: xlo = xmin if xmax: xhi = xmax ax.set_xlim(xlo, xhi) if ymin: ylo = ymin if ymax: yhi = ymax ax.set_ylim(ylo, yhi) if logx: ax.set_xscale('symlog') if log or logy: ax.set_yscale('symlog') plt.xlabel(label(axes[0])) plt.ylabel(label(signal)) plt.title(title) plt.ion() #Two dimensional plot else: from matplotlib.image import NonUniformImage from matplotlib.colors import LogNorm if len(data.shape) > 2: slab = [slice(None), slice(None)] for _dim in data.shape[2:]: slab.append(0) data = data[slab].view().reshape(data.shape[:2]) print "Warning: Only the top 2D slice of the data is plotted" x = axis_data[0] y = axis_data[1] if not zmin: zmin = np.min(data) if not zmax: zmax = np.max(data) z = np.clip(data,zmin,zmax).T if log: opts["norm"] = LogNorm() if z.min() <= 0 and np.issubdtype(z[0,0],int): z = np.clip(z,0.1,zmax) ax = plt.gca() extent = (x[0],x[-1],y[0],y[-1]) im = NonUniformImage(ax, extent=extent, origin=None, **opts) im.set_data(x,y,z) ax.images.append(im) xlo, xhi = ax.set_xlim(x[0],x[-1]) ylo, yhi = ax.set_ylim(y[0],y[-1]) if xmin: xlo = xmin else: xlo = x[0] if xmax: xhi = xmax else: xhi = x[-1] if ymin: yhi = ymin else: yhi = y[0] if ymax: yhi = ymax else: yhi = y[-1] ax.set_xlim(xlo, xhi) ax.set_ylim(ylo, yhi) plt.xlabel(label(axes[0])) plt.ylabel(label(axes[1])) plt.title(title) plt.colorbar(im) plt.gcf().canvas.draw_idle() @staticmethod def show(): import matplotlib.pyplot as plt plt.show() class NXgroup(NXobject): """ A NeXus group object. This is a subclass of NXobject and is the base class for the specific NeXus group classes, e.g., NXentry, NXsample, NXdata. NXgroup(*items, **opts) Parameters ---------- The NXgroup parameters consist of a list of positional and/or keyword arguments. Positional Arguments : These must be valid NeXus objects, either an NXfield or a NeXus group. These are added without modification as children of this group. Keyword Arguments : Apart from a list of special keywords shown below, keyword arguments are used to add children to the group using the keywords as attribute names. The values can either be valid NXfields or NXgroups, in which case the 'name' attribute is changed to the keyword, or they can be numerical or string data, which are converted to NXfield objects. Special Keyword Arguments: name : string The name of the NXgroup, which is directly accessible as the NXgroup attribute 'name'. If the NXgroup is initialized as the attribute of a parent group, the name is automatically set to the name of this attribute. If 'nxclass' is specified and has the usual prefix 'NX', the default name is the class name without this prefix. nxclass : string The class of the NXgroup. entries : dict A dictionary containing a list of group entries. This is an alternative way of adding group entries to the use of keyword arguments. file : filename The file from which the NXfield has been read. path : string The path to this object with respect to the root of the NeXus tree, using the convention for unix file paths. group : NXobject (NXgroup or subclass of NXgroup) The parent NeXus group, which is accessible as the group attribute 'group'. If the group is initialized as the attribute of a parent group, this is set to the parent group. Python Attributes ----------------- nxclass : string The class of the NXobject. nxname : string The name of the NXfield. entries : dictionary A dictionary of all the NeXus objects contained within an NXgroup. attrs : dictionary A dictionary of all the NeXus attributes, i.e., attribute with class NXattr. entries : dictionary A dictionary of all the NeXus objects contained within the group. attrs : dictionary A dictionary of all the group's NeXus attributes, which all have the class NXattr. nxpath : string The path to this object with respect to the root of the NeXus tree. For NeXus data read from a file, this will be a group of class NXroot, but if the NeXus tree was defined interactively, it can be any valid NXgroup. nxroot : NXgroup The root object of the NeXus tree containing this object. For NeXus data read from a file, this will be a group of class NXroot, but if the NeXus tree was defined interactively, it can be any valid NXgroup. NeXus Group Entries ------------------- Just as in a NeXus file, NeXus groups can contain either data or other groups, represented by NXfield and NXgroup objects respectively. To distinguish them from regular Python attributes, all NeXus objects are stored in the 'entries' dictionary of the NXgroup. However, they can usually be assigned or referenced as if they are Python attributes, i.e., using the dictionary name directly as the group attribute name, as long as this name is not the same as one of the Python attributes defined above or as one of the NXfield Python attributes. a) Assigning a NeXus object to a NeXus group In the example below, after assigning the NXgroup, the following three NeXus object assignments to entry.sample are all equivalent: >>> entry.sample = NXsample() >>> entry.sample['temperature'] = NXfield(40.0) >>> entry.sample.temperature = NXfield(40.0) >>> entry.sample.temperature = 40.0 >>> entry.sample.temperature NXfield(40.0) If the assigned value is not a valid NXobject, then it is cast as an NXfield with a type determined from the Python data type. >>> entry.sample.temperature = 40.0 >>> entry.sample.temperature NXfield(40.0) >>> entry.data.data.x=np.linspace(0,10,11).astype('float32') >>> entry.data.data.x NXfield([ 0. 1. 2. ..., 8. 9. 10.]) b) Referencing a NeXus object in a NeXus group If the name of the NeXus object is not the same as any of the Python attributes listed above, or the methods listed below, they can be referenced as if they were a Python attribute of the NXgroup. However, it is only possible to reference attributes with one of the proscribed names using the group dictionary, i.e., >>> entry.sample.tree = 100.0 >>> print entry.sample.tree sample:NXsample tree = 100.0 >>> entry.sample['tree'] NXfield(100.0) For this reason, it is recommended to use the group dictionary to reference all group objects within Python scripts. NeXus Attributes ---------------- NeXus attributes are not currently used much with NXgroups, except for the root group, which has a number of global attributes to store the file name, file creation time, and NeXus and HDF version numbers. However, the mechanism described for NXfields works here as well. All NeXus attributes are stored in the 'attrs' dictionary of the NXgroup, but can be referenced as if they are Python attributes as long as there is no name clash. >>> entry.sample.temperature = 40.0 >>> entry.sample.attrs['tree'] = 10.0 >>> print entry.sample.tree sample:NXsample @tree = 10.0 temperature = 40.0 >>> entry.sample.attrs['tree'] NXattr(10.0) Methods ------- insert(self, NXobject, name='unknown'): Insert a valid NXobject (NXfield or NXgroup) into the group. If NXobject has a 'name' attribute and the 'name' keyword is not given, then the object is inserted with the NXobject name. makelink(self, NXobject): Add the NXobject to the group entries as a link (NXlink). dir(self, attrs=False, recursive=False): Print the group directory. The directory is a list of NeXus objects within this group, either NeXus groups or NXfield data. If 'attrs' is True, NXfield attributes are displayed. If 'recursive' is True, the contents of child groups are also displayed. tree: Return the group tree. It invokes the 'dir' method with both 'attrs' and 'recursive' set to True. save(self, filename, format='w5') Save the NeXus group into a file The object is wrapped in an NXroot group (with name 'root') and an NXentry group (with name 'entry'), if necessary, in order to produce a valid NeXus file. Examples -------- >>> x = NXfield(np.linspace(0,2*np.pi,101), units='degree') >>> entry = NXgroup(x, name='entry', nxclass='NXentry') >>> entry.sample = NXgroup(temperature=NXfield(40.0,units='K'), nxclass='NXsample') >>> print entry.sample.tree sample:NXsample temperature = 40.0 @units = K Note: All the currently defined NeXus classes are defined as subclasses of the NXgroup class. It is recommended that these are used directly, so that the above examples become: >>> entry = NXentry(x) >>> entry.sample = NXsample(temperature=NXfield(40.0,units='K')) or >>> entry.sample.temperature = 40.0 >>> entry.sample.temperature.units='K' """ # Plotter to use for plot calls _plotter = PylabPlotter() def __init__(self, *items, **opts): if "name" in opts.keys(): self._name = opts["name"].replace(' ','_') del opts["name"] self._entries = {} if "entries" in opts.keys(): for k,v in opts["entries"].items(): setattr(self, k, v) del opts["entries"] self._attrs = AttrDict() if "attrs" in opts.keys(): self._setattrs(opts["attrs"]) del opts["attrs"] if "nxclass" in opts.keys(): self._class = opts["nxclass"] del opts["nxclass"] if "group" in opts.keys(): self._group = opts["group"] del opts["group"] for k,v in opts.items(): setattr(self, k, v) if self.nxclass.startswith("NX"): if self.nxname == "unknown": self._name = self.nxclass[2:] try: # If one exists, set the class to a valid NXgroup subclass self.__class__ = globals()[self.nxclass] except KeyError: pass for item in items: try: setattr(self, item.nxname, item) except AttributeError: raise NeXusError("Non-keyword arguments must be valid NXobjects") self._saved = False self._changed = True # def __cmp__(self, other): # """Sort groups by their distances or names.""" # try: # return cmp(self.distance, other.distance) # except KeyError: # return cmp(self.nxname, other.nxname) def __repr__(self): return "%s('%s')" % (self.__class__.__name__,self.nxname) def _str_value(self,indent=0): return "" def walk(self): yield self for node in self.entries.values(): for child in node.walk(): yield child def __getattr__(self, key): """ Provide direct access to groups via nxclass name. """ if key.startswith('NX'): return self.component(key) elif key in self.entries: return self.entries[key] elif key in self.attrs: return self.attrs[key].nxdata raise KeyError(key+" not in "+self.nxclass+":"+self.nxname) def __setattr__(self, name, value): """ Set an attribute as an object or regular Python attribute. It is assumed that attributes starting with 'nx' or '_' are regular Python attributes. All other attributes are converted to valid NXobjects, with class NXfield, NXgroup, or a sub-class of NXgroup, depending on the assigned value. The internal value of the attribute name, i.e., 'name', is set to the attribute name used in the assignment. The parent group of the attribute, i.e., 'group', is set to the parent group of the attribute. If the assigned value is a numerical (scalar or array) or string object, it is converted to an object of class NXfield, whose attribute, 'nxdata', is set to the assigned value. """ if name.startswith('_') or name.startswith('nx'): object.__setattr__(self, name, value) elif isinstance(value, NXattr): self._attrs[name] = value self._saved = False self._changed = True else: self[name] = value def __getitem__(self, index): """ Returns a slice from the NXgroup nxsignal attribute (if it exists) as a new NXdata group, if the index is a slice object. In most cases, the slice values are applied to the NXfield nxdata array and returned within an NXfield object with the same metadata. However, if the array is one-dimensional and the index start and stop values are real, the nxdata array is returned with values between the limits set by those axis values. This is to allow axis arrays to be limited by their actual value. This real-space slicing should only be used on monotonically increasing (or decreasing) one-dimensional arrays. """ if isinstance(index, str): #i.e., requesting a dictionary value return self._entries[index] if not self.nxsignal: raise NeXusError("No plottable signal") if not hasattr(self,"nxclass"): raise NeXusError("Indexing not allowed for groups of unknown class") if isinstance(index, int): axes = self.nxaxes axes[0] = axes[0][index] result = NXdata(self.nxsignal[index], axes) if self.nxerrors: result.errors = self.errors[index] elif isinstance(index, slice): axes = self.nxaxes axes[0] = axes[0][index] if isinstance(index.start, float) or isinstance(index.stop, float): index = slice(self.nxaxes[0].index(index.start), self.nxaxes[0].index(index.stop,max=True)+1) result = NXdata(self.nxsignal[index], axes) if self.nxerrors: result.errors = self.errors[index] else: result = NXdata(self.nxsignal[index], axes) if self.nxerrors: result.errors = self.errors[index] else: i = 0 slices = [] axes = self.nxaxes for ind in index: axes[i] = axes[i][ind] if isinstance(ind, slice) and \ (isinstance(ind.start, float) or isinstance(ind.stop, float)): slices.append(slice(self.nxaxes[i].index(ind.start), self.nxaxes[i].index(ind.stop))) else: slices.append(ind) i = i + 1 result = NXdata(self.nxsignal.__getitem__(tuple(slices)), axes) if self.nxerrors: result.errors = self.errors.__getitem__(tuple(slices)) axes = [] for axis in result.nxaxes: if len(axis) > 1: axes.append(axis) result.nxsignal.axes = ":".join([axis.nxname for axis in axes]) if self.nxtitle: result.title = self.nxtitle return result def __setitem__(self, key, value): """ Adds or modifies an item in the NeXus group. """ if key in self.entries: infile = self._entries[key]._infile if isinstance(self._entries[key], NXlink): if self._entries[key].nxlink: setattr(self._entries[key].nxlink.nxgroup, key, value) return attrs = self._entries[key].attrs else: infile = None attrs = {} if isinstance(value, NXlink): self._entries[key] = value elif isinstance(value, NXobject): if value.nxgroup is not None: memo = {} value = deepcopy(value, memo) value._attrs = copy(value._attrs) value._group = self value._name = key self._entries[key] = value else: self._entries[key] = NXfield(value=value, name=key, group=self, attrs=attrs) if infile is not None: self[key]._infile = infile self._changed = True def __deepcopy__(self, memo): dpcpy = self.__class__() memo[id(self)] = dpcpy for k,v in self.items(): if isinstance(v, NXgroup): dpcpy[k] = deepcopy(v, memo) else: dpcpy[k] = copy(v) for k, v in self.attrs.items(): dpcpy.attrs[k] = copy(v) return dpcpy def keys(self): """ Returns the names of NeXus objects in the group. """ return self._entries.keys() def values(self): """ Returns the values of NeXus objects in the group. """ return self._entries.values() def items(self): """ Returns a list of the NeXus objects in the group as (key,value) pairs. """ return self._entries.items() def has_key(self, name): """ Returns true if the NeXus object with the specified name is in the group. """ return self._entries.has_key(name) def insert(self, value, name='unknown'): """ Adds an attribute to the group. If it is not a valid NeXus object (NXfield or NXgroup), the attribute is converted to an NXfield. """ if isinstance(value, NXobject): if name == 'unknown': name = value.nxname if name in self._entries: raise NeXusError("'%s' already exists in group" % name) value._group = self self._entries[name] = value else: self._entries[name] = NXfield(value=value, name=name, group=self) def makelink(self, target): """ Creates a linked NXobject within the group. All attributes are inherited from the parent object including the name """ if isinstance(target, NXobject): self[target.nxname] = NXlink(target=target, group=self) else: raise NeXusError("Link target must be an NXobject") def read(self): """ Read the NXgroup and all its children from the NeXus file. """ if self.nxfile: with self as path: n, nxname, nxclass = path.getgroupinfo() if nxclass != self.nxclass: raise NeXusError("The NeXus group class does not match the file") self._setattrs(path.getattrs()) entries = path.entries() for name,nxclass in entries: path = self.nxpath + '/' + name if nxclass == 'SDS': attrs = self.nxfile.getattrs() if 'target' in attrs and attrs['target'] != path: self._entries[name] = NXlinkfield(target=attrs['target']) else: self._entries[name] = NXfield(name=name) else: attrs = self.nxfile.getattrs() if 'target' in attrs and attrs['target'] != path: self._entries[name] = NXlinkgroup(name=name, target=attrs['target']) else: self._entries[name] = NXgroup(nxclass=nxclass) self._entries[name]._group = self #Make sure non-linked variables are processed first. for entry in self._entries.values(): for node in entry.walk(): if not isinstance(node, NXlink): node.read() for entry in self._entries.values(): for node in entry.walk(): if isinstance(node, NXlink): node.read() self._infile = self._saved = self._changed = True else: raise IOError("Data is not attached to a file") def write(self): """ Write the NXgroup, including its children, to the NeXus file. """ if self.nxfile: if self.nxfile.mode == napi.ACC_READ: raise NeXusError("NeXus file is readonly") if not self.infile: with self.nxgroup as path: path.makegroup(self.nxname, self.nxclass) self._infile = True with self as path: path._writeattrs(self.attrs) for entry in self.walk(): if entry is not self: entry.write() self._infile = self._saved = True else: raise IOError("Group is not attached to a file") def sum(self, axis=None): """ Return the sum of the NXdata group using the Numpy sum method on the NXdata signal. The result contains a copy of all the metadata contained in the NXdata group. """ if not self.nxsignal: raise NeXusError("No signal to sum") if not hasattr(self,"nxclass"): raise NeXusError("Summing not allowed for groups of unknown class") if axis is None: return self.nxsignal.sum() else: signal = NXfield(self.nxsignal.sum(axis), name=self.nxsignal.nxname) axes = self.nxaxes summedaxis = axes.pop(axis) units = "" if hasattr(summedaxis, "units"): units = summedaxis.units signal.long_name = "Integral from %s to %s %s" % \ (summedaxis[0], summedaxis[-1], units) average = NXfield(0.5*(summedaxis.nxdata[0]+summedaxis.nxdata[-1]), name=summedaxis.nxname) if units: average.units = units result = NXdata(signal, axes, average) if self.nxerrors: errors = np.sqrt((self.nxerrors.nxdata**2).sum(axis)) result.errors = NXfield(errors, name="errors") if self.nxtitle: result.title = self.nxtitle return result def moment(self, order=1): """ Return an NXfield containing the moments of the NXdata group assuming the signal is one-dimensional. Currently, only the first moment has been defined. Eventually, the order of the moment will be defined by the 'order' parameter. """ if not self.nxsignal: raise NeXusError("No signal to calculate") elif len(self.nxsignal.shape) > 1: raise NeXusError("Operation only possible on one-dimensional signals") elif order > 1: raise NeXusError("Higher moments not yet implemented") if not hasattr(self,"nxclass"): raise NeXusError("Operation not allowed for groups of unknown class") return (centers(self.nxsignal,self.nxaxes)*self.nxsignal).sum() \ /self.nxsignal.sum() def component(self, nxclass): """ Find all child objects that have a particular class. """ return [E for _name,E in self.entries.items() if E.nxclass==nxclass] def signals(self): """ Return a dictionary of NXfield's containing signal data. The key is the value of the signal attribute. """ signals = {} for obj in self.entries.values(): if 'signal' in obj.attrs: signals[obj.nxsignal.nxdata] = obj return signals def _signal(self): """ Return the NXfield containing the signal data. """ for obj in self.entries.values(): if 'signal' in obj.attrs and str(obj.signal) == '1': # if isinstance(self[obj.nxname],NXlink): # return self[obj.nxname].nxlink # else: return self[obj.nxname] return None def _set_signal(self, signal): """ Setter for the signal attribute. The argument should be a valid NXfield within the group. """ self[signal.nxname].signal = NXattr(1) def _axes(self): """ Return a list of NXfields containing the axes. """ try: return [getattr(self,name) for name in _readaxes(self.nxsignal.axes)] except KeyError: axes = {} for obj in self.entries: if 'axis' in getattr(self,obj).attrs: axes[getattr(self,obj).axis] = getattr(self,obj) return [axes[key] for key in sorted(axes.keys())] def _set_axes(self, axes): """ Setter for the signal attribute. The argument should be a list of valid NXfields within the group. """ if not isinstance(axes, list): axes = [axes] self.nxsignal.axes = NXattr(":".join([axis.nxname for axis in axes])) def _errors(self): """ Return the NXfield containing the signal errors. """ try: return self.entries['errors'] except KeyError: return None def _title(self): """ Return the title as a string. If there is no title attribute in the string, the parent NXentry group in the group's path is searched. """ title = self.nxpath if 'title' in self.entries: return str(self.title) elif self.nxgroup: if 'title' in self.nxgroup.entries: return str(self.nxgroup.title) return self.nxpath def _getentries(self): return self._entries nxsignal = property(_signal, _set_signal, "Signal NXfield within group") nxaxes = property(_axes, _set_axes, "List of axes within group") nxerrors = property(_errors, "Errors NXfield within group") nxtitle = property(_title, "Title for group plot") entries = property(_getentries,doc="NeXus objects within group") def plot(self, fmt='bo', xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, **opts): """ Plot data contained within the group. The format argument is used to set the color and type of the markers or lines for one-dimensional plots, using the standard matplotlib syntax. The default is set to blue circles. All keyword arguments accepted by matplotlib.pyplot.plot can be used to customize the plot. In addition to the matplotlib keyword arguments, the following are defined: log = True - plot the intensity on a log scale logy = True - plot the y-axis on a log scale logx = True - plot the x-axis on a log scale over = True - plot on the current figure Raises NeXusError if the data could not be plotted. """ group = self if self.nxclass == "NXroot": group = group.NXentry[0] if group.nxclass == "NXentry": try: group = group.NXdata[0] except IndexError: raise NeXusError('No NXdata group found') # Find a plottable signal signal = group.nxsignal if not signal: raise NeXusError('No plottable signal defined') # Find errors errors= group.nxerrors # Find the associated axes axes = group.nxaxes # Construct title title = group.nxtitle # Plot with the available plotter group._plotter.plot(signal, axes, title, errors, fmt, xmin, xmax, ymin, ymax, zmin, zmax, **opts) def oplot(self, fmt='bo', **opts): """ Plot the data contained within the group over the current figure. """ self.plot(fmt=fmt, over=True, **opts) def logplot(self, fmt='bo', xmin=None, xmax=None, ymin=None, ymax=None, zmin=None, zmax=None, **opts): """ Plot the data intensity contained within the group on a log scale. """ self.plot(fmt=fmt, log=True, xmin=xmin, xmax=xmax, ymin=ymin, ymax=ymax, zmin=zmin, zmax=zmax, **opts) class NXlink(NXobject): """ Class for NeXus linked objects. The real object will be accessible by following the link attribute. """ _class = "NXlink" def __init__(self, target=None, name='link', group=None): self._group = group self._class = "NXlink" if isinstance(target, NXobject): self._name = target.nxname self._target = target.nxpath self.nxlink.attrs["target"] = target.nxpath if target.nxclass == "NXlink": raise NeXusError("Cannot link to another NXlink object") elif target.nxclass == "NXfield": self.__class__ = NXlinkfield else: self.__class__ = NXlinkgroup else: self._name = name self._target = target def __getattr__(self, key): try: try: return self.nxlink.__dict__[key] except KeyError: return self.nxlink.__getattr__(key) except KeyError: raise KeyError((key+" not in %s" % self._target)) def __setattr__(self, name, value): if name.startswith('_') or name.startswith('nx'): object.__setattr__(self, name, value) elif self.nxlink: self.nxlink.__setattr__(name, value) def __repr__(self): return "NXlink('%s')"%(self._target) def __str__(self): return str(self.nxlink) def _str_tree(self, indent=0, attrs=False, recursive=False): if self.nxlink: return self.nxlink._str_tree(indent, attrs, recursive) else: return " "*indent+self.nxname+' -> '+self._target def _getlink(self): link = self.nxroot if link: try: for level in self._target[1:].split('/'): link = link.entries[level] return link except AttributeError: return None else: return None def _getattrs(self): return self.nxlink.attrs nxlink = property(_getlink, "Linked object") attrs = property(_getattrs,doc="NeXus attributes for object") def read(self): """ Read the linked NXobject. """ self.nxlink.read() self._infile = self._saved = self._changed = True class NXlinkfield(NXlink, NXfield): """ Class for a NeXus linked field. The real field will be accessible by following the link attribute. """ def write(self): """ Write the linked NXfield. """ self.nxlink.write() if not self.infile: with self.nxlink as path: target = path.getdataID() with self.nxgroup as path: path.makelink(target) self._infile = self._saved = True def get(self, offset, size): """ Get a slab from the data array. Offsets are 0-origin. Shape can be inferred from the data. Offset and shape must each have one entry per dimension. This operation should be performed in a "with group.data" conext. Raises ValueError cannot convert units. Corresponds to NXgetslab(handle,data,offset,shape) """ if self.nxfile: with self.nxlink as path: value = path.getslab(offset,size) else: raise IOError("Data is not attached to a file") NXlinkdata = NXlinkfield # For backward compatibility class NXlinkgroup(NXlink, NXgroup): """ Class for a NeXus linked group. The real group will be accessible by following the link attribute. """ def write(self): """ Write the linked NXgroup. """ self.nxlink.write() if not self.infile: with self.nxlink as path: target = path.getgroupID() with self.nxgroup as path: path.makelink(target) self._infile = self._saved = True def _getentries(self): return self.nxlink.entries entries = property(_getentries,doc="NeXus objects within group") class NXentry(NXgroup): """ NXentry group. This is a subclass of the NXgroup class. Each NXdata and NXmonitor object of the same name will be added together, raising an NeXusError if any of the groups do not exist in both NXentry groups or if any of the NXdata additions fail. The resulting NXentry group contains a copy of all the other metadata contained in the first group. Note that other extensible data, such as the run duration, are not currently added together. See the NXgroup documentation for more details. """ def __init__(self, *items, **opts): self._class = "NXentry" NXgroup.__init__(self, *items, **opts) def __add__(self, other): """ Add two NXentry objects """ result = NXentry(entries=self.entries, attrs=self.attrs) try: names = [group.nxname for group in self.component("NXdata")] for name in names: if isinstance(other.entries[name], NXdata): result.entries[name] = self.entries[name] + other.entries[name] else: raise KeyError names = [group.nxname for group in self.component("NXmonitor")] for name in names: if isinstance(other.entries[name], NXmonitor): result.entries[name] = self.entries[name] + other.entries[name] else: raise KeyError return result except KeyError: raise NeXusError("Inconsistency between two NXentry groups") def __sub__(self, other): """ Subtract two NXentry objects """ result = NXentry(entries=self.entries, attrs=self.attrs) try: names = [group.nxname for group in self.component("NXdata")] for name in names: if isinstance(other.entries[name], NXdata): result.entries[name] = self.entries[name] - other.entries[name] else: raise KeyError names = [group.nxname for group in self.component("NXmonitor")] for name in names: if isinstance(other.entries[name], NXmonitor): result.entries[name] = self.entries[name] - other.entries[name] else: raise KeyError return result except KeyError: raise NeXusError("Inconsistency between two NXentry groups") class NXsubentry(NXentry): """ NXsubentry group. This is a subclass of the NXsubentry class. See the NXgroup documentation for more details. """ def __init__(self, *items, **opts): self._class = "NXsubentry" NXgroup.__init__(self, *items, **opts) class NXdata(NXgroup): """ NXdata group. This is a subclass of the NXgroup class. The constructor assumes that the first argument contains the signal and the second contains either the axis, for one-dimensional data, or a list of axes, for multidimensional data. These arguments can either be NXfield objects or Numpy arrays, which are converted to NXfield objects with default names. Alternatively, the signal and axes NXfields can be defined using the 'nxsignal' and 'nxaxes' properties. See the examples below. Various arithmetic operations (addition, subtraction, multiplication, and division) have been defined for combining NXdata groups with other NXdata groups, Numpy arrays, or constants, raising a NeXusError if the shapes don't match. Data errors are propagated in quadrature if they are defined, i.e., if the 'nexerrors' attribute is not None, Attributes ---------- nxsignal : The NXfield containing the attribute 'signal' with value 1 nxaxes : A list of NXfields containing the signal axes nxerrors : The NXfield containing the errors Methods ------- plot(self, fmt, over=False, log=False, logy=False, logx=False, **opts) Plot the NXdata group using the defined signal and axes. Valid Matplotlib parameters, specifying markers, colors, etc, can be specified using format argument or through keyword arguments. logplot(self, fmt, over=False, logy=False, logx=False, **opts) Plot the NXdata group using the defined signal and axes with the intensity plotted on a logarithmic scale. In one-dimensional plots, this is the y-axis. In two-dimensional plots, it is the color scale. oplot(self, fmt, **opts) Plot the NXdata group using the defined signal and axes over the current plot. moment(self, order=1) Calculate moments of the NXdata group. This assumes that the signal is one-dimenional. Currently, only the first moment is implemented. Examples -------- There are three methods of creating valid NXdata groups with the signal and axes NXfields defined according to the NeXus standard. 1) Create the NXdata group with Numpy arrays that will be assigned default names. >>> x = np.linspace(0, 2*np.pi, 101) >>> line = NXdata(sin(x), x) data:NXdata signal = float64(101) @axes = x @signal = 1 axis1 = float64(101) 2) Create the NXdata group with NXfields that have their internal names already assigned. >>> x = NXfield(linspace(0,2*pi,101), name='x') >>> y = NXfield(linspace(0,2*pi,101), name='y') >>> X, Y = np.meshgrid(x, y) >>> z = NXfield(sin(X) * sin(Y), name='z') >>> entry = NXentry() >>> entry.grid = NXdata(z, (x, y)) >>> grid.tree() entry:NXentry grid:NXdata x = float64(101) y = float64(101) z = float64(101x101) @axes = x:y @signal = 1 3) Create the NXdata group with keyword arguments defining the names and set the signal and axes using the nxsignal and nxaxes properties. >>> x = linspace(0,2*pi,101) >>> y = linspace(0,2*pi,101) >>> X, Y = np.meshgrid(x, y) >>> z = sin(X) * sin(Y) >>> entry = NXentry() >>> entry.grid = NXdata(z=sin(X)*sin(Y), x=x, y=y) >>> entry.grid.nxsignal = entry.grid.z >>> entry.grid.nxaxes = [entry.grid.x.entry.grid.y] >>> grid.tree() entry:NXentry grid:NXdata x = float64(101) y = float64(101) z = float64(101x101) @axes = x:y @signal = 1 """ def __init__(self, signal=None, axes=None, *items, **opts): self._class = "NXdata" NXgroup.__init__(self, *items, **opts) if signal is not None: if isinstance(signal,NXfield): if signal.nxname == "unknown": signal.nxname = "signal" if "signal" not in signal.attrs: signal.signal = 1 self[signal.nxname] = signal signalname = signal.nxname else: self["signal"] = signal self["signal"].signal = 1 signalname = "signal" if axes is not None: if not isinstance(axes,tuple) and not isinstance(axes,list): axes = [axes] axisnames = {} i = 0 for axis in axes: i = i + 1 if isinstance(axis,NXfield): if axis._name == "unknown": axis._name = "axis%s" % i self[axis.nxname] = axis axisnames[i] = axis.nxname else: axisname = "axis%s" % i self[axisname] = axis axisnames[i] = axisname self[signalname].axes = ":".join(axisnames.values()) def __add__(self, other): """ Define a method for adding a NXdata group to another NXdata group or to a number. Only the signal data is affected. The result contains a copy of all the metadata contained in the first NXdata group. The module checks that the dimensions are compatible, but does not check that the NXfield names or values are identical. This is so that spelling variations or rounding errors do not make the operation fail. However, it is up to the user to ensure that the results make sense. """ result = NXdata(entries=self.entries, attrs=self.attrs) if isinstance(other, NXdata): if self.nxsignal and self.nxsignal.shape == other.nxsignal.shape: result.entries[self.nxsignal.nxname] = self.nxsignal + other.nxsignal if self.nxerrors: if other.nxerrors: result.errors = np.sqrt(self.errors.nxdata**2+other.errors.nxdata**2) else: result.errors = self.errors return result elif isinstance(other, NXgroup): raise NeXusError("Cannot add two arbitrary groups") else: result.entries[self.nxsignal.nxname] = self.nxsignal + other result.entries[self.nxsignal.nxname].nxname = self.nxsignal.nxname return result def __sub__(self, other): """ Define a method for subtracting a NXdata group or a number from the NXdata group. Only the signal data is affected. The result contains a copy of all the metadata contained in the first NXdata group. The module checks that the dimensions are compatible, but does not check that the NXfield names or values are identical. This is so that spelling variations or rounding errors do not make the operation fail. However, it is up to the user to ensure that the results make sense. """ result = NXdata(entries=self.entries, attrs=self.attrs) if isinstance(other, NXdata): if self.nxsignal and self.nxsignal.shape == other.nxsignal.shape: result.entries[self.nxsignal.nxname] = self.nxsignal - other.nxsignal if self.nxerrors: if other.nxerrors: result.errors = np.sqrt(self.errors.nxdata**2+other.errors.nxdata**2) else: result.errors = self.errors return result elif isinstance(other, NXgroup): raise NeXusError("Cannot subtract two arbitrary groups") else: result.entries[self.nxsignal.nxname] = self.nxsignal - other result.entries[self.nxsignal.nxname].nxname = self.nxsignal.nxname return result def __mul__(self, other): """ Define a method for multiplying the NXdata group with a NXdata group or a number. Only the signal data is affected. The result contains a copy of all the metadata contained in the first NXdata group. The module checks that the dimensions are compatible, but does not check that the NXfield names or values are identical. This is so that spelling variations or rounding errors do not make the operation fail. However, it is up to the user to ensure that the results make sense. """ result = NXdata(entries=self.entries, attrs=self.attrs) if isinstance(other, NXdata): # error here signal not defined in this scope #if self.nxsignal and signal.shape == other.nxsignal.shape: if self.nxsignal and self.nxsignal.shape == other.nxsignal.shape: result.entries[self.nxsignal.nxname] = self.nxsignal * other.nxsignal if self.nxerrors: if other.nxerrors: result.errors = np.sqrt((self.errors.nxdata*other.nxsignal.nxdata)**2+ (other.errors.nxdata*self.nxsignal.nxdata)**2) else: result.errors = self.errors return result elif isinstance(other, NXgroup): raise NeXusError("Cannot multiply two arbitrary groups") else: result.entries[self.nxsignal.nxname] = self.nxsignal * other result.entries[self.nxsignal.nxname].nxname = self.nxsignal.nxname if self.nxerrors: result.errors = self.errors * other return result def __rmul__(self, other): """ Define a method for multiplying NXdata groups. This variant makes __mul__ commutative. """ return self.__mul__(other) def __div__(self, other): """ Define a method for dividing the NXdata group by a NXdata group or a number. Only the signal data is affected. The result contains a copy of all the metadata contained in the first NXdata group. The module checks that the dimensions are compatible, but does not check that the NXfield names or values are identical. This is so that spelling variations or rounding errors do not make the operation fail. However, it is up to the user to ensure that the results make sense. """ result = NXdata(entries=self.entries, attrs=self.attrs) if isinstance(other, NXdata): if self.nxsignal and self.nxsignal.shape == other.nxsignal.shape: # error here, signal and othersignal not defined here #result.entries[self.nxsignal.nxname] = signal / othersignal result.entries[self.nxsignal.nxname] = self.nxsignal / other.nxsignal resultvalues = result.entries[self.nxsignal.nxname].nxdata if self.nxerrors: if other.nxerrors: result.errors = (np.sqrt(self.errors.nxdata**2 + (resultvalues*other.errors.nxdata)**2) / other.nxsignal) else: result.errors = self.errors return result elif isinstance(other, NXgroup): raise NeXusError("Cannot divide two arbitrary groups") else: result.entries[self.nxsignal.nxname] = self.nxsignal / other result.entries[self.nxsignal.nxname].nxname = self.nxsignal.nxname if self.nxerrors: result.errors = self.errors / other return result class NXmonitor(NXdata): """ NXmonitor group. This is a subclass of the NXdata class. See the NXdata and NXgroup documentation for more details. """ def __init__(self, signal=None, axes=(), *items, **opts): NXdata.__init__(self, signal=signal, axes=axes, *items, **opts) self._class = "NXmonitor" if "name" not in opts.keys(): self._name = "monitor" class NXlog(NXgroup): """ NXlog group. This is a subclass of the NXgroup class. Methods ------- plot(self, **opts) Plot the logged values against the elapsed time. Valid Matplotlib parameters, specifying markers, colors, etc, can be specified using the 'opts' dictionary. See the NXgroup documentation for more details. """ def __init__(self, *items, **opts): self._class = "NXlog" NXgroup.__init__(self, *items, **opts) def plot(self, **opts): axis = [self.time] title = "%s Log" % self.value.nxname.upper() self._plotter.plot(self.value, axis, title, **opts) #------------------------------------------------------------------------- #Add remaining base classes as subclasses of NXgroup and append to __all__ for _class in _nxclasses: if _class not in globals(): docstring = """ %s group. This is a subclass of the NXgroup class. See the NXgroup documentation for more details. """ % _class globals()[_class]=type(_class, (NXgroup,), {'_class':_class,'__doc__':docstring}) __all__.append(_class) #------------------------------------------------------------------------- def centers(signal, axes): """ Return the centers of the axes. This works regardless if the axes contain bin boundaries or centers. """ def findc(axis, dimlen): if axis.shape[0] == dimlen+1: return (axis.nxdata[:-1] + axis.nxdata[1:])/2 else: assert axis.shape[0] == dimlen return axis.nxdata return [findc(a,signal.shape[i]) for i,a in enumerate(axes)] def setmemory(value): """ Set the memory limit for data arrays (in MB). """ global NX_MEMORY NX_MEMORY = value def label(field): """ Return a label for a data field suitable for use on a graph axis. """ if hasattr(field,'long_name'): return field.long_name elif hasattr(field,'units'): return "%s (%s)"%(field.nxname,field.units) else: return field.nxname # File level operations def load(filename, mode='r'): """ Read a NeXus file returning a tree of objects. This is aliased to 'read' because of potential name clashes with Numpy """ file = NeXusTree(filename,mode) tree = file.readfile() file.close() return tree #Definition for when there are name clashes with Numpy nxload = load __all__.append('nxload') def save(filename, group, format='w5'): """ Write a NeXus file from a tree of objects. """ if group.nxclass == "NXroot": tree = group elif group.nxclass == "NXentry": tree = NXroot(group) else: tree = NXroot(NXentry(group)) file = NeXusTree(filename, format) file.writefile(tree) file.close() def tree(file): """ Read and summarize the named NeXus file. """ nxfile = load(file) nxfile.tree def demo(argv): """ Process a list of command line commands. 'argv' should contain program name, command, arguments, where command is one of the following: copy fromfile.nxs tofile.nxs ls f1.nxs f2.nxs ... """ if len(argv) > 1: op = argv[1] else: op = 'help' if op == 'ls': for f in argv[2:]: dir(f) elif op == 'copy' and len(argv)==4: tree = load(argv[2]) save(argv[3], tree) elif op == 'plot' and len(argv)==4: tree = load(argv[2]) for entry in argv[3].split('.'): tree = getattr(tree,entry) tree.plot() tree._plotter.show() else: usage = """ usage: %s cmd [args] copy fromfile.nxs tofile.nxs ls *.nxs plot file.nxs entry.data """%(argv[0],) print usage if __name__ == "__main__": import sys demo(sys.argv) python-nxs/nxs/unit.py000066400000000000000000000165741306222103500153550ustar00rootroot00000000000000# This program is public domain # Author: Paul Kienzle """ Define unit conversion support for NeXus style units. The unit format is somewhat complicated. There are variant spellings and incorrect capitalization to worry about, as well as forms such as "mili*metre" and "1e-7 seconds". This is a minimal implementation of units including only what I happen to need now. It does not support the complete dimensional analysis provided by the package udunits on which NeXus is based, or even the units used in the NeXus definition files. Unlike other units modules, this module does not carry the units along with the value, but merely provides a conversion function for transforming values. Usage example:: import nxs.unit u = nxs.unit.Converter('mili*metre') # Units stored in mm v = u(3000,'m') # Convert the value 3000 mm into meters NeXus example:: # Load sample orientation in radians regardless of how it is stored. # 1. Open the path file.openpath('/entry1/sample/sample_orientation') # 2. scan the attributes, retrieving 'units' units = [for attr,value in file.attrs() if attr == 'units'] # 3. set up the converter (assumes that units actually exists) u = nxs.unit.Converter(units[0]) # 4. read the data and convert to the correct units v = u(file.read(),'radians') This is a standalone module, not relying on either DANSE or NeXus, and can be used for other unit conversion tasks. Note: minutes are used for angle and seconds are used for time. We cannot tell what the correct interpretation is without knowing something about the fields themselves. If this becomes an issue, we will need to allow the application to set the dimension for the units rather than getting the dimension from the units as we are currently doing. """ # TODO: Add udunits to NAPI rather than reimplementing it in python # TODO: Alternatively, parse the udunits database directly # UDUnits: # http://www.unidata.ucar.edu/software/udunits/udunits-1/udunits.txt # TODO: Allow application to impose the map on the units from __future__ import division __all__ = ['Converter'] import math # Limited form of units for returning objects of a specific type. # Maybe want to do full units handling with e.g., pyre's # unit class. For now lets keep it simple. Note that def _build_metric_units(unit,abbr): """ Construct standard SI names for the given unit. Builds e.g., s, ns second, nanosecond, nano*second seconds, nanoseconds Includes prefixes for femto through peta. Ack! Allows, e.g., Coulomb and coulomb even though Coulomb is not a unit because some NeXus files store it that way! Returns a dictionary of names and scales. """ prefix = dict(peta=1e15,tera=1e12,giga=1e9,mega=1e6,kilo=1e3, deci=1e-1,centi=1e-2,milli=1e-3,mili=1e-3,micro=1e-6, nano=1e-9,pico=1e-12,femto=1e-15) short_prefix = dict(P=1e15,T=1e12,G=1e9,M=1e6,k=1e3, d=1e-1,c=1e-2,m=1e-3,u=1e-6, n=1e-9,p=1e-12,f=1e-15) map = {abbr:1} map.update([(P+abbr,scale) for (P,scale) in short_prefix.iteritems()]) for name in [unit,unit.capitalize()]: map.update({name:1,name+'s':1}) map.update([(P+name,scale) for (P,scale) in prefix.iteritems()]) map.update([(P+'*'+name,scale) for (P,scale) in prefix.iteritems()]) map.update([(P+name+'s',scale) for (P,scale) in prefix.iteritems()]) return map def _build_plural_units(**kw): """ Construct names for the given units. Builds singular and plural form. """ map = {} map.update([(name,scale) for name,scale in kw.iteritems()]) map.update([(name+'s',scale) for name,scale in kw.iteritems()]) return map def _build_all_units(): # Various distance measures distance = _build_metric_units('meter','m') distance.update(_build_metric_units('metre','m')) distance.update(_build_plural_units(micron=1e-6, Angstrom=1e-10)) distance.update({'A':1e-10, 'Ang':1e-10}) # Various time measures. # Note: minutes are used for angle rather than time time = _build_metric_units('second','s') time.update(_build_plural_units(hour=3600,day=24*3600,week=7*24*3600)) # Various angle measures. # Note: seconds are used for time rather than angle angle = _build_plural_units(degree=1, minute=1/60., arcminute=1/60., arcsecond=1/3600., radian=180/math.pi) angle.update(deg=1, arcmin=1/60., arcsec=1/3600., rad=180/math.pi) frequency = _build_metric_units('hertz','Hz') frequency.update(_build_metric_units('Hertz','Hz')) frequency.update(_build_plural_units(rpm=1/60.)) # Note: degrees are used for angle # Note: temperature needs an offset as well as a scale temperature = _build_metric_units('kelvin','K') temperature.update(_build_metric_units('Kelvin','K')) charge = _build_metric_units('coulomb','C') charge.update({'microAmp*hour':0.0036}) sld = { '10^-6 Angstrom^-2': 1e-6, 'Angstrom^-2': 1} Q = { 'invAng': 1, 'invAngstroms': 1, '10^-3 Angstrom^-1': 1e-3, 'nm^-1': 10 } # APS files may be using 'a.u.' for 'arbitrary units'. Other # facilities are leaving the units blank, using ??? or not even # writing the units attributes. unknown = {None:1, '???':1, '': 1, 'a.u.':1} dims = [unknown, distance, time, angle, frequency, temperature, charge, sld, Q] return dims class Converter(object): """ Unit converter for NeXus style units. """ # Define the units, using both American and European spelling. scalemap = None scalebase = 1 dims = _build_all_units() def __init__(self,name): self.base = name for map in self.dims: if name in map: self.scalemap = map self.scalebase = self.scalemap[name] break else: self.scalemap = {'': 1} self.scalebase = 1 #raise ValueError, "Unknown unit %s"%name def scale(self, units=""): if units == "" or self.scalemap is None: return 1 return self.scalebase/self.scalemap[units] def __call__(self, value, units=""): # Note: calculating a*1 rather than simply returning a would produce # an unnecessary copy of the array, which in the case of the raw # counts array would be bad. Sometimes copying and other times # not copying is also bad, but copy on modify semantics isn't # supported. if units == "" or self.scalemap is None: return value try: return value * (self.scalebase/self.scalemap[units]) except KeyError: raise KeyError("%s not in %s"%(units," ".join(self.scalemap.keys()))) def _check(expect,get): if expect != get: raise ValueError, "Expected %s but got %s"%(expect,get) #print expect,"==",get def test(): _check(2,Converter('mm')(2000,'m')) # 2000 mm -> 2 m _check(0.003,Converter('microseconds')(3,units='ms')) # 3 us -> 0.003 ms _check(45,Converter('nanokelvin')(45)) # 45 nK -> 45 nK # TODO: more tests _check(0.5,Converter('seconds')(1800,units='hours')) # 1800 -> 0.5 hr _check(2.5,Converter('a.u.')(2.5,units='')) if __name__ == "__main__": test() python-nxs/nxstest.py000066400000000000000000000331031306222103500152610ustar00rootroot00000000000000# This program is public domain # Author: Paul Kienzle """ NeXus tests converted to python. """ import nxs,os,numpy,sys def memfootprint(): import gc objs = gc.get_objects() classes = set( c.__class__ for c in gc.get_objects() if hasattr(c,'__class__') ) # print "\n".join([c.__name__ for c in classes]) print "#objects=",len(objs) print "#classes=",len(classes) def leak_test1(n = 1000, mode='w5'): # import gc # gc.enable() # gc.set_debug(gc.DEBUG_LEAK) filename = "leak_test1.nxs" try: os.unlink(filename) except OSError: pass file = nxs.open(filename,mode) file.close() print "File should exist now" for i in range(n): if i%100 == 0: print "loop count %d"%i memfootprint() file.open() file.close() # gc.collect() os.unlink(filename) def _show(file, indent=0): prefix = ' '*indent link = file.link() if link: print "%(prefix)s-> %(link)s" % locals() return for attr,value in file.attrs(): print "%(prefix)s@%(attr)s: %(value)s" % locals() for name,nxclass in file.entries(): if nxclass == "SDS": shape,dtype = file.getinfo() dims = "x".join([str(x) for x in shape]) print "%(prefix)s%(name)s %(dtype)s %(dims)s" % locals() link = file.link() if link: print " %(prefix)s-> %(link)s" % locals() else: for attr,value in file.attrs(): print " %(prefix)s@%(attr)s: %(value)s" % locals() if numpy.prod(shape) < 8: value = file.getdata() print " %s%s"%(prefix,str(value)) else: print "%(prefix)s%(name)s %(nxclass)s" % locals() _show(file, indent+2) def show_structure(filename): file = nxs.open(filename) print "=== File",file.inquirefile() _show(file) def populate(filename,mode): c1 = numpy.array(['abcd','efgh','ijkl','mnop','qrst']) i1 = numpy.arange(4,dtype='uint8') i2 = numpy.arange(4,dtype='int16')*1000 i4 = numpy.arange(4,dtype='int32')*1000000 i8 = numpy.arange(4,dtype='int64')*1000000000000 r4 = numpy.arange(20,dtype='float32').reshape((5,4)) r8 = numpy.arange(20,dtype='float64').reshape((5,4)) comp_array=numpy.ones((100,20),dtype='int32') for i in range(100): comp_array[i,:] *= i file = nxs.open(filename,mode) file.setnumberformat('float32','%9.3g') file.makegroup("entry","NXentry") file.opengroup("entry","NXentry") file.putattr("hugo","namenlos") file.putattr("cucumber","passion") #file.putattr("embedded_null","embedded\000null") # Write character data file.makedata("ch_data",'char',[10]) file.opendata("ch_data") file.putdata("NeXus data") file.closedata() file.makedata("c1_data",'char',[5,4]) file.opendata("c1_data") file.putdata(c1) file.closedata() # Write numeric data for var in ['i1','i2','i4','i8','r4']: if mode == 'w4' and var == 'i8': continue name = var+'_data' val = locals()[var] file.makedata(name,val.dtype,val.shape) file.opendata(name) file.putdata(val) file.closedata() # Write r8_data file.makedata('r8_data','float64',[5,4]) file.opendata('r8_data') file.putslab(r8[4,:],[4,0],[1,4]) file.putslab(r8[0:4,:],[0,0],[4,4]) file.putattr("ch_attribute","NeXus") file.putattr("i4_attribute",42,dtype='int32') file.putattr("r4_attribute",3.14159265,dtype='float32') ## Oops... NAPI doesn't support array attributes #file.putattr("i4_array",[3,2],dtype='int32') #file.putattr("r4_array",[3.14159265,2.718281828],dtype='float32') dataID = file.getdataID() file.closedata() # Create the NXdata group file.makegroup("data","NXdata") file.opengroup("data","NXdata") # .. demonstrate linking file.makelink(dataID) # .. demonstrate compressed data file.compmakedata("comp_data",'int32',[100,20],'lzw',[20,20]) file.opendata('comp_data') file.putdata(comp_array) file.closedata() file.flush() # .. demonstrate extensible data file.makedata('flush_data','int32',[nxs.UNLIMITED]) file.opendata('flush_data') for i in range(7): file.putslab(i,[i],[1]) file.closedata() file.flush() file.closegroup() # Create NXsample group file.makegroup('sample','NXsample') file.opengroup('sample','NXsample') file.makedata('ch_data','char',[20]) file.opendata('ch_data') file.putdata('NeXus sample') file.closedata() sampleID = file.getgroupID() file.closegroup() file.closegroup() # Demonstrate named links file.makegroup('link','NXentry') file.opengroup('link','NXentry') file.makelink(sampleID) file.makenamedlink('renLinkGroup',sampleID) file.makenamedlink('renLinkData',dataID) file.closegroup() file.close() return filename failures = 0 def fail(msg): global failures print "FAIL:",msg failures += 1 def dicteq(a,b): """ Compare two dictionaries printing how they differ. """ for k,v in a.iteritems(): if k not in b: print k,"not in",b return False if v != b[k]: print v,"not equal",b[k] return False for k,v in b.iteritems(): if k not in a: print k,"not in",a return False return True def check(filename, mode): global failures failures = 0 file = nxs.open(filename,'rw') if filename != file.inquirefile(): fail("Files don't match") # check headers num_attrs = file.getattrinfo() wxattrs = ['xmlns','xmlns:xsi','xsi:schemaLocation', 'XML_version'] w4attrs = ['HDF_version'] w5attrs = ['HDF5_Version'] extras = dict(wx=wxattrs,w4=w4attrs,w5=w5attrs) expected_attrs = ['NeXus_version','file_name','file_time']+extras[mode] for i in range(num_attrs): name,dims,type = file.getnextattr() if name not in expected_attrs: fail("attribute %s unexpected"%(name)) if num_attrs != len(expected_attrs): fail("Expected %d root attributes but got %d" % (len(expected_attrs),num_attrs)) file.opengroup('entry','NXentry') expect = dict(hugo='namenlos',cucumber='passion') #expect['embedded_null'] = "embedded\000null" get = dict((k,v) for k,v in file.attrs()) same = dicteq(get,expect) if not same: fail("/entry attributes are %s"%(get)) # Check that the numbers are written correctly for name,dtype,shape,scale in \ [('i1','int8',(4),1), ('i2','int16',(4),1000), ('i4','int32',(4),1000000), ('i8','int64',(4),1000000000000), ('r4','float32',(5,4),1), ('r8','float64',(5,4),1) ]: if mode == 'w4' and name == 'i8': continue n = numpy.prod(shape) expected = numpy.arange(n,dtype=dtype).reshape(shape)*scale file.opendata(name+'_data') get = file.getdata() file.closedata() if not (get == expected).all(): fail("%s retrieved %s"%(dtype,get)) # Check attribute types file.opendata('r8_data') get = file.getattr("ch_attribute",5,'char') if not get == "NeXus": fail("ch_attribute retrieved %s"%(get)) get = file.getattr("i4_attribute",1,'int32') if not get == numpy.int32(42): fail("i4_attribute retrieved %s"%(get)) get = file.getattr("r4_attribute",1,'float32') if ((mode=='wx' and not abs(get-3.14159265) < 1e-6) or (mode!='wx' and not get == numpy.float32(3.14159265))): fail("r4_attribute retrieved %s"%(get)) ## Oops... NAPI doesn't support array attributes #expect = numpy.array([3,2],dtype='int32') #get = file.getattr("i4_array",2,'int32') #if not (get==expect).all(): fail('i4_array retrieved %s'%(get)) #expect = numpy.array([3.14159265,2.718281828],dtype='float32') #get = file.getattr("r4_array",2,dtype='float32') #if not (get==expect).all(): fail("r4_array retrieved %s"%(get)) file.closedata() file.opendata('c1_data') rawshape,rawdtype = file.getrawinfo() shape,dtype = file.getinfo() get = file.getdata() file.closedata() if not (shape[0]==5 and shape[1]==4 and dtype=='char'): fail("returned string array info is incorrect") if not (rawshape[0]==5 and rawshape[1]==4 and rawdtype=='char'): fail("returned string array storage info is incorrect") print rawshape,dtype if not (get[0]=="abcd" and get[4]=="qrst"): fail("returned string is incorrect") print shape,dtype # Check reading from compressed datasets comp_array=numpy.ones((100,20),dtype='int32') for i in range(100): comp_array[i,:] *= i expected = comp_array file.opengroup('data','NXdata') #/entry/data file.opendata('comp_data') #/entry/data/comp_data get = file.getdata() file.closedata() #/entry/data/comp_data file.closegroup() #/entry/data if not (get == expected).all(): fail("compressed data differs") print get # Check strings file.opengroup('sample','NXsample') #/entry/sample file.opendata('ch_data') #/entry/sample/ch_data rawshape,rawdtype = file.getrawinfo() shape,dtype = file.getinfo() get = file.getdata() file.closedata() #/entry/sample/ch_data file.closegroup() #/entry/sample if not (shape[0]==12 and dtype=='char'): fail("returned string info is incorrect") print shape,dtype if not (rawshape[0]==20 and rawdtype=='char'): fail("returned string storage info is incorrect") print shape,dtype if not (get == "NeXus sample"): fail("returned string is incorrect") print shape,dtype file.closegroup() #/entry # Check read slab (e.g., from extensible) # Check links file.opengroup('entry','NXentry') file.opengroup('sample','NXsample') sampleid = file.getgroupID() file.closegroup() #/entry/sample file.opengroup('data','NXdata') #/entry/data file.opendata('r8_data') #/entry/data/r8_data dataid = file.getdataID() file.closedata() #/entry/data/r8_data file.closegroup() #/entry/data file.opendata('r8_data') data2id = file.getdataID() file.closedata() file.closegroup() #/entry if not (file.sameID(dataid,data2id)): fail("/entry/data/r8_data not linked to /entry/r8_data") # Check openpath and getslab file.openpath('/entry/data/comp_data') get = file.getslab([4,4],[5,3]) expected = comp_array[4:(4+5),4:(4+3)] if not (get == expected).all(): fail("retrieved compressed slabs differ") print get file.openpath('/entry/data/comp_data') get = file.getslab([4,4],[5,3]) expected = comp_array[4:(4+5),4:(4+3)] if not (get == expected).all(): fail("after reopen: retrieved compressed slabs differ") print get file.openpath('../r8_data') for k,v in file.attrs(): if k == 'target' and v != '/entry/r8_data': fail("relative openpath was not successful") return failures == 0 def populate_external(filename,mode): ext = dict(w5='.h5',w4='.hdf',wx='.xml')[mode] file = nxs.open(filename,mode) file.makegroup('entry1','NXentry') file.linkexternal('entry1','NXentry','nxfile://data/dmc01'+ext) file.makegroup('entry2','NXentry') file.linkexternal('entry2','NXentry','nxfile://data/dmc02'+ext) file.makegroup('entry3','NXentry') file.close() def check_external(filename,mode): ext = dict(w5='.h5',w4='.hdf',wx='.xml')[mode] file = nxs.open(filename,'rw') file.openpath('/entry1/start_time') time = file.getdata() get = file.inquirefile() expected = 'nxfile://data/dmc01'+ext if expected != get: fail("first external file returned %s"%(get)) file.openpath('/entry2/sample/sample_name') sample = file.getdata() get = file.inquirefile() expected = 'nxfile://data/dmc02'+ext if expected != get: fail("second external file returned %s"%(get)) file.openpath('/') remote = file.isexternalgroup('entry1','NXentry') if remote is None: fail("failed to identify /entry1 as external") remote = file.isexternalgroup('entry3','NXentry') if remote is not None: fail('incorrectly identified /entry3 as external') file.close() def test_external(mode,quiet=True): ext = dict(w5='.h5',w4='.hdf',wx='.xml')[mode] filename = 'nxext'+ext populate_external(external,mode) if not quiet: show_structure(external) failures = check_external(filename,mode) return failures def test_mode(mode,quiet=True,external=False): ext = dict(w5='.h5',w4='.hdf',wx='.xml')[mode] filename = 'NXtest'+ext populate(filename,mode=mode) if not quiet and 'NX_LOAD_PATH' in os.environ: show_structure('dmc01'+ext) if not quiet: show_structure(filename) failures = check(filename,mode) if external: failures += test_external(mode,quiet) return failures def test(): tests = 0 if '-q' in sys.argv: quiet = True else: quiet = False if '-x' in sys.argv: external = True else: external = False if 'hdf4' in sys.argv: test_mode('w4',quiet,external) tests += 1 if 'xml' in sys.argv: test_mode('wx',quiet,external) tests += 1 if 'hdf5' in sys.argv: test_mode('w5',quiet,external) tests += 1 if tests == 0: test_mode('w5',quiet,external) if __name__ == "__main__": test() #leak_test1(n=10000) python-nxs/run_nxstest000077500000000000000000000002511306222103500155170ustar00rootroot00000000000000#!/bin/sh # # this script allows nxstest to be run again the built, rather than installed, # version of nexus # env LD_LIBRARY_PATH=../../src/.libs python nxstest.py $* python-nxs/setup.cfg000066400000000000000000000001221306222103500150130ustar00rootroot00000000000000[bdist] formats = rpm [bdist_rpm] requires = nexus, numpy doc_files = README.rst python-nxs/setup.py000077500000000000000000000013221306222103500147120ustar00rootroot00000000000000#!/usr/bin/env python # # This file is for a manual install of python on windows # If python is already present it should be detected by # the windows installer and the nxs directory copied to # the python site-packages directory # # On Linux the Makefile will also install python, but not # using this file - instead it uses the automake python install bits # # To use this file type: # # python setup.py install # from setuptools import setup setup(name='NeXus', version='4.4.1', description='Python Bindings to libNeXus', author='Paul Kienzle', url="https://github.com/nexusformat/python-nxs", packages = ['nxs'], test_suite="nxs.test", license='LGPLv2' )