pax_global_header00006660000000000000000000000064140372561330014516gustar00rootroot0000000000000052 comment=724fb01fc95a183ba9b63747abba1176e9f95463 fypp-3.1/000077500000000000000000000000001403725613300123375ustar00rootroot00000000000000fypp-3.1/.gitignore000066400000000000000000000001111403725613300143200ustar00rootroot00000000000000*~ .idea *.pyc dist build docs/_build *.egg-info .tox _gitmsg.saved.txt fypp-3.1/.travis.yml000066400000000000000000000001601403725613300144450ustar00rootroot00000000000000language: python python: - "3.5" - "3.6" - "3.7" - "3.8" - "3.9" script: test/runtests.sh fypp-3.1/CHANGELOG.rst000066400000000000000000000117121403725613300143620ustar00rootroot00000000000000========== Change Log ========== 3.1 === Added ----- * Global variables _SYSTEM_ and _MACHINE_ to query environment. * Emission of standard (#line pragma styled) line directives. * Factory method arguments in Fypp constructor: evaluator_factory, parser_factor, builder_factory and renderer_factory. Changed ------- * Support for Python 2.7, 3.3 and 3.4 dropped, support for Python 3.9 added. Fixed ----- 3.0 === Added ----- * Implement variable keyword argument in macros. * Add block / contains / endblock construct as alternative for call / nextarg / endcall. * Escaping of preprocessor comments * Possibility of specifying character encoding for file I/O with UTF-8 as default. Changed ------- * Injecting local variables into macros by passing arbitrary (non-declared) keyword arguments is not possible any more. This feature made it impossible to detect typos in keyword argument names in macro calls. [Backwards incompatible] * Variable positional argument in a macro resolves to a list not to a tuple for more consistency with Python. Fixed ----- * Wrong command-line parser initialisation in waf frontend. * _LINE_ and _FILE_ were incorrect if the called macro contained a call directive with an evaluation in its argument. 2.1.1 ===== Fixed ----- * Wrong _LINE_ and _FILE_ values when calling a macro during evaluation of the arguments of a call directive. 2.1 === Fixed ----- * Variable definition without value. Changed ------- * Hosting site and branch names (develop -> master, master -> release). 2.0.1 ===== Fixed ----- * Missing files in Python source distribution package. 2.0 === Added ----- * Direct call format resembling ordinary function call. * Inline direct call directive. * Keyword arguments in direct call and call directive. * Generalized call directive with arbitrary argument types. * Macros with variable number of arguments. * Default values for macro arguments. * Allow names in enddef and endcall directives for better readability. * Del directive and delvar() function. * Assert directive. * Global directive and globalvar() function. * Python-like consistent global and local scopes and scope lookup rules. * Predefined variables _THIS_FILE_ and _THIS_LINE_. * Additional flags in line numbering directives when opening a file or returning to a previous file. * Additional testing with tox for developers. * Python 2.6, 3.0 and 3.1 compatibility. Changed ------- * Setvar directive not allowed as alternative to set any more. [Backwards incompatible] * Old direct call syntax (@:macro arg1) not supported any more [Backwards incompatible] * Inline form of def directive not allowed any more. [Backwards incompatible] * Execution of arbitrary Python script at startup (option -i) has been removed. [Backwards incompatible] * Minimal API change: process_* methods of Fypp do not accept the optional argument env any more. [Backwards incompatible] * Equal sign must be used as separator in set directive for better readability. [Backwards incompatible] * Function setvar() accepts arbitrary number of argument pairs. * Reverse order exception printing, exception first occurring printed as last. * Command line tool formats error messages in GNU-like format. * Make equal sign in set directive mandatory and in setvar directive forbidden. * Search paths for module imports behave more Python-like. * Removed builtins callable() and memoryview() from restricted environment as they are not available in all supported Python versions. Fixed ----- * Line numbering with flags fixes gfortrans confusion with line numbers. 1.2 === Added ----- * Allow (and promote) usage of set directive instead of setvar. * Implement stop request via stop directive. * Assignment to variable tuples. * Hierarchial exception testing. Fixed ----- * Wrong file name in error report, when exception occurs in a macro defined in an included file. 1.1 === Added ----- * Allow inline eval and control directives in direct macro call arguments. * Add waf integration modules. * Examples and build system intergration chapters in user guide. * Change log file. 1.0 === Added ----- * Optional suppression of line numbering in continuation lines. * Optional creation of parent folders for output file. Changed ------- * Class Fypp independent of ArgumentParser. Fixed ----- * Fix false error, when include was within a directive. * Wrong line number offset in eval directives. 0.12 ==== Added ----- * Implement direct call. Changed ------- * Remove paranthesis from direct call. 0.11 ==== Added ----- * Implement call directive. * More precise error messages. * Folding prevention for comment lines. * Smart line folding, fixed format line folding. * Python 2.7 compatibility. Changed ------- * Control directive prefix changed from ``@`` to ``#``. * Rename function `default()` into `getvar()`. Fixed ----- * Superfluous trailing newlines in macro calls. 0.9 === Added ----- * Basic functionality. fypp-3.1/LICENSE.txt000066400000000000000000000024461403725613300141700ustar00rootroot00000000000000Copyright (c) 2016-2021 Bálint Aradi, Universität Bremen All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. fypp-3.1/MANIFEST.in000066400000000000000000000002111403725613300140670ustar00rootroot00000000000000include bin/fypp include LICENSE.txt include CHANGELOG.rst recursive-include test *.sh *.inc *.py global-exclude *.pyc global-exclude *~ fypp-3.1/README.rst000066400000000000000000000140521403725613300140300ustar00rootroot00000000000000********************************************* Fypp — Python powered Fortran metaprogramming ********************************************* .. image:: https://travis-ci.org/aradi/fypp.svg?branch=develop :target: https://travis-ci.org/aradi/fypp Fypp is a Python powered preprocessor. It can be used for any programming languages but its primary aim is to offer a Fortran preprocessor, which helps to extend Fortran with condititional compiling and template metaprogramming capabilities. Instead of introducing its own expression syntax, it uses Python expressions in its preprocessor directives, offering the consistency and versatility of Python when formulating metaprogramming tasks. It puts strong emphasis on robustness and on neat integration into developing toolchains. The project is `hosted on github `_. `Detailed DOCUMENTATION `_ is available on `readthedocs.org `_. Fypp is released under the *BSD 2-clause license*. Main features ============= * Definition, evaluation and removal of variables:: #:if DEBUG > 0 print *, "Some debug information" #:endif #:set LOGLEVEL = 2 print *, "LOGLEVEL: ${LOGLEVEL}$" #:del LOGLEVEL * Macro definitions and macro calls:: #:def ASSERT(cond) #:if DEBUG > 0 if (.not. ${cond}$) then print *, "Assert failed in file ${_FILE_}$, line ${_LINE_}$" error stop end if #:endif #:enddef ASSERT ! Invoked via direct call (argument needs no quotation) @:ASSERT(size(myArray) > 0) ! Invoked as Python expression (argument needs quotation) $:ASSERT('size(myArray) > 0') * Conditional output:: program test #:if defined('WITH_MPI') use mpi #:elif defined('WITH_OPENMP') use openmp #:else use serial #:endif * Iterated output (e.g. for generating Fortran templates):: interface myfunc #:for dtype in ['real', 'dreal', 'complex', 'dcomplex'] module procedure myfunc_${dtype}$ #:endfor end interface myfunc * Inline directives:: logical, parameter :: hasMpi = #{if defined('MPI')}# .true. #{else}# .false. #{endif}# * Insertion of arbitrary Python expressions:: character(*), parameter :: comp_date = "${time.strftime('%Y-%m-%d')}$" * Inclusion of files during preprocessing:: #:include "macrodefs.fypp" * Using Fortran-style continutation lines in preprocessor directives:: #:if var1 > var2 & & or var2 > var4 print *, "Doing something here" #:endif * Passing (unquoted) multiline string arguments to callables:: #! Callable needs only string argument #:def DEBUG_CODE(code) #:if DEBUG > 0 $:code #:endif #:enddef DEBUG_CODE #! Pass code block as first positional argument #:block DEBUG_CODE if (size(array) > 100) then print *, "DEBUG: spuriously large array" end if #:endblock DEBUG_CODE #! Callable needs also non-string argument types #:def REPEAT_CODE(code, repeat) #:for ind in range(repeat) $:code #:endfor #:enddef REPEAT_CODE #! Pass code block as positional argument and 3 as keyword argument "repeat" #:block REPEAT_CODE(repeat=3) this will be repeated 3 times #:endblock REPEAT_CODE * Preprocessor comments:: #! This will not show up in the output #! Also the newline characters at the end of the lines will be suppressed * Suppressing the preprocessor output in selected regions:: #! Definitions are read, but no output (e.g. newlines) will be produced #:mute #:include "macrodefs.fypp" #:endmute * Explicit request for stopping the preprocessor:: #:if DEBUGLEVEL < 0 #:stop 'Negative debug level not allowed!' #:endif * Easy check for macro parameter sanity:: #:def mymacro(RANK) #! Macro only works for RANK 1 and above #:assert RANK > 0 : #:enddef mymacro * Line numbering directives in output:: program test #:if defined('MPI') use mpi #:endif : transformed to :: # 1 "test.fypp" 1 program test # 3 "test.fypp" use mpi # 5 "test.fypp" : when variable ``MPI`` is defined and Fypp was instructed to generate line markers. * Automatic folding of generated lines exceeding line length limit Installing ========== Fypp needs a working Python interpreter. It is compatible with Python 2 (version 2.6 and above) and Python 3 (all versions). Automatic install ----------------- Use Pythons command line installer ``pip`` in order to download the stable release from the `Fypp page on PyPI `_ and install it on your system:: pip install fypp This installs both, the command line tool ``fypp`` and the Python module ``fypp.py``. Latter you can import if you want to access the functionality of Fypp directly from within your Python scripts. Manual install -------------- For a manual install, you can download the source code of the **stable** releases from the `Fypp project website `_. If you wish to obtain the latest **development** version, clone the projects repository:: git clone https://github.com/aradi/fypp.git and check out the `master` branch. The command line tool is a single stand-alone script. You can run it directly from the source folder :: FYPP_SOURCE_FOLDER/bin/fypp or after copying it from the `bin` folder to any location listed in your `PATH` environment variable, by just issuing :: fypp The python module ``fypp.py`` can be found in ``FYP_SOURCE_FOLDER/src``. Running ======= The Fypp command line tool reads a file, preprocesses it and writes it to another file, so you would typically invoke it like:: fypp source.fpp source.f90 which would process `source.fpp` and write the result to `source.f90`. If input and output files are not specified, information is read from stdin and written to stdout. The behavior of Fypp can be influenced with various command line options. A summary of all command line options can be obtained by:: fypp -h fypp-3.1/bin/000077500000000000000000000000001403725613300131075ustar00rootroot00000000000000fypp-3.1/bin/fypp000077500000000000000000003403751403725613300140270ustar00rootroot00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- ################################################################################ # # fypp -- Python powered Fortran preprocessor # # Copyright (c) 2016-2021 Bálint Aradi, Universität Bremen # # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are met: # # 1. Redistributions of source code must retain the above copyright notice, this # list of conditions and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright notice, # this list of conditions and the following disclaimer in the documentation # and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 'AS IS' # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # ################################################################################ '''For using the functionality of the Fypp preprocessor from within Python, one usually interacts with the following two classes: * `Fypp`_: The actual Fypp preprocessor. It returns for a given input the preprocessed output. * `FyppOptions`_: Contains customizable settings controlling the behaviour of `Fypp`_. Alternatively, the function `get_option_parser()`_ can be used to obtain an option parser, which can create settings based on command line arguments. If processing stops prematurely, an instance of one of the following subclasses of `FyppError`_ is raised: * FyppFatalError: Unexpected error (e.g. bad input, missing files, etc.) * FyppStopRequest: Stop was triggered by an explicit request in the input (by a stop- or an assert-directive). ''' import sys import types import inspect import re import os import errno import time import optparse import io import platform import builtins # Prevent cluttering user directory with Python bytecode sys.dont_write_bytecode = True VERSION = '3.1' STDIN = '' FILEOBJ = '' STRING = '' ERROR_EXIT_CODE = 1 USER_ERROR_EXIT_CODE = 2 _ALL_DIRECTIVES_PATTERN = r''' # comment block (?:^[ \t]*\#!.*\n)+ | # line directive (with optional continuation lines) ^[ \t]*(?P[\#\$@]):[ \t]* (?P.+?(?:&[ \t]*\n(?:[ \t]*&)?.*?)*)?[ \t]*\n | # inline eval directive (?P[$\#@])\{[ \t]*(?P.+?)?[ \t]*\}(?P=idirtype) ''' _ALL_DIRECTIVES_REGEXP = re.compile( _ALL_DIRECTIVES_PATTERN, re.VERBOSE | re.MULTILINE) _CONTROL_DIR_REGEXP = re.compile( r'(?P[a-zA-Z_]\w*)[ \t]*(?:[ \t]+(?P[^ \t].*))?$') _DIRECT_CALL_REGEXP = re.compile( r'(?P[a-zA-Z_][\w.]*)[ \t]*\((?P.+?)?\)$') _DIRECT_CALL_KWARG_REGEXP = re.compile( r'(?:(?P[a-zA-Z_]\w*)\s*=(?=[^=]|$))?') _DEF_PARAM_REGEXP = re.compile( r'^(?P[a-zA-Z_]\w*)[ \t]*\(\s*(?P.+)?\s*\)$') _SIMPLE_CALLABLE_REGEXP = re.compile( r'^(?P[a-zA-Z_][\w.]*)[ \t]*(?:\([ \t]*(?P.*)[ \t]*\))?$') _IDENTIFIER_NAME_REGEXP = re.compile(r'^(?P[a-zA-Z_]\w*)$') _PREFIXED_IDENTIFIER_NAME_REGEXP = re.compile(r'^(?P[a-zA-Z_][\w.]*)$') _SET_PARAM_REGEXP = re.compile( r'^(?P(?:[(]\s*)?[a-zA-Z_]\w*(?:\s*,\s*[a-zA-Z_]\w*)*(?:\s*[)])?)\s*'\ r'(?:=\s*(?P.*))?$') _DEL_PARAM_REGEXP = re.compile( r'^(?:[(]\s*)?[a-zA-Z_]\w*(?:\s*,\s*[a-zA-Z_]\w*)*(?:\s*[)])?$') _FOR_PARAM_REGEXP = re.compile( r'^(?P[a-zA-Z_]\w*(\s*,\s*[a-zA-Z_]\w*)*)\s+in\s+(?P.+)$') _INCLUDE_PARAM_REGEXP = re.compile(r'^(\'|")(?P.*?)\1$') _COMMENTLINE_REGEXP = re.compile(r'^[ \t]*!.*$') _CONTLINE_REGEXP = re.compile(r'&[ \t]*\n(?:[ \t]*&)?') _UNESCAPE_TEXT_REGEXP1 = re.compile(r'([$#@])\\(\\*)([{:])') _UNESCAPE_TEXT_REGEXP2 = re.compile(r'#\\(\\*)([!])') _UNESCAPE_TEXT_REGEXP3 = re.compile(r'(\})\\(\\*)([$#@])') _INLINE_EVAL_REGION_REGEXP = re.compile(r'\${.*?}\$') _RESERVED_PREFIX = '__' _RESERVED_NAMES = set(['defined', 'setvar', 'getvar', 'delvar', 'globalvar', '_LINE_', '_FILE_', '_THIS_FILE_', '_THIS_LINE_', '_TIME_', '_DATE_', '_SYSTEM_', '_MACHINE_']) _LINENUM_NEW_FILE = 1 _LINENUM_RETURN_TO_FILE = 2 _QUOTES_FORTRAN = '\'"' _OPENING_BRACKETS_FORTRAN = '{([' _CLOSING_BRACKETS_FORTRAN = '})]' _ARGUMENT_SPLIT_CHAR_FORTRAN = ',' class FyppError(Exception): '''Signalizes error occurring during preprocessing. Args: msg (str): Error message. fname (str): File name. None (default) if file name is not available. span (tuple of int): Beginning and end line of the region where error occurred or None if not available. If fname was not None, span must not be None. Attributes: msg (str): Error message. fname (str or None): File name or None if not available. span (tuple of int or None): Beginning and end line of the region where error occurred or None if not available. Line numbers start from zero. For directives, which do not consume end of the line, start and end lines are identical. ''' def __init__(self, msg, fname=None, span=None): super().__init__() self.msg = msg self.fname = fname self.span = span def __str__(self): msg = [self.__class__.__name__, ': '] if self.fname is not None: msg.append("file '" + self.fname + "'") if self.span[1] > self.span[0] + 1: msg.append(', lines {0}-{1}'.format( self.span[0] + 1, self.span[1])) else: msg.append(', line {0}'.format(self.span[0] + 1)) msg.append('\n') if self.msg: msg.append(self.msg) if self.__cause__ is not None: msg.append('\n' + str(self.__cause__)) return ''.join(msg) class FyppFatalError(FyppError): '''Signalizes an unexpected error during processing.''' class FyppStopRequest(FyppError): '''Signalizes an explicitely triggered stop (e.g. via stop directive)''' class Parser: '''Parses a text and generates events when encountering Fypp constructs. Args: includedirs (list): List of directories, in which include files should be searched for, when they are not found at the default location. encoding (str): Encoding to use when reading the file (default: utf-8) ''' def __init__(self, includedirs=None, encoding='utf-8'): # Directories to search for include files if includedirs is None: self._includedirs = [] else: self._includedirs = includedirs # Encoding self._encoding = encoding # Name of current file self._curfile = None # Directory of current file self._curdir = None def parsefile(self, fobj): '''Parses file or a file like object. Args: fobj (str or file): Name of a file or a file like object. ''' if isinstance(fobj, str): if fobj == STDIN: self._includefile(None, sys.stdin, STDIN, os.getcwd()) else: inpfp = _open_input_file(fobj, self._encoding) self._includefile(None, inpfp, fobj, os.path.dirname(fobj)) inpfp.close() else: self._includefile(None, fobj, FILEOBJ, os.getcwd()) def _includefile(self, span, fobj, fname, curdir): oldfile = self._curfile olddir = self._curdir self._curfile = fname self._curdir = curdir self._parse_txt(span, fname, fobj.read()) self._curfile = oldfile self._curdir = olddir def parse(self, txt): '''Parses string. Args: txt (str): Text to parse. ''' self._curfile = STRING self._curdir = '' self._parse_txt(None, self._curfile, txt) def handle_include(self, span, fname): '''Called when parser starts to process a new file. It is a dummy methond and should be overridden for actual use. Args: span (tuple of int): Start and end line of the include directive or None if called the first time for the main input. fname (str): Name of the file. ''' self._log_event('include', span, filename=fname) def handle_endinclude(self, span, fname): '''Called when parser finished processing a file. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the include directive or None if called the first time for the main input. fname (str): Name of the file. ''' self._log_event('endinclude', span, filename=fname) def handle_set(self, span, name, expr): '''Called when parser encounters a set directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the variable. expr (str): String representation of the expression to be assigned to the variable. ''' self._log_event('set', span, name=name, expression=expr) def handle_def(self, span, name, args): '''Called when parser encounters a def directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the macro to be defined. argexpr (str): String with argument definition (or None) ''' self._log_event('def', span, name=name, arguments=args) def handle_enddef(self, span, name): '''Called when parser encounters an enddef directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name found after the enddef directive. ''' self._log_event('enddef', span, name=name) def handle_del(self, span, name): '''Called when parser encounters a del directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the variable to delete. ''' self._log_event('del', span, name=name) def handle_if(self, span, cond): '''Called when parser encounters an if directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. cond (str): String representation of the branching condition. ''' self._log_event('if', span, condition=cond) def handle_elif(self, span, cond): '''Called when parser encounters an elif directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. cond (str): String representation of the branching condition. ''' self._log_event('elif', span, condition=cond) def handle_else(self, span): '''Called when parser encounters an else directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('else', span) def handle_endif(self, span): '''Called when parser encounters an endif directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('endif', span) def handle_for(self, span, varexpr, iterator): '''Called when parser encounters a for directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. varexpr (str): String representation of the loop variable expression. iterator (str): String representation of the iterable. ''' self._log_event('for', span, variable=varexpr, iterable=iterator) def handle_endfor(self, span): '''Called when parser encounters an endfor directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('endfor', span) def handle_call(self, span, name, argexpr, blockcall): '''Called when parser encounters a call directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the callable to call argexpr (str or None): Argument expression containing additional arguments for the call. blockcall (bool): Whether the alternative "block / contains / endblock" calling directive has been used. ''' self._log_event('call', span, name=name, argexpr=argexpr, blockcall=blockcall) def handle_nextarg(self, span, name, blockcall): '''Called when parser encounters a nextarg directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str or None): Name of the argument following next or None if it should be the next positional argument. blockcall (bool): Whether the alternative "block / contains / endblock" calling directive has been used. ''' self._log_event('nextarg', span, name=name, blockcall=blockcall) def handle_endcall(self, span, name, blockcall): '''Called when parser encounters an endcall directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name found after the endcall directive. blockcall (bool): Whether the alternative "block / contains / endblock" calling directive has been used. ''' self._log_event('endcall', span, name=name, blockcall=blockcall) def handle_eval(self, span, expr): '''Called when parser encounters an eval directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. expr (str): String representation of the Python expression to be evaluated. ''' self._log_event('eval', span, expression=expr) def handle_global(self, span, name): '''Called when parser encounters a global directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the variable which should be made global. ''' self._log_event('global', span, name=name) def handle_text(self, span, txt): '''Called when parser finds text which must left unaltered. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. txt (str): Text. ''' self._log_event('text', span, content=txt) def handle_comment(self, span): '''Called when parser finds a preprocessor comment. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('comment', span) def handle_mute(self, span): '''Called when parser finds a mute directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('mute', span) def handle_endmute(self, span): '''Called when parser finds an endmute directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('endmute', span) def handle_stop(self, span, msg): '''Called when parser finds an stop directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. msg (str): Stop message. ''' self._log_event('stop', span, msg=msg) def handle_assert(self, span): '''Called when parser finds an assert directive. It is a dummy method and should be overridden for actual use. Args: span (tuple of int): Start and end line of the directive. ''' self._log_event('assert', span) @staticmethod def _log_event(event, span=(-1, -1), **params): print('{0}: {1} --> {2}'.format(event, span[0], span[1])) for parname, parvalue in params.items(): print(' {0}: ->|{1}|<-'.format(parname, parvalue)) print() def _parse_txt(self, includespan, fname, txt): self.handle_include(includespan, fname) self._parse(txt) self.handle_endinclude(includespan, fname) def _parse(self, txt, linenr=0, directcall=False): pos = 0 for match in _ALL_DIRECTIVES_REGEXP.finditer(txt): start, end = match.span() if start > pos: endlinenr = linenr + txt.count('\n', pos, start) self._process_text(txt[pos:start], (linenr, endlinenr)) linenr = endlinenr endlinenr = linenr + txt.count('\n', start, end) span = (linenr, endlinenr) ldirtype, ldir, idirtype, idir = match.groups() if directcall and (idirtype is None or idirtype != '$'): msg = 'only inline eval directives allowed in direct calls' raise FyppFatalError(msg, self._curfile, span) elif idirtype is not None: if idir is None: msg = 'missing inline directive content' raise FyppFatalError(msg, self._curfile, span) dirtype = idirtype content = idir elif ldirtype is not None: if ldir is None: msg = 'missing line directive content' raise FyppFatalError(msg, self._curfile, span) dirtype = ldirtype content = _CONTLINE_REGEXP.sub('', ldir) else: # Comment directive dirtype = None if dirtype == '$': self.handle_eval(span, content) elif dirtype == '#': self._process_control_dir(content, span) elif dirtype == '@': self._process_direct_call(content, span) else: self.handle_comment(span) pos = end linenr = endlinenr if pos < len(txt): endlinenr = linenr + txt.count('\n', pos) self._process_text(txt[pos:], (linenr, endlinenr)) def _process_text(self, txt, span): escaped_txt = self._unescape(txt) self.handle_text(span, escaped_txt) def _process_control_dir(self, content, span): match = _CONTROL_DIR_REGEXP.match(content) if not match: msg = "invalid control directive content '{0}'".format(content) raise FyppFatalError(msg, self._curfile, span) directive, param = match.groups() if directive == 'if': self._check_param_presence(True, 'if', param, span) self.handle_if(span, param) elif directive == 'else': self._check_param_presence(False, 'else', param, span) self.handle_else(span) elif directive == 'elif': self._check_param_presence(True, 'elif', param, span) self.handle_elif(span, param) elif directive == 'endif': self._check_param_presence(False, 'endif', param, span) self.handle_endif(span) elif directive == 'def': self._check_param_presence(True, 'def', param, span) self._check_not_inline_directive('def', span) self._process_def(param, span) elif directive == 'enddef': self._process_enddef(param, span) elif directive == 'set': self._check_param_presence(True, 'set', param, span) self._process_set(param, span) elif directive == 'del': self._check_param_presence(True, 'del', param, span) self._process_del(param, span) elif directive == 'for': self._check_param_presence(True, 'for', param, span) self._process_for(param, span) elif directive == 'endfor': self._check_param_presence(False, 'endfor', param, span) self.handle_endfor(span) elif directive == 'call' or directive == 'block': self._check_param_presence(True, directive, param, span) self._process_call(param, span, directive == 'block') elif directive == 'nextarg' or directive == 'contains': self._process_nextarg(param, span, directive == 'contains') elif directive == 'endcall' or directive == 'endblock': self._process_endcall(param, span, directive == 'endblock') elif directive == 'include': self._check_param_presence(True, 'include', param, span) self._check_not_inline_directive('include', span) self._process_include(param, span) elif directive == 'mute': self._check_param_presence(False, 'mute', param, span) self._check_not_inline_directive('mute', span) self.handle_mute(span) elif directive == 'endmute': self._check_param_presence(False, 'endmute', param, span) self._check_not_inline_directive('endmute', span) self.handle_endmute(span) elif directive == 'stop': self._check_param_presence(True, 'stop', param, span) self._check_not_inline_directive('stop', span) self.handle_stop(span, param) elif directive == 'assert': self._check_param_presence(True, 'assert', param, span) self._check_not_inline_directive('assert', span) self.handle_assert(span, param) elif directive == 'global': self._check_param_presence(True, 'global', param, span) self._process_global(param, span) else: msg = "unknown directive '{0}'".format(directive) raise FyppFatalError(msg, self._curfile, span) def _process_direct_call(self, callexpr, span): match = _DIRECT_CALL_REGEXP.match(callexpr) if not match: msg = "invalid direct call expression" raise FyppFatalError(msg, self._curfile, span) callname = match.group('callname') self.handle_call(span, callname, None, False) callparams = match.group('callparams') if callparams is None or not callparams.strip(): args = [] else: try: args = [arg.strip() for arg in _argsplit_fortran(callparams)] except Exception as exc: msg = 'unable to parse direct call argument' raise FyppFatalError(msg, self._curfile, span) from exc for arg in args: match = _DIRECT_CALL_KWARG_REGEXP.match(arg) argval = arg[match.end():].strip() # Remove enclosing braces if present if argval.startswith('{'): argval = argval[1:-1] keyword = match.group('kwname') self.handle_nextarg(span, keyword, False) self._parse(argval, linenr=span[0], directcall=True) self.handle_endcall(span, callname, False) def _process_def(self, param, span): match = _DEF_PARAM_REGEXP.match(param) if not match: msg = "invalid macro definition '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) name = match.group('name') argexpr = match.group('args') self.handle_def(span, name, argexpr) def _process_enddef(self, param, span): if param is not None: match = _IDENTIFIER_NAME_REGEXP.match(param) if not match: msg = "invalid enddef parameter '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) param = match.group('name') self.handle_enddef(span, param) def _process_set(self, param, span): match = _SET_PARAM_REGEXP.match(param) if not match: msg = "invalid variable assignment '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) self.handle_set(span, match.group('name'), match.group('expr')) def _process_global(self, param, span): match = _DEL_PARAM_REGEXP.match(param) if not match: msg = "invalid variable specification '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) self.handle_global(span, param) def _process_del(self, param, span): match = _DEL_PARAM_REGEXP.match(param) if not match: msg = "invalid variable specification '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) self.handle_del(span, param) def _process_for(self, param, span): match = _FOR_PARAM_REGEXP.match(param) if not match: msg = "invalid for loop declaration '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) loopexpr = match.group('loopexpr') loopvars = [s.strip() for s in loopexpr.split(',')] self.handle_for(span, loopvars, match.group('iter')) def _process_call(self, param, span, blockcall): match = _SIMPLE_CALLABLE_REGEXP.match(param) if not match: msg = "invalid callable expression '{}'".format(param) raise FyppFatalError(msg, self._curfile, span) name, args = match.groups() self.handle_call(span, name, args, blockcall) def _process_nextarg(self, param, span, blockcall): if param is not None: match = _IDENTIFIER_NAME_REGEXP.match(param) if not match: msg = "invalid nextarg parameter '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) param = match.group('name') self.handle_nextarg(span, param, blockcall) def _process_endcall(self, param, span, blockcall): if param is not None: match = _PREFIXED_IDENTIFIER_NAME_REGEXP.match(param) if not match: msg = "invalid endcall parameter '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) param = match.group('name') self.handle_endcall(span, param, blockcall) def _process_include(self, param, span): match = _INCLUDE_PARAM_REGEXP.match(param) if not match: msg = "invalid include file declaration '{0}'".format(param) raise FyppFatalError(msg, self._curfile, span) fname = match.group('fname') for incdir in [self._curdir] + self._includedirs: fpath = os.path.join(incdir, fname) if os.path.exists(fpath): break else: msg = "include file '{0}' not found".format(fname) raise FyppFatalError(msg, self._curfile, span) inpfp = _open_input_file(fpath, self._encoding) self._includefile(span, inpfp, fpath, os.path.dirname(fpath)) inpfp.close() def _process_mute(self, span): if span[0] == span[1]: msg = 'Inline form of mute directive not allowed' raise FyppFatalError(msg, self._curfile, span) self.handle_mute(span) def _process_endmute(self, span): if span[0] == span[1]: msg = 'Inline form of endmute directive not allowed' raise FyppFatalError(msg, self._curfile, span) self.handle_endmute(span) def _check_param_presence(self, presence, directive, param, span): if (param is not None) != presence: if presence: msg = 'missing data in {0} directive'.format(directive) else: msg = 'forbidden data in {0} directive'.format(directive) raise FyppFatalError(msg, self._curfile, span) def _check_not_inline_directive(self, directive, span): if span[0] == span[1]: msg = 'Inline form of {0} directive not allowed'.format(directive) raise FyppFatalError(msg, self._curfile, span) @staticmethod def _unescape(txt): txt = _UNESCAPE_TEXT_REGEXP1.sub(r'\1\2\3', txt) txt = _UNESCAPE_TEXT_REGEXP2.sub(r'#\1\2', txt) txt = _UNESCAPE_TEXT_REGEXP3.sub(r'\1\2\3', txt) return txt class Builder: '''Builds a tree representing a text with preprocessor directives. ''' def __init__(self): # The tree, which should be built. self._tree = [] # List of all open constructs self._open_blocks = [] # Nodes to which the open blocks have to be appended when closed self._path = [] # Nr. of open blocks when file was opened. Used for checking whether all # blocks have been closed, when file processing finishes. self._nr_prev_blocks = [] # Current node, to which content should be added self._curnode = self._tree # Current file self._curfile = None def reset(self): '''Resets the builder so that it starts to build a new tree.''' self._tree = [] self._open_blocks = [] self._path = [] self._nr_prev_blocks = [] self._curnode = self._tree self._curfile = None def handle_include(self, span, fname): '''Should be called to signalize change to new file. Args: span (tuple of int): Start and end line of the include directive or None if called the first time for the main input. fname (str): Name of the file to be included. ''' self._path.append(self._curnode) self._curnode = [] self._open_blocks.append( ('include', self._curfile, [span], fname, None)) self._curfile = fname self._nr_prev_blocks.append(len(self._open_blocks)) def handle_endinclude(self, span, fname): '''Should be called when processing of a file finished. Args: span (tuple of int): Start and end line of the include directive or None if called the first time for the main input. fname (str): Name of the file which has been included. ''' nprev_blocks = self._nr_prev_blocks.pop(-1) if len(self._open_blocks) > nprev_blocks: directive, fname, spans = self._open_blocks[-1][0:3] msg = '{0} directive still unclosed when reaching end of file'\ .format(directive) raise FyppFatalError(msg, self._curfile, spans[0]) block = self._open_blocks.pop(-1) directive, blockfname, spans = block[0:3] if directive != 'include': msg = 'internal error: last open block is not \'include\' when '\ 'closing file \'{0}\''.format(fname) raise FyppFatalError(msg) if span != spans[0]: msg = 'internal error: span for include and endinclude differ ('\ '{0} vs {1}'.format(span, spans[0]) raise FyppFatalError(msg) oldfname, _ = block[3:5] if fname != oldfname: msg = 'internal error: mismatching file name in close_file event'\ " (expected: '{0}', got: '{1}')".format(oldfname, fname) raise FyppFatalError(msg, fname) block = directive, blockfname, spans, fname, self._curnode self._curnode = self._path.pop(-1) self._curnode.append(block) self._curfile = blockfname def handle_if(self, span, cond): '''Should be called to signalize an if directive. Args: span (tuple of int): Start and end line of the directive. param (str): String representation of the branching condition. ''' self._path.append(self._curnode) self._curnode = [] self._open_blocks.append(('if', self._curfile, [span], [cond], [])) def handle_elif(self, span, cond): '''Should be called to signalize an elif directive. Args: span (tuple of int): Start and end line of the directive. cond (str): String representation of the branching condition. ''' self._check_for_open_block(span, 'elif') block = self._open_blocks[-1] directive, _, spans = block[0:3] self._check_if_matches_last(directive, 'if', spans[-1], span, 'elif') conds, contents = block[3:5] conds.append(cond) contents.append(self._curnode) spans.append(span) self._curnode = [] def handle_else(self, span): '''Should be called to signalize an else directive. Args: span (tuple of int): Start and end line of the directive. ''' self._check_for_open_block(span, 'else') block = self._open_blocks[-1] directive, _, spans = block[0:3] self._check_if_matches_last(directive, 'if', spans[-1], span, 'else') conds, contents = block[3:5] conds.append('True') contents.append(self._curnode) spans.append(span) self._curnode = [] def handle_endif(self, span): '''Should be called to signalize an endif directive. Args: span (tuple of int): Start and end line of the directive. ''' self._check_for_open_block(span, 'endif') block = self._open_blocks.pop(-1) directive, _, spans = block[0:3] self._check_if_matches_last(directive, 'if', spans[-1], span, 'endif') _, contents = block[3:5] contents.append(self._curnode) spans.append(span) self._curnode = self._path.pop(-1) self._curnode.append(block) def handle_for(self, span, loopvar, iterator): '''Should be called to signalize a for directive. Args: span (tuple of int): Start and end line of the directive. varexpr (str): String representation of the loop variable expression. iterator (str): String representation of the iterable. ''' self._path.append(self._curnode) self._curnode = [] self._open_blocks.append(('for', self._curfile, [span], loopvar, iterator, None)) def handle_endfor(self, span): '''Should be called to signalize an endfor directive. Args: span (tuple of int): Start and end line of the directive. ''' self._check_for_open_block(span, 'endfor') block = self._open_blocks.pop(-1) directive, fname, spans = block[0:3] self._check_if_matches_last(directive, 'for', spans[-1], span, 'endfor') loopvar, iterator, dummy = block[3:6] spans.append(span) block = (directive, fname, spans, loopvar, iterator, self._curnode) self._curnode = self._path.pop(-1) self._curnode.append(block) def handle_def(self, span, name, argexpr): '''Should be called to signalize a def directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the macro to be defined. argexpr (str): Macro argument definition or None ''' self._path.append(self._curnode) self._curnode = [] defblock = ('def', self._curfile, [span], name, argexpr, None) self._open_blocks.append(defblock) def handle_enddef(self, span, name): '''Should be called to signalize an enddef directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the enddef statement. Could be None, if enddef was specified without name. ''' self._check_for_open_block(span, 'enddef') block = self._open_blocks.pop(-1) directive, fname, spans = block[0:3] self._check_if_matches_last(directive, 'def', spans[-1], span, 'enddef') defname, argexpr, dummy = block[3:6] if name is not None and name != defname: msg = "wrong name in enddef directive "\ "(expected '{0}', got '{1}')".format(defname, name) raise FyppFatalError(msg, fname, span) spans.append(span) block = (directive, fname, spans, defname, argexpr, self._curnode) self._curnode = self._path.pop(-1) self._curnode.append(block) def handle_call(self, span, name, argexpr, blockcall): '''Should be called to signalize a call directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the callable to call argexpr (str or None): Argument expression containing additional arguments for the call. blockcall (bool): Whether the alternative "block / contains / endblock" calling directive has been used. ''' self._path.append(self._curnode) self._curnode = [] directive = 'block' if blockcall else 'call' self._open_blocks.append( (directive, self._curfile, [span, span], name, argexpr, [], [])) def handle_nextarg(self, span, name, blockcall): '''Should be called to signalize a nextarg directive. Args: span (tuple of int): Start and end line of the directive. name (str or None): Name of the argument following next or None if it should be the next positional argument. blockcall (bool): Whether the alternative "block / contains / endblock" calling directive has been used. ''' self._check_for_open_block(span, 'nextarg') block = self._open_blocks[-1] directive, fname, spans = block[0:3] if blockcall: opened, current = 'block', 'contains' else: opened, current = 'call', 'nextarg' self._check_if_matches_last(directive, opened, spans[-1], span, current) args, argnames = block[5:7] args.append(self._curnode) spans.append(span) if name is not None: argnames.append(name) elif argnames: msg = 'non-keyword argument following keyword argument' raise FyppFatalError(msg, fname, span) self._curnode = [] def handle_endcall(self, span, name, blockcall): '''Should be called to signalize an endcall directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the endcall statement. Could be None, if endcall was specified without name. blockcall (bool): Whether the alternative "block / contains / endblock" calling directive has been used. ''' self._check_for_open_block(span, 'endcall') block = self._open_blocks.pop(-1) directive, fname, spans = block[0:3] callname, callargexpr, args, argnames = block[3:7] if blockcall: opened, current = 'block', 'endblock' else: opened, current = 'call', 'endcall' self._check_if_matches_last(directive, opened, spans[0], span, current) if name is not None and name != callname: msg = "wrong name in {0} directive "\ "(expected '{1}', got '{2}')".format(current, callname, name) raise FyppFatalError(msg, fname, span) args.append(self._curnode) # If nextarg or endcall immediately followed call, then first argument # is empty and should be removed (to allow for calls without arguments # and named first argument in calls) if args and not args[0]: if len(argnames) == len(args): del argnames[0] del args[0] del spans[1] spans.append(span) block = (directive, fname, spans, callname, callargexpr, args, argnames) self._curnode = self._path.pop(-1) self._curnode.append(block) def handle_set(self, span, name, expr): '''Should be called to signalize a set directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the variable. expr (str): String representation of the expression to be assigned to the variable. ''' self._curnode.append(('set', self._curfile, span, name, expr)) def handle_global(self, span, name): '''Should be called to signalize a global directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the variable(s) to make global. ''' self._curnode.append(('global', self._curfile, span, name)) def handle_del(self, span, name): '''Should be called to signalize a del directive. Args: span (tuple of int): Start and end line of the directive. name (str): Name of the variable(s) to delete. ''' self._curnode.append(('del', self._curfile, span, name)) def handle_eval(self, span, expr): '''Should be called to signalize an eval directive. Args: span (tuple of int): Start and end line of the directive. expr (str): String representation of the Python expression to be evaluated. ''' self._curnode.append(('eval', self._curfile, span, expr)) def handle_comment(self, span): '''Should be called to signalize a comment directive. The content of the comment is not needed by the builder, but it needs the span of the comment to generate proper line numbers if needed. Args: span (tuple of int): Start and end line of the directive. ''' self._curnode.append(('comment', self._curfile, span)) def handle_text(self, span, txt): '''Should be called to pass text which goes to output unaltered. Args: span (tuple of int): Start and end line of the text. txt (str): Text. ''' self._curnode.append(('txt', self._curfile, span, txt)) def handle_mute(self, span): '''Should be called to signalize a mute directive. Args: span (tuple of int): Start and end line of the directive. ''' self._path.append(self._curnode) self._curnode = [] self._open_blocks.append(('mute', self._curfile, [span], None)) def handle_endmute(self, span): '''Should be called to signalize an endmute directive. Args: span (tuple of int): Start and end line of the directive. ''' self._check_for_open_block(span, 'endmute') block = self._open_blocks.pop(-1) directive, fname, spans = block[0:3] self._check_if_matches_last(directive, 'mute', spans[-1], span, 'endmute') spans.append(span) block = (directive, fname, spans, self._curnode) self._curnode = self._path.pop(-1) self._curnode.append(block) def handle_stop(self, span, msg): '''Should be called to signalize a stop directive. Args: span (tuple of int): Start and end line of the directive. ''' self._curnode.append(('stop', self._curfile, span, msg)) def handle_assert(self, span, cond): '''Should be called to signalize an assert directive. Args: span (tuple of int): Start and end line of the directive. ''' self._curnode.append(('assert', self._curfile, span, cond)) @property def tree(self): '''Returns the tree built by the Builder.''' return self._tree def _check_for_open_block(self, span, directive): if len(self._open_blocks) <= self._nr_prev_blocks[-1]: msg = 'unexpected {0} directive'.format(directive) raise FyppFatalError(msg, self._curfile, span) def _check_if_matches_last(self, lastdir, curdir, lastspan, curspan, directive): if curdir != lastdir: msg = "mismatching '{0}' directive (last block opened was '{1}')"\ .format(directive, lastdir) raise FyppFatalError(msg, self._curfile, curspan) inline_last = lastspan[0] == lastspan[1] inline_cur = curspan[0] == curspan[1] if inline_last != inline_cur: if inline_cur: msg = 'expecting line form of directive {0}'.format(directive) else: msg = 'expecting inline form of directive {0}'.format(directive) raise FyppFatalError(msg, self._curfile, curspan) elif inline_cur and curspan[0] != lastspan[0]: msg = 'inline directives of the same construct must be in the '\ 'same row' raise FyppFatalError(msg, self._curfile, curspan) class Renderer: ''''Renders a tree. Args: evaluator (Evaluator, optional): Evaluator to use when rendering eval directives. If None (default), Evaluator() is used. linenums (bool, optional): Whether linenums should be generated, defaults to False. contlinenums (bool, optional): Whether linenums for continuation should be generated, defaults to False. linenumformat (str, optional): 'std', 'cpp' or 'gfortran5' depending what kind of line directives should be created. Default: 'cpp'. Format 'std' emits #line pragmas, 'cpp' resembles GNU cpps special format, and 'gfortran5' adds to cpp a workaround for a bug introduced in GFortran 5. linefolder (callable): Callable to use when folding a line. ''' def __init__(self, evaluator=None, linenums=False, contlinenums=False, linenumformat=None, linefolder=None): # Evaluator to use for Python expressions self._evaluator = Evaluator() if evaluator is None else evaluator self._evaluator.updateglobals(_SYSTEM_=platform.system(), _MACHINE_=platform.machine()) # Whether rendered output is diverted and will be processed # further before output (if True: no line numbering and post processing) self._diverted = False # Whether file name and line numbers should be kept fixed and # not updated (typically when rendering macro content) self._fixedposition = False # Whether line numbering directives should be emitted self._linenums = linenums # Whether line numbering directives in continuation lines are needed. self._contlinenums = contlinenums # Line number formatter function and whether gfortran5 fix is needed if linenumformat is None or linenumformat in ('cpp', 'gfortran5'): self._linenumdir = linenumdir_cpp self._linenum_gfortran5 = linenumformat == 'gfortran5' else: self._linenumdir = linenumdir_std self._linenum_gfortran5 = False # Callable to be used for folding lines if linefolder is None: self._linefolder = lambda line: [line] else: self._linefolder = linefolder def render(self, tree, divert=False, fixposition=False): '''Renders a tree. Args: tree (fypp-tree): Tree to render. divert (bool): Whether output will be diverted and sent for further processing, so that no line numbering directives and postprocessing are needed at this stage. (Default: False) fixposition (bool): Whether file name and line position (variables _FILE_ and _LINE_) should be kept at their current values or should be updated continuously. (Default: False). Returns: str: Rendered string. ''' diverted = self._diverted self._diverted = divert fixedposition_old = self._fixedposition self._fixedposition = self._fixedposition or fixposition output, eval_inds, eval_pos = self._render(tree) if not self._diverted and eval_inds: self._postprocess_eval_lines(output, eval_inds, eval_pos) self._diverted = diverted self._fixedposition = fixedposition_old txt = ''.join(output) return txt def _render(self, tree): output = [] eval_inds = [] eval_pos = [] for node in tree: cmd = node[0] if cmd == 'txt': output.append(node[3]) elif cmd == 'if': out, ieval, peval = self._get_conditional_content(*node[1:5]) eval_inds += _shiftinds(ieval, len(output)) eval_pos += peval output += out elif cmd == 'eval': out, ieval, peval = self._get_eval(*node[1:4]) eval_inds += _shiftinds(ieval, len(output)) eval_pos += peval output += out elif cmd == 'def': result = self._define_macro(*node[1:6]) output.append(result) elif cmd == 'set': result = self._define_variable(*node[1:5]) output.append(result) elif cmd == 'del': self._delete_variable(*node[1:4]) elif cmd == 'for': out, ieval, peval = self._get_iterated_content(*node[1:6]) eval_inds += _shiftinds(ieval, len(output)) eval_pos += peval output += out elif cmd == 'call' or cmd == 'block': out, ieval, peval = self._get_called_content(*node[1:7]) eval_inds += _shiftinds(ieval, len(output)) eval_pos += peval output += out elif cmd == 'include': out, ieval, peval = self._get_included_content(*node[1:5]) eval_inds += _shiftinds(ieval, len(output)) eval_pos += peval output += out elif cmd == 'comment': output.append(self._get_comment(*node[1:3])) elif cmd == 'mute': output.append(self._get_muted_content(*node[1:4])) elif cmd == 'stop': self._handle_stop(*node[1:4]) elif cmd == 'assert': result = self._handle_assert(*node[1:4]) output.append(result) elif cmd == 'global': self._add_global(*node[1:4]) else: msg = "internal error: unknown command '{0}'".format(cmd) raise FyppFatalError(msg) return output, eval_inds, eval_pos def _get_eval(self, fname, span, expr): try: result = self._evaluate(expr, fname, span[0]) except Exception as exc: msg = "exception occurred when evaluating '{0}'".format(expr) raise FyppFatalError(msg, fname, span) from exc out = [] ieval = [] peval = [] if result is not None: out.append(str(result)) if not self._diverted: ieval.append(0) peval.append((span, fname)) if span[0] != span[1]: out.append('\n') return out, ieval, peval def _get_conditional_content(self, fname, spans, conditions, contents): out = [] ieval = [] peval = [] multiline = (spans[0][0] != spans[-1][1]) for condition, content, span in zip(conditions, contents, spans): try: cond = bool(self._evaluate(condition, fname, span[0])) except Exception as exc: msg = "exception occurred when evaluating '{0}'"\ .format(condition) raise FyppFatalError(msg, fname, span) from exc if cond: if self._linenums and not self._diverted and multiline: out.append(self._linenumdir(span[1], fname)) outcont, ievalcont, pevalcont = self._render(content) ieval += _shiftinds(ievalcont, len(out)) peval += pevalcont out += outcont break if self._linenums and not self._diverted and multiline: out.append(self._linenumdir(spans[-1][1], fname)) return out, ieval, peval def _get_iterated_content(self, fname, spans, loopvars, loopiter, content): out = [] ieval = [] peval = [] try: iterobj = iter(self._evaluate(loopiter, fname, spans[0][0])) except Exception as exc: msg = "exception occurred when evaluating '{0}'"\ .format(loopiter) raise FyppFatalError(msg, fname, spans[0]) from exc multiline = (spans[0][0] != spans[-1][1]) for var in iterobj: if len(loopvars) == 1: self._define(loopvars[0], var) else: for varname, value in zip(loopvars, var): self._define(varname, value) if self._linenums and not self._diverted and multiline: out.append(self._linenumdir(spans[0][1], fname)) outcont, ievalcont, pevalcont = self._render(content) ieval += _shiftinds(ievalcont, len(out)) peval += pevalcont out += outcont if self._linenums and not self._diverted and multiline: out.append(self._linenumdir(spans[1][1], fname)) return out, ieval, peval def _get_called_content(self, fname, spans, name, argexpr, contents, argnames): posargs, kwargs = self._get_call_arguments(fname, spans, argexpr, contents, argnames) try: callobj = self._evaluate(name, fname, spans[0][0]) result = callobj(*posargs, **kwargs) except Exception as exc: msg = "exception occurred when calling '{0}'".format(name) raise FyppFatalError(msg, fname, spans[0]) from exc self._update_predef_globals(fname, spans[0][0]) span = (spans[0][0], spans[-1][1]) out = [] ieval = [] peval = [] if result is not None: out = [str(result)] if not self._diverted: ieval = [0] peval = [(span, fname)] if span[0] != span[1]: out.append('\n') return out, ieval, peval def _get_call_arguments(self, fname, spans, argexpr, contents, argnames): if argexpr is None: posargs = [] kwargs = {} else: # Parse and evaluate arguments passed in call header self._evaluator.openscope() try: posargs, kwargs = self._evaluate( '__getargvalues(' + argexpr + ')', fname, spans[0][0]) except Exception as exc: msg = "unable to parse argument expression '{0}'"\ .format(argexpr) raise FyppFatalError(msg, fname, spans[0]) from exc self._evaluator.closescope() # Render arguments passed in call body args = [] for content in contents: self._evaluator.openscope() rendered = self.render(content, divert=True) self._evaluator.closescope() if rendered.endswith('\n'): rendered = rendered[:-1] args.append(rendered) # Separate arguments in call body into positional and keyword ones: if argnames: posargs += args[:len(args) - len(argnames)] offset = len(args) - len(argnames) for iargname, argname in enumerate(argnames): ind = offset + iargname if argname in kwargs: msg = "keyword argument '{0}' already defined"\ .format(argname) raise FyppFatalError(msg, fname, spans[ind + 1]) kwargs[argname] = args[ind] else: posargs += args return posargs, kwargs def _get_included_content(self, fname, spans, includefname, content): includefile = spans[0] is not None out = [] if self._linenums and not self._diverted: if includefile or self._linenum_gfortran5: out += self._linenumdir(0, includefname, _LINENUM_NEW_FILE) else: out += self._linenumdir(0, includefname) outcont, ieval, peval = self._render(content) ieval = _shiftinds(ieval, len(out)) out += outcont if self._linenums and not self._diverted and includefile: out += self._linenumdir(spans[0][1], fname, _LINENUM_RETURN_TO_FILE) return out, ieval, peval def _define_macro(self, fname, spans, name, argexpr, content): if argexpr is None: args = [] defaults = {} varpos = None varkw = None else: # Try to create a lambda function with the argument expression self._evaluator.openscope() lambdaexpr = 'lambda ' + argexpr + ': None' try: func = self._evaluate(lambdaexpr, fname, spans[0][0]) except Exception as exc: msg = "exception occurred when evaluating argument expression "\ "'{0}'".format(argexpr) raise FyppFatalError(msg, fname, spans[0]) from exc self._evaluator.closescope() try: args, defaults, varpos, varkw = _get_callable_argspec(func) except Exception as exc: msg = "invalid argument expression '{0}'".format(argexpr) raise FyppFatalError(msg, fname, spans[0]) from exc named_args = args if varpos is None else args + [varpos] named_args = named_args if varkw is None else named_args + [varkw] for arg in named_args: if arg in _RESERVED_NAMES or arg.startswith(_RESERVED_PREFIX): msg = "invalid argument name '{0}'".format(arg) raise FyppFatalError(msg, fname, spans[0]) result = '' try: macro = _Macro( name, fname, spans, args, defaults, varpos, varkw, content, self, self._evaluator, self._evaluator.localscope) self._define(name, macro) except Exception as exc: msg = "exception occurred when defining macro '{0}'"\ .format(name) raise FyppFatalError(msg, fname, spans[0]) from exc if self._linenums and not self._diverted: result = self._linenumdir(spans[1][1], fname) return result def _define_variable(self, fname, span, name, valstr): result = '' try: if valstr is None: expr = None else: expr = self._evaluate(valstr, fname, span[0]) self._define(name, expr) except Exception as exc: msg = "exception occurred when setting variable(s) '{0}' to '{1}'"\ .format(name, valstr) raise FyppFatalError(msg, fname, span) from exc multiline = (span[0] != span[1]) if self._linenums and not self._diverted and multiline: result = self._linenumdir(span[1], fname) return result def _delete_variable(self, fname, span, name): result = '' try: self._evaluator.undefine(name) except Exception as exc: msg = "exception occurred when deleting variable(s) '{0}'"\ .format(name) raise FyppFatalError(msg, fname, span) from exc multiline = (span[0] != span[1]) if self._linenums and not self._diverted and multiline: result = self._linenumdir(span[1], fname) return result def _add_global(self, fname, span, name): result = '' try: self._evaluator.addglobal(name) except Exception as exc: msg = "exception occurred when making variable(s) '{0}' global"\ .format(name) raise FyppFatalError(msg, fname, span) from exc multiline = (span[0] != span[1]) if self._linenums and not self._diverted and multiline: result = self._linenumdir(span[1], fname) return result def _get_comment(self, fname, span): if self._linenums and not self._diverted: return self._linenumdir(span[1], fname) return '' def _get_muted_content(self, fname, spans, content): self._render(content) if self._linenums and not self._diverted: return self._linenumdir(spans[-1][1], fname) return '' def _handle_stop(self, fname, span, msgstr): try: msg = str(self._evaluate(msgstr, fname, span[0])) except Exception as exc: msg = "exception occurred when evaluating stop message '{0}'"\ .format(msgstr) raise FyppFatalError(msg, fname, span) from exc raise FyppStopRequest(msg, fname, span) def _handle_assert(self, fname, span, expr): result = '' try: cond = bool(self._evaluate(expr, fname, span[0])) except Exception as exc: msg = "exception occurred when evaluating assert condition '{0}'"\ .format(expr) raise FyppFatalError(msg, fname, span) from exc if not cond: msg = "Assertion failed ('{0}')".format(expr) raise FyppStopRequest(msg, fname, span) if self._linenums and not self._diverted: result = self._linenumdir(span[1], fname) return result def _evaluate(self, expr, fname, linenr): self._update_predef_globals(fname, linenr) result = self._evaluator.evaluate(expr) self._update_predef_globals(fname, linenr) return result def _update_predef_globals(self, fname, linenr): self._evaluator.updatelocals( _DATE_=time.strftime('%Y-%m-%d'), _TIME_=time.strftime('%H:%M:%S'), _THIS_FILE_=fname, _THIS_LINE_=linenr + 1) if not self._fixedposition: self._evaluator.updateglobals(_FILE_=fname, _LINE_=linenr + 1) def _define(self, var, value): self._evaluator.define(var, value) def _postprocess_eval_lines(self, output, eval_inds, eval_pos): ilastproc = -1 for ieval, ind in enumerate(eval_inds): span, fname = eval_pos[ieval] if ind <= ilastproc: continue iprev, eolprev = self._find_last_eol(output, ind) inext, eolnext = self._find_next_eol(output, ind) curline = self._glue_line(output, ind, iprev, eolprev, inext, eolnext) output[iprev + 1:inext] = [''] * (inext - iprev - 1) output[ind] = self._postprocess_eval_line(curline, fname, span) ilastproc = inext @staticmethod def _find_last_eol(output, ind): 'Find last newline before current position.' iprev = ind - 1 while iprev >= 0: eolprev = output[iprev].rfind('\n') if eolprev != -1: break iprev -= 1 else: iprev = 0 eolprev = -1 return iprev, eolprev @staticmethod def _find_next_eol(output, ind): 'Find last newline before current position.' # find first eol after expr. evaluation inext = ind + 1 while inext < len(output): eolnext = output[inext].find('\n') if eolnext != -1: break inext += 1 else: inext = len(output) - 1 eolnext = len(output[-1]) - 1 return inext, eolnext @staticmethod def _glue_line(output, ind, iprev, eolprev, inext, eolnext): 'Create line from parts between specified boundaries.' curline_parts = [] if iprev != ind: curline_parts = [output[iprev][eolprev + 1:]] output[iprev] = output[iprev][:eolprev + 1] curline_parts.extend(output[iprev + 1:ind]) curline_parts.extend(output[ind]) curline_parts.extend(output[ind + 1:inext]) if inext != ind: curline_parts.append(output[inext][:eolnext + 1]) output[inext] = output[inext][eolnext + 1:] return ''.join(curline_parts) def _postprocess_eval_line(self, evalline, fname, span): lines = evalline.split('\n') # If line ended on '\n', last element is ''. We remove it and # add the trailing newline later manually. trailing_newline = (lines[-1] == '') if trailing_newline: del lines[-1] lnum = self._linenumdir(span[0], fname) if self._linenums else '' clnum = lnum if self._contlinenums else '' linenumsep = '\n' + lnum clinenumsep = '\n' + clnum foldedlines = [self._foldline(line) for line in lines] outlines = [clinenumsep.join(lines) for lines in foldedlines] result = linenumsep.join(outlines) # Add missing trailing newline if trailing_newline: trailing = '\n' if self._linenums: # Last line was folded, but no linenums were generated for # the continuation lines -> current line position is not # in sync with the one calculated from the last line number unsync = ( len(foldedlines) and len(foldedlines[-1]) > 1 and not self._contlinenums) # Eval directive in source consists of more than one line multiline = span[1] - span[0] > 1 if unsync or multiline: # For inline eval directives span[0] == span[1] # -> next line is span[0] + 1 and not span[1] as for # line eval directives nextline = max(span[1], span[0] + 1) trailing += self._linenumdir(nextline, fname) else: trailing = '' return result + trailing def _foldline(self, line): if _COMMENTLINE_REGEXP.match(line) is None: return self._linefolder(line) return [line] class Evaluator: '''Provides an isolated environment for evaluating Python expressions. It restricts the builtins which can be used within this environment to a (hopefully safe) subset. Additionally it defines the functions which are provided by the preprocessor for the eval directives. Args: env (dict, optional): Initial definitions for the environment, defaults to None. ''' # Restricted builtins working in all supported Python verions. Version # specific ones are added dynamically in _get_restricted_builtins(). _RESTRICTED_BUILTINS = { 'abs': builtins.abs, 'all': builtins.all, 'any': builtins.any, 'bin': builtins.bin, 'bool': builtins.bool, 'bytearray': builtins.bytearray, 'bytes': builtins.bytes, 'chr': builtins.chr, 'classmethod': builtins.classmethod, 'complex': builtins.complex, 'delattr': builtins.delattr, 'dict': builtins.dict, 'dir': builtins.dir, 'divmod': builtins.divmod, 'enumerate': builtins.enumerate, 'filter': builtins.filter, 'float': builtins.float, 'format': builtins.format, 'frozenset': builtins.frozenset, 'getattr': builtins.getattr, 'globals': builtins.globals, 'hasattr': builtins.hasattr, 'hash': builtins.hash, 'hex': builtins.hex, 'id': builtins.id, 'int': builtins.int, 'isinstance': builtins.isinstance, 'issubclass': builtins.issubclass, 'iter': builtins.iter, 'len': builtins.len, 'list': builtins.list, 'locals': builtins.locals, 'map': builtins.map, 'max': builtins.max, 'min': builtins.min, 'next': builtins.next, 'object': builtins.object, 'oct': builtins.oct, 'ord': builtins.ord, 'pow': builtins.pow, 'property': builtins.property, 'range': builtins.range, 'repr': builtins.repr, 'reversed': builtins.reversed, 'round': builtins.round, 'set': builtins.set, 'setattr': builtins.setattr, 'slice': builtins.slice, 'sorted': builtins.sorted, 'staticmethod': builtins.staticmethod, 'str': builtins.str, 'sum': builtins.sum, 'super': builtins.super, 'tuple': builtins.tuple, 'type': builtins.type, 'vars': builtins.vars, 'zip': builtins.zip, } def __init__(self, env=None): # Global scope self._globals = env if env is not None else {} # Local scope(s) self._locals = None self._locals_stack = [] # Variables which are references to entries in global scope self._globalrefs = None self._globalrefs_stack = [] # Current scope (globals + locals in all embedding and in current scope) self._scope = self._globals # Turn on restricted mode self._restrict_builtins() def evaluate(self, expr): '''Evaluate a Python expression using the `eval()` builtin. Args: expr (str): String represantion of the expression. Return: Python object: Result of the expression evaluation. ''' result = eval(expr, self._scope) return result def import_module(self, module): '''Import a module into the evaluator. Note: Import only trustworthy modules! Module imports are global, therefore, importing a malicious module which manipulates other global modules could affect code behaviour outside of the Evaluator as well. Args: module (str): Python module to import. Raises: FyppFatalError: If module could not be imported. ''' rootmod = module.split('.', 1)[0] try: imported = __import__(module, self._scope) self.define(rootmod, imported) except Exception as exc: msg = "failed to import module '{0}'".format(module) raise FyppFatalError(msg) from exc def define(self, name, value): '''Define a Python entity. Args: name (str): Name of the entity. value (Python object): Value of the entity. Raises: FyppFatalError: If name starts with the reserved prefix or if it is a reserved name. ''' varnames = self._get_variable_names(name) if len(varnames) == 1: value = (value,) elif len(varnames) != len(value): msg = 'value for tuple assignment has incompatible length' raise FyppFatalError(msg) for varname, varvalue in zip(varnames, value): self._check_variable_name(varname) if self._locals is None: self._globals[varname] = varvalue else: if varname in self._globalrefs: self._globals[varname] = varvalue else: self._locals[varname] = varvalue self._scope[varname] = varvalue def undefine(self, name): '''Undefine a Python entity. Args: name (str): Name of the entity to undefine. Raises: FyppFatalError: If name starts with the reserved prefix or if it is a reserved name. ''' varnames = self._get_variable_names(name) for varname in varnames: self._check_variable_name(varname) deleted = False if self._locals is None: if varname in self._globals: del self._globals[varname] deleted = True else: if varname in self._locals: del self._locals[varname] del self._scope[varname] deleted = True elif varname in self._globalrefs and varname in self._globals: del self._globals[varname] del self._scope[varname] deleted = True if not deleted: msg = "lookup for an erasable instance of '{0}' failed"\ .format(varname) raise FyppFatalError(msg) def addglobal(self, name): '''Define a given entity as global. Args: name (str): Name of the entity to make global. Raises: FyppFatalError: If entity name is invalid or if the current scope is a local scope and entity is already defined in it. ''' varnames = self._get_variable_names(name) for varname in varnames: self._check_variable_name(varname) if self._locals is not None: if varname in self._locals: msg = "variable '{0}' already defined in local scope"\ .format(varname) raise FyppFatalError(msg) self._globalrefs.add(varname) def updateglobals(self, **vardict): '''Update variables in the global scope. This is a shortcut function to inject protected variables in the global scope without extensive checks (as in define()). Vardict must not contain any global entries which can be shadowed in local scopes (e.g. should only contain variables with forbidden prefix). Args: **vardict: variable definitions. ''' self._scope.update(vardict) if self._locals is not None: self._globals.update(vardict) def updatelocals(self, **vardict): '''Update variables in the local scope. This is a shortcut function to inject variables in the local scope without extensive checks (as in define()). Vardict must not contain any entries which have been made global via addglobal() before. In order to ensure this, updatelocals() should be called immediately after openscope(), or with variable names, which are warrantedly not globals (e.g variables starting with forbidden prefix) Args: **vardict: variable definitions. ''' self._scope.update(vardict) if self._locals is not None: self._locals.update(vardict) def openscope(self, customlocals=None): '''Opens a new (embedded) scope. Args: customlocals (dict): By default, the locals of the embedding scope are visible in the new one. When this is not the desired behaviour a dictionary of customized locals can be passed, and those locals will become the only visible ones. ''' self._locals_stack.append(self._locals) self._globalrefs_stack.append(self._globalrefs) if customlocals is not None: self._locals = customlocals.copy() elif self._locals is not None: self._locals = self._locals.copy() else: self._locals = {} self._globalrefs = set() self._scope = self._globals.copy() self._scope.update(self._locals) def closescope(self): '''Close scope and restore embedding scope.''' self._locals = self._locals_stack.pop(-1) self._globalrefs = self._globalrefs_stack.pop(-1) if self._locals is not None: self._scope = self._globals.copy() self._scope.update(self._locals) else: self._scope = self._globals @property def globalscope(self): 'Dictionary of the global scope.' return self._globals @property def localscope(self): 'Dictionary of the current local scope.' return self._locals def _restrict_builtins(self): builtindict = self._get_restricted_builtins() builtindict['__import__'] = self._func_import builtindict['defined'] = self._func_defined builtindict['setvar'] = self._func_setvar builtindict['getvar'] = self._func_getvar builtindict['delvar'] = self._func_delvar builtindict['globalvar'] = self._func_globalvar builtindict['__getargvalues'] = self._func_getargvalues self._globals['__builtins__'] = builtindict @classmethod def _get_restricted_builtins(cls): bidict = dict(cls._RESTRICTED_BUILTINS) return bidict @staticmethod def _get_variable_names(varexpr): lpar = varexpr.startswith('(') rpar = varexpr.endswith(')') if lpar != rpar: msg = "unbalanced parenthesis around variable varexpr(s) in '{0}'"\ .format(varexpr) raise FyppFatalError(msg, None, None) if lpar: varexpr = varexpr[1:-1] varnames = [s.strip() for s in varexpr.split(',')] return varnames @staticmethod def _check_variable_name(varname): if varname.startswith(_RESERVED_PREFIX): msg = "Name '{0}' starts with reserved prefix '{1}'"\ .format(varname, _RESERVED_PREFIX) raise FyppFatalError(msg, None, None) if varname in _RESERVED_NAMES: msg = "Name '{0}' is reserved and can not be redefined"\ .format(varname) raise FyppFatalError(msg, None, None) def _func_defined(self, var): defined = var in self._scope return defined def _func_import(self, name, *_, **__): module = self._scope.get(name, None) if module is not None and isinstance(module, types.ModuleType): return module msg = "Import of module '{0}' via '__import__' not allowed".format(name) raise ImportError(msg) def _func_setvar(self, *namesvalues): if len(namesvalues) % 2: msg = 'setvar function needs an even number of arguments' raise FyppFatalError(msg) for ind in range(0, len(namesvalues), 2): self.define(namesvalues[ind], namesvalues[ind + 1]) def _func_getvar(self, name, defvalue=None): if name in self._scope: return self._scope[name] return defvalue def _func_delvar(self, *names): for name in names: self.undefine(name) def _func_globalvar(self, *names): for name in names: self.addglobal(name) @staticmethod def _func_getargvalues(*args, **kwargs): return list(args), kwargs class _Macro: '''Represents a user defined macro. This object should only be initiatied by a Renderer instance, as it needs access to Renderers internal variables and methods. Args: name (str): Name of the macro. fname (str): The file where the macro was defined. spans (str): Line spans of macro definition. argnames (list of str): Macro dummy arguments. varpos (str): Name of variable positional argument or None. varkw (str): Name of variable keyword argument or None. content (list): Content of the macro as tree. renderer (Renderer): Renderer to use for evaluating macro content. localscope (dict): Dictionary with local variables, which should be used the local scope, when the macro is called. Default: None (empty local scope). ''' def __init__(self, name, fname, spans, argnames, defaults, varpos, varkw, content, renderer, evaluator, localscope=None): self._name = name self._fname = fname self._spans = spans self._argnames = argnames self._defaults = defaults self._varpos = varpos self._varkw = varkw self._content = content self._renderer = renderer self._evaluator = evaluator self._localscope = localscope if localscope is not None else {} def __call__(self, *args, **keywords): argdict = self._process_arguments(args, keywords) self._evaluator.openscope(customlocals=self._localscope) self._evaluator.updatelocals(**argdict) output = self._renderer.render(self._content, divert=True, fixposition=True) self._evaluator.closescope() if output.endswith('\n'): return output[:-1] return output def _process_arguments(self, args, keywords): kwdict = dict(keywords) argdict = {} nargs = min(len(args), len(self._argnames)) for iarg in range(nargs): argdict[self._argnames[iarg]] = args[iarg] if nargs < len(args): if self._varpos is None: msg = "macro '{0}' called with too many positional arguments "\ "(expected: {1}, received: {2})"\ .format(self._name, len(self._argnames), len(args)) raise FyppFatalError(msg, self._fname, self._spans[0]) else: argdict[self._varpos] = list(args[nargs:]) elif self._varpos is not None: argdict[self._varpos] = [] for argname in self._argnames[:nargs]: if argname in kwdict: msg = "got multiple values for argument '{0}'".format(argname) raise FyppFatalError(msg, self._fname, self._spans[0]) if nargs < len(self._argnames): for argname in self._argnames[nargs:]: if argname in kwdict: argdict[argname] = kwdict.pop(argname) elif argname in self._defaults: argdict[argname] = self._defaults[argname] else: msg = "macro '{0}' called without mandatory positional "\ "argument '{1}'".format(self._name, argname) raise FyppFatalError(msg, self._fname, self._spans[0]) if kwdict and self._varkw is None: kwstr = "', '".join(kwdict.keys()) msg = "macro '{0}' called with unknown keyword argument(s) '{1}'"\ .format(self._name, kwstr) raise FyppFatalError(msg, self._fname, self._spans[0]) if self._varkw is not None: argdict[self._varkw] = kwdict return argdict class Processor: '''Connects various objects with each other to create a processor. Args: parser (Parser, optional): Parser to use for parsing text. If None (default), `Parser()` is used. builder (Builder, optional): Builder to use for building the tree representation of the text. If None (default), `Builder()` is used. renderer (Renderer, optional): Renderer to use for rendering the output. If None (default), `Renderer()` is used with a default Evaluator(). evaluator (Evaluator, optional): Evaluator to use for evaluating Python expressions. If None (default), `Evaluator()` is used. ''' def __init__(self, parser=None, builder=None, renderer=None, evaluator=None): self._parser = Parser() if parser is None else parser self._builder = Builder() if builder is None else builder if renderer is None: evaluator = Evaluator() if evaluator is None else evaluator self._renderer = Renderer(evaluator) else: self._renderer = renderer self._parser.handle_include = self._builder.handle_include self._parser.handle_endinclude = self._builder.handle_endinclude self._parser.handle_if = self._builder.handle_if self._parser.handle_else = self._builder.handle_else self._parser.handle_elif = self._builder.handle_elif self._parser.handle_endif = self._builder.handle_endif self._parser.handle_eval = self._builder.handle_eval self._parser.handle_text = self._builder.handle_text self._parser.handle_def = self._builder.handle_def self._parser.handle_enddef = self._builder.handle_enddef self._parser.handle_set = self._builder.handle_set self._parser.handle_del = self._builder.handle_del self._parser.handle_global = self._builder.handle_global self._parser.handle_for = self._builder.handle_for self._parser.handle_endfor = self._builder.handle_endfor self._parser.handle_call = self._builder.handle_call self._parser.handle_nextarg = self._builder.handle_nextarg self._parser.handle_endcall = self._builder.handle_endcall self._parser.handle_comment = self._builder.handle_comment self._parser.handle_mute = self._builder.handle_mute self._parser.handle_endmute = self._builder.handle_endmute self._parser.handle_stop = self._builder.handle_stop self._parser.handle_assert = self._builder.handle_assert def process_file(self, fname): '''Processeses a file. Args: fname (str): Name of the file to process. Returns: str: Processed content. ''' self._parser.parsefile(fname) return self._render() def process_text(self, txt): '''Processes a string. Args: txt (str): Text to process. Returns: str: Processed content. ''' self._parser.parse(txt) return self._render() def _render(self): output = self._renderer.render(self._builder.tree) self._builder.reset() return ''.join(output) class Fypp: '''Fypp preprocessor. You can invoke it like :: tool = fypp.Fypp() tool.process_file('file.in', 'file.out') to initialize Fypp with default options, process `file.in` and write the result to `file.out`. If the input should be read from a string, the ``process_text()`` method can be used:: tool = fypp.Fypp() output = tool.process_text('#:if DEBUG > 0\\nprint *, "DEBUG"\\n#:endif\\n') If you want to fine tune Fypps behaviour, pass a customized `FyppOptions`_ instance at initialization:: options = fypp.FyppOptions() options.fixed_format = True tool = fypp.Fypp(options) Alternatively, you can use the command line parser ``optparse.OptionParser`` to set options for Fypp. The function ``get_option_parser()`` returns you a default option parser. You can then use its ``parse_args()`` method to obtain settings by reading the command line arguments:: optparser = fypp.get_option_parser() options, leftover = optparser.parse_args() tool = fypp.Fypp(options) The command line options can also be passed directly as a list when calling ``parse_args()``:: args = ['-DDEBUG=0', 'input.fpp', 'output.f90'] optparser = fypp.get_option_parser() options, leftover = optparser.parse_args(args=args) tool = fypp.Fypp(options) For even more fine-grained control over how Fypp works, you can pass in custom factory methods that handle construction of the evaluator, parser, builder and renderer components. These factory methods must have the same signature as the corresponding component's constructor. As an example of using a builder that's customized by subclassing:: class MyBuilder(fypp.Builder): def __init__(self): super().__init__() ...additional initialization... tool = fypp.Fypp(options, builder_factory=MyBuilder) Args: options (object): Object containing the settings for Fypp. You typically would pass a customized `FyppOptions`_ instance or an ``optparse.Values`` object as returned by the option parser. If not present, the default settings in `FyppOptions`_ are used. evaluator_factory (function): Factory function that returns an Evaluator object. Its call signature must match that of the Evaluator constructor. If not present, ``Evaluator`` is used. parser_factory (function): Factory function that returns a Parser object. Its call signature must match that of the Parser constructor. If not present, ``Parser`` is used. builder_factory (function): Factory function that returns a Builder object. Its call signature must match that of the Builder constructor. If not present, ``Builder`` is used. renderer_factory (function): Factory function that returns a Renderer object. Its call signature must match that of the Renderer constructor. If not present, ``Renderer`` is used. ''' def __init__(self, options=None, evaluator_factory=Evaluator, parser_factory=Parser, builder_factory=Builder, renderer_factory=Renderer): syspath = self._get_syspath_without_scriptdir() self._adjust_syspath(syspath) if options is None: options = FyppOptions() if inspect.signature(evaluator_factory) == inspect.signature(Evaluator): evaluator = evaluator_factory() else: raise FyppFatalError('evaluator_factory has incorrect signature') self._encoding = options.encoding if options.modules: self._import_modules(options.modules, evaluator, syspath, options.moduledirs) if options.defines: self._apply_definitions(options.defines, evaluator) if inspect.signature(parser_factory) == inspect.signature(Parser): parser = parser_factory(includedirs=options.includes, encoding=self._encoding) else: raise FyppFatalError('parser_factory has incorrect signature') if inspect.signature(builder_factory) == inspect.signature(Builder): builder = builder_factory() else: raise FyppFatalError('builder_factory has incorrect signature') fixed_format = options.fixed_format linefolding = not options.no_folding if linefolding: folding = 'brute' if fixed_format else options.folding_mode linelength = 72 if fixed_format else options.line_length indentation = 5 if fixed_format else options.indentation prefix = '&' suffix = '' if fixed_format else '&' linefolder = FortranLineFolder(linelength, indentation, folding, prefix, suffix) else: linefolder = DummyLineFolder() linenums = options.line_numbering contlinenums = (options.line_numbering_mode != 'nocontlines') self._create_parent_folder = options.create_parent_folder if inspect.signature(renderer_factory) == inspect.signature(Renderer): renderer = renderer_factory( evaluator, linenums=linenums, contlinenums=contlinenums, linenumformat=options.line_marker_format, linefolder=linefolder) else: raise FyppFatalError('renderer_factory has incorrect signature') self._preprocessor = Processor(parser, builder, renderer) def process_file(self, infile, outfile=None): '''Processes input file and writes result to output file. Args: infile (str): Name of the file to read and process. If its value is '-', input is read from stdin. outfile (str, optional): Name of the file to write the result to. If its value is '-', result is written to stdout. If not present, result will be returned as string. env (dict, optional): Additional definitions for the evaluator. Returns: str: Result of processed input, if no outfile was specified. ''' infile = STDIN if infile == '-' else infile output = self._preprocessor.process_file(infile) if outfile is None: return output if outfile == '-': outfile = sys.stdout else: outfile = _open_output_file(outfile, self._encoding, self._create_parent_folder) outfile.write(output) if outfile != sys.stdout: outfile.close() return None def process_text(self, txt): '''Processes a string. Args: txt (str): String to process. env (dict, optional): Additional definitions for the evaluator. Returns: str: Processed content. ''' return self._preprocessor.process_text(txt) @staticmethod def _apply_definitions(defines, evaluator): for define in defines: words = define.split('=', 2) name = words[0] value = None if len(words) > 1: try: value = evaluator.evaluate(words[1]) except Exception as exc: msg = "exception at evaluating '{0}' in definition for " \ "'{1}'".format(words[1], name) raise FyppFatalError(msg) from exc evaluator.define(name, value) def _import_modules(self, modules, evaluator, syspath, moduledirs): lookuppath = [] if moduledirs is not None: lookuppath += [os.path.abspath(moddir) for moddir in moduledirs] lookuppath.append(os.path.abspath('.')) lookuppath += syspath self._adjust_syspath(lookuppath) for module in modules: evaluator.import_module(module) self._adjust_syspath(syspath) @staticmethod def _get_syspath_without_scriptdir(): '''Remove the folder of the fypp binary from the search path''' syspath = list(sys.path) scriptdir = os.path.abspath(os.path.dirname(sys.argv[0])) if os.path.abspath(syspath[0]) == scriptdir: del syspath[0] return syspath @staticmethod def _adjust_syspath(syspath): sys.path = syspath class FyppOptions(optparse.Values): '''Container for Fypp options with default values. Attributes: defines (list of str): List of variable definitions in the form of 'VARNAME=VALUE'. Default: [] includes (list of str): List of paths to search when looking for include files. Default: [] line_numbering (bool): Whether line numbering directives should appear in the output. Default: False line_numbering_mode (str): Line numbering mode 'full' or 'nocontlines'. Default: 'full'. line_marker_format (str): Line marker format. Currently 'std', 'cpp' and 'gfortran5' are supported, where 'std' emits ``#line`` pragmas similar to standard tools, 'cpp' produces line directives as emitted by GNU cpp, and 'gfortran5' cpp line directives with a workaround for a bug introduced in GFortran 5. Default: 'cpp'. line_length (int): Length of output lines. Default: 132. folding_mode (str): Folding mode 'smart', 'simple' or 'brute'. Default: 'smart'. no_folding (bool): Whether folding should be suppressed. Default: False. indentation (int): Indentation in continuation lines. Default: 4. modules (list of str): Modules to import at initialization. Default: []. moduledirs (list of str): Module lookup directories for importing user specified modules. The specified paths are looked up *before* the standard module locations in sys.path. fixed_format (bool): Whether input file is in fixed format. Default: False. encoding (str): Character encoding for reading/writing files. Allowed values are Pythons codec identifiers, e.g. 'ascii', 'utf-8', etc. Default: 'utf-8'. Reading from stdin and writing to stdout is always encoded according to the current locale and is not affected by this setting. create_parent_folder (bool): Whether the parent folder for the output file should be created if it does not exist. Default: False. ''' def __init__(self): optparse.Values.__init__(self) self.defines = [] self.includes = [] self.line_numbering = False self.line_numbering_mode = 'full' self.line_marker_format = 'cpp' self.line_length = 132 self.folding_mode = 'smart' self.no_folding = False self.indentation = 4 self.modules = [] self.moduledirs = [] self.fixed_format = False self.encoding = 'utf-8' self.create_parent_folder = False class FortranLineFolder: '''Implements line folding with Fortran continuation lines. Args: maxlen (int, optional): Maximal line length (default: 132). indent (int, optional): Indentation for continuation lines (default: 4). method (str, optional): Folding method with following options: * ``brute``: folding with maximal length of continuation lines, * ``simple``: indents with respect of indentation of first line, * ``smart``: like ``simple``, but tries to fold at whitespaces. prefix (str, optional): String to use at the beginning of a continuation line (default: '&'). suffix (str, optional): String to use at the end of the line preceding a continuation line (default: '&') ''' def __init__(self, maxlen=132, indent=4, method='smart', prefix='&', suffix='&'): # Line length should be long enough that contintuation lines can host at # east one character apart of indentation and two continuation signs minmaxlen = indent + len(prefix) + len(suffix) + 1 if maxlen < minmaxlen: msg = 'Maximal line length less than {0} when using an indentation'\ ' of {1}'.format(minmaxlen, indent) raise FyppFatalError(msg) self._maxlen = maxlen self._indent = indent self._prefix = ' ' * self._indent + prefix self._suffix = suffix if method not in ['brute', 'smart', 'simple']: raise FyppFatalError('invalid folding type') if method == 'brute': self._inherit_indent = False self._fold_position_finder = self._get_maximal_fold_pos elif method == 'simple': self._inherit_indent = True self._fold_position_finder = self._get_maximal_fold_pos elif method == 'smart': self._inherit_indent = True self._fold_position_finder = self._get_smart_fold_pos def __call__(self, line): '''Folds a line. Can be directly called to return the list of folded lines:: linefolder = FortranLineFolder(maxlen=10) linefolder(' print *, "some Fortran line"') Args: line (str): Line to fold. Returns: list of str: Components of folded line. They should be assembled via ``\\n.join()`` to obtain the string representation. ''' if self._maxlen < 0 or len(line) <= self._maxlen: return [line] if self._inherit_indent: indent = len(line) - len(line.lstrip()) prefix = ' ' * indent + self._prefix else: indent = 0 prefix = self._prefix suffix = self._suffix return self._split_line(line, self._maxlen, prefix, suffix, self._fold_position_finder) @staticmethod def _split_line(line, maxlen, prefix, suffix, fold_position_finder): # length of continuation lines with 1 or two continuation chars. maxlen1 = maxlen - len(prefix) maxlen2 = maxlen1 - len(suffix) start = 0 end = fold_position_finder(line, start, maxlen - len(suffix)) result = [line[start:end] + suffix] while end < len(line) - maxlen1: start = end end = fold_position_finder(line, start, start + maxlen2) result.append(prefix + line[start:end] + suffix) result.append(prefix + line[end:]) return result @staticmethod def _get_maximal_fold_pos(_, __, end): return end @staticmethod def _get_smart_fold_pos(line, start, end): linelen = end - start ispace = line.rfind(' ', start, end) # The space we waste for smart folding should be max. 1/3rd of the line if ispace != -1 and ispace >= start + (2 * linelen) // 3: return ispace return end class DummyLineFolder: '''Implements a dummy line folder returning the line unaltered.''' def __call__(self, line): '''Returns the entire line without any folding. Returns: list of str: Components of folded line. They should be assembled via ``\\n.join()`` to obtain the string representation. ''' return [line] def get_option_parser(): '''Returns an option parser for the Fypp command line tool. Returns: OptionParser: Parser which can create an optparse.Values object with Fypp settings based on command line arguments. ''' defs = FyppOptions() fypp_name = 'fypp' fypp_desc = 'Preprocesses source code with Fypp directives. The input is '\ 'read from INFILE (default: \'-\', stdin) and written to '\ 'OUTFILE (default: \'-\', stdout).' fypp_version = fypp_name + ' ' + VERSION usage = '%prog [options] [INFILE] [OUTFILE]' parser = optparse.OptionParser(prog=fypp_name, description=fypp_desc, version=fypp_version, usage=usage) msg = 'define variable, value is interpreted as ' \ 'Python expression (e.g \'-DDEBUG=1\' sets DEBUG to the ' \ 'integer 1) or set to None if omitted' parser.add_option('-D', '--define', action='append', dest='defines', metavar='VAR[=VALUE]', default=defs.defines, help=msg) msg = 'add directory to the search paths for include files' parser.add_option('-I', '--include', action='append', dest='includes', metavar='INCDIR', default=defs.includes, help=msg) msg = 'import a python module at startup (import only trustworthy modules '\ 'as they have access to an **unrestricted** Python environment!)' parser.add_option('-m', '--module', action='append', dest='modules', metavar='MOD', default=defs.modules, help=msg) msg = 'directory to be searched for user imported modules before '\ 'looking up standard locations in sys.path' parser.add_option('-M', '--module-dir', action='append', dest='moduledirs', metavar='MODDIR', default=defs.moduledirs, help=msg) msg = 'emit line numbering markers' parser.add_option('-n', '--line-numbering', action='store_true', dest='line_numbering', default=defs.line_numbering, help=msg) msg = 'line numbering mode, \'full\' (default): line numbering '\ 'markers generated whenever source and output lines are out '\ 'of sync, \'nocontlines\': line numbering markers omitted '\ 'for continuation lines' parser.add_option('-N', '--line-numbering-mode', metavar='MODE', choices=['full', 'nocontlines'], default=defs.line_numbering_mode, dest='line_numbering_mode', help=msg) msg = 'line numbering marker format, currently \'std\', \'cpp\' and '\ '\'gfortran5\' are supported, where \'std\' emits #line pragmas '\ 'similar to standard tools, \'cpp\' produces line directives as '\ 'emitted by GNU cpp, and \'gfortran5\' cpp line directives with a '\ 'workaround for a bug introduced in GFortran 5. Default: \'cpp\'.' parser.add_option('--line-marker-format', metavar='FMT', choices=['cpp', 'gfortran5', 'std'], dest='line_marker_format', default=defs.line_marker_format, help=msg) msg = 'maximal line length (default: 132), lines modified by the '\ 'preprocessor are folded if becoming longer' parser.add_option('-l', '--line-length', type=int, metavar='LEN', dest='line_length', default=defs.line_length, help=msg) msg = 'line folding mode, \'smart\' (default): indentation context '\ 'and whitespace aware, \'simple\': indentation context aware, '\ '\'brute\': mechnical folding' parser.add_option('-f', '--folding-mode', metavar='MODE', choices=['smart', 'simple', 'brute'], dest='folding_mode', default=defs.folding_mode, help=msg) msg = 'suppress line folding' parser.add_option('-F', '--no-folding', action='store_true', dest='no_folding', default=defs.no_folding, help=msg) msg = 'indentation to use for continuation lines (default 4)' parser.add_option('--indentation', type=int, metavar='IND', dest='indentation', default=defs.indentation, help=msg) msg = 'produce fixed format output (any settings for options '\ '--line-length, --folding-method and --indentation are ignored)' parser.add_option('--fixed-format', action='store_true', dest='fixed_format', default=defs.fixed_format, help=msg) msg = 'character encoding for reading/writing files. Default: \'utf-8\'. '\ 'Note: reading from stdin and writing to stdout is encoded '\ 'according to the current locale and is not affected by this setting.' parser.add_option('--encoding', metavar='ENC', default=defs.encoding, help=msg) msg = 'create parent folders of the output file if they do not exist' parser.add_option('-p', '--create-parents', action='store_true', dest='create_parent_folder', default=defs.create_parent_folder, help=msg) return parser def run_fypp(): '''Run the Fypp command line tool.''' options = FyppOptions() optparser = get_option_parser() opts, leftover = optparser.parse_args(values=options) infile = leftover[0] if len(leftover) > 0 else '-' outfile = leftover[1] if len(leftover) > 1 else '-' try: tool = Fypp(opts) tool.process_file(infile, outfile) except FyppStopRequest as exc: sys.stderr.write(_formatted_exception(exc)) sys.exit(USER_ERROR_EXIT_CODE) except FyppFatalError as exc: sys.stderr.write(_formatted_exception(exc)) sys.exit(ERROR_EXIT_CODE) def linenumdir_cpp(linenr, fname, flag=None): """Returns a GNU cpp style line directive. Args: linenr (int): Line nr (starting with zero). fname (str): File name. flag (str): Optional flag to print after the directive Returns: Line number directive as string. """ if flag is None: return '# {0} "{1}"\n'.format(linenr + 1, fname) return '# {0} "{1}" {2}\n'.format(linenr + 1, fname, flag) def linenumdir_std(linenr, fname, flag=None): """Returns standard #line pragma styled line directive. Args: linenr (int): Line nr (starting with zero). fname (str): File name. flag (str): Optional flag to print after the directive. Note, this option is only there to be API compatible with linenumdir_cpp(), but is ignored otherwise, since #line pragmas do not allow for extra file opening/closing flags. Returns: Line number directive as string. """ return "#line {0} \"{1}\"\n".format(linenr + 1, fname) def _shiftinds(inds, shift): return [ind + shift for ind in inds] def _open_input_file(inpfile, encoding=None): try: inpfp = io.open(inpfile, 'r', encoding=encoding) except IOError as exc: msg = "Failed to open file '{0}' for read".format(inpfile) raise FyppFatalError(msg) from exc return inpfp def _open_output_file(outfile, encoding=None, create_parents=False): if create_parents: parentdir = os.path.abspath(os.path.dirname(outfile)) if not os.path.exists(parentdir): try: os.makedirs(parentdir) except OSError as exc: if exc.errno != errno.EEXIST: msg = "Folder '{0}' can not be created"\ .format(parentdir) raise FyppFatalError(msg) from exc try: outfp = io.open(outfile, 'w', encoding=encoding) except IOError as exc: msg = "Failed to open file '{0}' for write".format(outfile) raise FyppFatalError(msg) from exc return outfp # Signature objects are available from Python 3.3 (and deprecated from 3.5) def _get_callable_argspec(func): sig = inspect.signature(func) args = [] defaults = {} varpos = None varkw = None for param in sig.parameters.values(): if param.kind == param.POSITIONAL_OR_KEYWORD: args.append(param.name) if param.default != param.empty: defaults[param.name] = param.default elif param.kind == param.VAR_POSITIONAL: varpos = param.name elif param.kind == param.VAR_KEYWORD: varkw = param.name else: msg = "argument '{0}' has invalid argument type".format(param.name) raise FyppFatalError(msg) return args, defaults, varpos, varkw def _blank_match(match): size = match.end() - match.start() return " " * size def _argsplit_fortran(argtxt): txt = _INLINE_EVAL_REGION_REGEXP.sub(_blank_match, argtxt) splitpos = [-1] quote = None closing_brace_stack = [] closing_brace = None for ind, char in enumerate(txt): if quote: if char == quote: quote = None continue if char in _QUOTES_FORTRAN: quote = char continue if char in _OPENING_BRACKETS_FORTRAN: closing_brace_stack.append(closing_brace) ind = _OPENING_BRACKETS_FORTRAN.index(char) closing_brace = _CLOSING_BRACKETS_FORTRAN[ind] continue if char in _CLOSING_BRACKETS_FORTRAN: if char == closing_brace: closing_brace = closing_brace_stack.pop(-1) continue else: msg = "unexpected closing delimiter '{0}' in expression '{1}' "\ "at position {2}".format(char, argtxt, ind + 1) raise FyppFatalError(msg) if not closing_brace and char == _ARGUMENT_SPLIT_CHAR_FORTRAN: splitpos.append(ind) if quote or closing_brace: msg = "open quotes or brackets in expression '{0}'".format(argtxt) raise FyppFatalError(msg) splitpos.append(len(txt)) fragments = [argtxt[start + 1 : end] for start, end in zip(splitpos, splitpos[1:])] return fragments def _formatted_exception(exc): error_header_formstr = '{file}:{line}: ' error_body_formstr = 'error: {errormsg} [{errorclass}]' if not isinstance(exc, FyppError): return error_body_formstr.format( errormsg=str(exc), errorclass=exc.__class__.__name__) out = [] if exc.fname is not None: if exc.span[1] > exc.span[0] + 1: line = '{0}-{1}'.format(exc.span[0] + 1, exc.span[1]) else: line = '{0}'.format(exc.span[0] + 1) out.append(error_header_formstr.format(file=exc.fname, line=line)) out.append(error_body_formstr.format(errormsg=exc.msg, errorclass=exc.__class__.__name__)) if exc.__cause__ is not None: out.append('\n' + _formatted_exception(exc.__cause__)) out.append('\n') return ''.join(out) if __name__ == '__main__': run_fypp() fypp-3.1/docs/000077500000000000000000000000001403725613300132675ustar00rootroot00000000000000fypp-3.1/docs/Makefile000066400000000000000000000011361403725613300147300ustar00rootroot00000000000000# Minimal makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = python3 -msphinx SPHINXPROJ = Fypp SOURCEDIR = . BUILDDIR = _build # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) fypp-3.1/docs/conf.py000066400000000000000000000121171403725613300145700ustar00rootroot00000000000000#!/usr/bin/env python3 # -*- coding: utf-8 -*- # # Fypp documentation build configuration file, created by # sphinx-quickstart on Tue Sep 12 17:02:09 2017. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # import os import sys sys.path.insert(0, os.path.abspath('../src')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.napoleon'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # # source_suffix = ['.rst', '.md'] source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = 'Fypp' copyright = '2016-2021, Bálint Aradi' author = 'Bálint Aradi' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '3.1' # The full version, including alpha/beta/rc tags. release = '3.1' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # #html_theme = 'alabaster' html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # # html_theme_options = {} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Custom sidebar templates, must be a dictionary that maps document names # to template names. # # This is required for the alabaster theme # refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars html_sidebars = { '**': [ 'about.html', 'navigation.html', 'relations.html', # needs 'show_related': True theme option to display 'searchbox.html', 'donate.html', ] } # -- Options for HTMLHelp output ------------------------------------------ # Output file base name for HTML help builder. htmlhelp_basename = 'Fyppdoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # # 'preamble': '', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'Fypp.tex', 'Fypp Documentation', 'Bálint Aradi', 'manual'), ] # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'fypp', 'Fypp Documentation', [author], 1) ] # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'Fypp', 'Fypp Documentation', author, 'Fypp', 'One line description of project.', 'Miscellaneous'), ] fypp-3.1/docs/fypp.rst000066400000000000000000001721161403725613300150070ustar00rootroot00000000000000.. highlight:: none ************ Introduction ************ Fypp is a Python powered preprocessor. It can be used for any programming languages but its primary aim is to offer a Fortran preprocessor, which helps to extend Fortran with condititional compiling and template metaprogramming capabilities. Instead of introducing its own expression syntax, it uses Python expressions in its preprocessor directives, offering the consistency and versatility of Python when formulating metaprogramming tasks. It puts strong emphasis on robustness and on neat integration into developing toolchains. Fypp was inspired by the `pyratemp `_ templating engine [#]_. Although it shares many concepts with pyratemp, it was written from scratch focusing on the special needs when preprocessing source code. Fypp natively supports the output of line numbering markers, which are used by many compilers to generate compiler messages with correct line numbers. Unlike most cpp/fpp-like preprocessors or the coco preprocessor, Fypp also supports iterations, multiline macros, continuation lines in preprocessor directives and automatic line folding. It generally tries to extend the modern Fortran language with metaprogramming capabilities without tempting you to use it for tasks which could/should be done in Fortran itself. The project is `hosted on github `_ with documentation available on `readthedocs.org `_. Fypp is released under the *BSD 2-clause license*. This document describes Fypp Version 3.1. Features ======== Below you find a summary over Fypps main features. Each of them is described more in detail in the individual sections further down. * Definition, evaluation and removal of variables:: #:if DEBUG > 0 print *, "Some debug information" #:endif #:set LOGLEVEL = 2 print *, "LOGLEVEL: ${LOGLEVEL}$" #:del LOGLEVEL * Macro definitions and macro calls:: #:def ASSERT(cond) #:if DEBUG > 0 if (.not. ${cond}$) then print *, "Assert failed in file ${_FILE_}$, line ${_LINE_}$" error stop end if #:endif #:enddef ASSERT ! Invoked via direct call (argument needs no quotation) @:ASSERT(size(myArray) > 0) ! Invoked as Python expression (argument needs quotation) $:ASSERT('size(myArray) > 0') * Conditional output:: program test #:if defined('WITH_MPI') use mpi #:elif defined('WITH_OPENMP') use openmp #:else use serial #:endif * Iterated output (e.g. for generating Fortran templates):: interface myfunc #:for dtype in ['real', 'dreal', 'complex', 'dcomplex'] module procedure myfunc_${dtype}$ #:endfor end interface myfunc * Inline directives:: logical, parameter :: hasMpi = #{if defined('MPI')}# .true. #{else}# .false. #{endif}# * Insertion of arbitrary Python expressions:: character(*), parameter :: comp_date = "${time.strftime('%Y-%m-%d')}$" * Inclusion of files during preprocessing:: #:include "macrodefs.fypp" * Using Fortran-style continutation lines in preprocessor directives:: #:if var1 > var2 & & or var2 > var4 print *, "Doing something here" #:endif * Passing (unquoted) multiline string arguments to callables:: #! Callable needs only string argument #:def DEBUG_CODE(code) #:if DEBUG > 0 $:code #:endif #:enddef DEBUG_CODE #! Pass code block as first positional argument #:block DEBUG_CODE if (size(array) > 100) then print *, "DEBUG: spuriously large array" end if #:endblock DEBUG_CODE #! Callable needs also non-string argument types #:def REPEAT_CODE(code, repeat) #:for ind in range(repeat) $:code #:endfor #:enddef REPEAT_CODE #! Pass code block as positional argument and 3 as keyword argument "repeat" #:block REPEAT_CODE(repeat=3) this will be repeated 3 times #:endblock REPEAT_CODE * Preprocessor comments:: #! This will not show up in the output #! Also the newline characters at the end of the lines will be suppressed * Suppressing the preprocessor output in selected regions:: #! Definitions are read, but no output (e.g. newlines) will be produced #:mute #:include "macrodefs.fypp" #:endmute * Explicit request for stopping the preprocessor:: #:if DEBUGLEVEL < 0 #:stop 'Negative debug level not allowed!' #:endif * Easy check for macro parameter sanity:: #:def mymacro(RANK) #! Macro only works for RANK 1 and above #:assert RANK > 0 : #:enddef mymacro * Line numbering markers in output:: program test #:if defined('MPI') use mpi #:endif : transformed to :: # 1 "test.fypp" 1 program test # 3 "test.fypp" use mpi # 5 "test.fypp" : when variable ``MPI`` is defined and Fypp was instructed to generate line markers. * Automatic folding of generated lines exceeding line length limit *************** Getting started *************** Installing ========== Fypp needs a working Python interpreter. It is compatible with Python 2 (version 2.6 and above) and Python 3 (all versions). Automatic install ----------------- Use Pythons command line installer ``pip`` in order to download the stable release from the `Fypp page on PyPI `_ and install it on your system:: pip install fypp This installs both, the command line tool ``fypp`` and the Python module ``fypp.py``. Latter you can import if you want to access the functionality of Fypp directly from within your Python scripts. Manual install -------------- For a manual install, you can download the source code of the latest **stable** release from the `Fypp project website `_. If you wish to obtain the latest **development** version, clone the projects repository:: git clone https://github.com/aradi/fypp.git and check out the `master` branch. The command line tool is a single stand-alone script. You can run it directly from the source folder :: FYPP_SOURCE_FOLDER/bin/fypp or after copying it from the `bin` folder to any location listed in your `PATH` environment variable, by just issuing :: fypp The python module ``fypp.py`` can be found in ``FYP_SOURCE_FOLDER/src``. Testing ======= Simple manual testing can be done by issuing the command :: ./test/runtests.sh from the root of the Fypp source tree. This executes the unit tests shipped with Fypp with the default Python interpreter in your path. If you wish to use a specific interpreter, you can pass it as argument to the script:: ./test/runtests.sh python3 You can also pass multiple interpreters as separate arguments. In that case the testing will be carried out for each of them. Testing for developers ---------------------- If you wish to contribute to Fypp, you should have `tox` installed on your system, so that you can test the packaged project in isolated environments before issuing a pull request. In order to execute the unit tests with `tox`, run :: tox from the root folder of the source tree. This tries to test Fypp with various different python interpreters. If you want to limit testing to selected interpeters only, select the environment with the appropriate command line switch, e.g. :: tox -e py34 Running ======= The Fypp command line tool reads a file, preprocesses it and writes it to another file, so you would typically invoke it like:: fypp source.fpp source.f90 which would process `source.fpp` and write the result to `source.f90`. If input and output files are not specified, information is read from stdin and written to stdout. The behavior of Fypp can be influenced with various command line options. A summary of all command line options can be obtained by:: fypp -h ********************* Preprocessor language ********************* General syntax ============== Fypp has three types of preprocessor directives, all of them having a line and an inline form: * Control directives * Line form, starting with ``#:`` (hashmark colon):: #:if 1 > 2 Some code #:endif * Inline form, enclosed between ``#{`` and ``}#``:: #{if 1 > 2}#Some code#{endif}# * Eval directives * Line form, starting with ``$:`` (dollar colon):: $:time.strftime('%Y-%m-%d') * Inline form, enclosed between ``${`` and ``}$``:: print *, "Compilation date: ${time.strftime('%Y-%m-%d')}$" * Direct call directive * Line form, starting with ``@:`` (at colon):: @:mymacro(a < b) * Inline form, enclosed between ``@{`` and ``}@``:: print *, @{mymacro(a < b)}@ The line form must always start at the beginning of a line (preceded by optional whitespace characters only) and it ends at the end of the line. The inline form can appear anywhere, but if the construct consists of several directives (e.g. ``#{if ...}#`` and ``#{endif}#``), all of them must appear on the same line. While both forms can be used at the same time, they must be consistent for a particular construct, e.g. a directive opened as line directive can not be closed by an inline directive and vica versa. Whitespaces in preprocessor commands are ignored if they appear after the opening colon or curly brace or before the closing curly brace. So the following examples are pairwise equivalent:: #:if 1 > 2 #: if 1 > 2 #{if 1 > 2}# #{ if 1 > 2 }# $:time.strftime('%Y-%m-%d') $: time.strftime('%Y-%m-%d') ${time.strftime('%Y-%m-%d')}$ ${ time.strftime('%Y-%m-%d') }$ Starting whitespaces before line directives are ignored, enabling you to choose any indentation strategy you like for the directives:: program test : do ii = 1, nn print *, ii #:if DEBUG > 0 print *, "Some debug info about iteration ${ii}$" #:endif print *, "Normal code" end do : end program test Preprocessor directives can be arbitrarily nested:: #:if DEBUG > 0 #:if DO_LOGGING ... #:endif #:endif Every open directive must be closed before the end of the file is reached. In all control directives, the whitespace separating the name of the directive from the following parameter is obligatory. Therefore, the following example is syntactically incorrect:: #! Incorrect due to missing whitespace after 'if' #:if(1 > 2) Expression evaluation ===================== Python expressions can occur either as part of control directives, like :: #:if DEBUG > 0 #:for dtype in ['real(dp)', 'integer', 'logical'] or directly inserted into the code using eval directives. :: $:time.strftime('%Y-%m-%d') print *, "${time.strftime('%Y-%m-%d')}$" Expressions are always evaluated by using Pythons ``eval()`` builtin and must be, therefore, syntactically and semantically correct Python expressions. Although, this may require some additional quotations as compared to other preprocessor languages :: #:if defined('DEBUG') #! The Python function defined() expects a string argument #:for dtype in ['real(dp)', 'integer', 'logical'] #! dtype runs over strings it enables consistent expressions with (hopefully) least surprises (once you know, how to formulate the expression in Python, you exactly know, how to write it for Fypp). Also, note, that variable names, macros etc. are for Python (and therefore also for Fypp) case sensitive. When you access a variable in an expression, it must have been already defined before, either via command line options or via preprocessor directives. For example the directive :: #:if DEBUG > 0 can only be evaluated, if the variable `DEBUG` had been already defined before. Python sandbox ============== Python expressions are evaluated in an isolated Python environment, which contains a restricted set of Python built-in functions and a few predefined variables and functions (see below). There are no modules loaded by default, and for safety reasons, no modules can be loaded once the preprocessing has started, but can be loaded at startup if needed. Predefined variables -------------------- The isolated Python environment for the expression evaluation contains following predefined global variables: * ``_THIS_LINE_``: number of current line * ``_THIS_FILE_``: name of current file * ``_LINE_``: number of current line in the processed input file * ``_FILE_``: name of processed input file :: print *, "This is line nr. ${_LINE_}$ in file '${_FILE_}$'" * ``_DATE_``: current date in ISO format * ``_TIME_``: current time:: print *, "Rendering started ${_DATE_}$ ${_TIME_}$" * ``_SYSTEM_``: Name of the system Fypp runs on, as returned by Pythons ``platform.system()`` function (e.g. ``Linux``, ``Windows``, ``Darwin``, etc.) * ``_MACHINE_``: Name of the current machine Fypp runs on, as returned by Pythons ``platform.machine()`` function (e.g. ``x86_64``) The predefined variables ``_FILE_`` and ``_LINE_`` differ from their counterparts ``_THIS_FILE_`` and ``_THIS_LINE_`` only within macros. When a macro is executed, the variables ``_THIS_FILE_`` and ``_THIS_LINE_`` specify the position, where the expression containing these variables is located, while the variables ``_FILE_`` and ``_LINE_`` refer to the position in the processed file, from where the macro was called (and where the result of the evaluation will be inserted later). For example, the input :: #:def macro() IN MACRO: _THIS_LINE_=${_THIS_LINE_}$, _LINE_=${_LINE_}$ #:enddef macro GLOBAL: _THIS_LINE_=${_THIS_LINE_}$, _LINE_=${_LINE_}$ | ${macro()}$ yields after being processed by Fypp:: GLOBAL: _THIS_LINE_=5, _LINE_=5 | IN MACRO: _THIS_LINE_=2, _LINE_=5 If from within a macro an other macro is called, the variables ``_FILE_`` and ``_LINE_`` will keep their original values, while ``_THIS_FILE_`` and ``_THIS_LINE_`` will be continuously updated within the nested macro as well. Predefined functions -------------------- Following predefined functions are available: * ``defined(VARNAME)``: Returns ``True`` if a variable with a given name has been already defined. The variable name must be provided as string:: #:if defined('WITH_MPI') * ``getvar(VARNAME, DEFAULTVALUE)``: Returns the value of a variable or a default value if the variable is not defined. The variable name must be provided as string:: #:if getvar('DEBUG', 0) * ``setvar(VARNAME, VALUE)``: Sets a variable to given value. It is identical to the `set directive`_. The variable name expression has the same format as in the ``#:set`` directive, but must be quoted:: $:setvar('i', 12) print *, "VAR I: ${i}$" Multiple assignments may be specified as subsequent argument pairs:: $:setvar('i', 1, 'j', 2) print *, "VAR I: ${i}$, VAR J: ${j}$" * ``delvar(VARNAME)``: Removes a variable or a macro definition from the local scope. It is identical to the `del directive`_. The variable name expression must be provided as in the ``#:del`` directive, but must be quoted:: $:delvar('i') Additional variable name expressions may be specified as subsequent arguments:: $:delvar('i', 'j') * ``globalvar(VARNAME)``: Adds a given variable as global variable to the current scope. It is identical to the `global directive`_. The variable name expression must be provided as in the ``#:global`` directive, but must be quoted:: $:globalvar('i') Multiple variable name expressions may be specified as subsequent arguments. Initializing variables ---------------------- Initial values for preprocessor variables can be set via the command line option (``-D``) at startup:: fypp -DDEBUG=0 -DWITH_MPI The assigned value for a given variable is evaluated in Python. If no value is provided, `None` is assigned. Importing modules at startup ---------------------------- .. warning:: Modules imported at startup have access to the full **unrestricted** Python environment and can execute any Python code. Import only trustworthy modules! If a Python module is required for the preprocessing, it can be imported before the preprocessing starts via the command line option (``-m``):: fypp -m time The example above would allow to process the line:: character(*), parameter :: comp_date = "${time.strftime('%Y-%m-%d')}$" If more than one module is needed, each of them can imported with an individual ``-m`` option:: fypp -m time -m math When importing modules with the ``-m`` option, the module search path consists of the current directory, the directories in the `PYTHONPATH` environment variable and the standard Python module paths. Further lookup paths can be specified using the option ``-M``:: fypp -M mymoddir1 -M mymoddir2 -m mymodule -m mymodule2 The module directories are looked up in the order they are specified *before* searching at the default locations. Modules are imported also in the order of their specification at the command line. Each module imported at startup has its own name space. Entities in the imported modules can be accessed during the preprocessing in the usual pythonic way. After importing the module ``mymodule`` as in the example above, entities in the module could be accessed as:: ${mymodule.SOME_CONSTANT}$ $:mymodule.SOME_CONSTANT $:mymodule.some_function() @:mymodule.some_function() #:call mymodule.some_function #:endcall mymodule.some_function #:block mymodule.some_function #:endblock mymodule.some_function Eval directive ============== A result of a Python expression can be inserted into the code by using eval directives ``$:`` (line form) or ``${`` and ``}$`` (inline form). The expression is evaluated using Python's built-in function `eval()`. If it evaluates to `None`, no output is produced. Otherwise the result is converted to a string and written to the output. The eval directive has both, a line and an inline variant:: $:somePythonFunction() print *, "DEBUG LEVEL: ${DEBUG}$" .. warning:: Lines containing eval directive(s) will be folded using Fortran continuation lines when getting longer than a specified maximum. They must, therefore, not contain anything which could lead to invalid source code, when being folded at an arbitrary position (e.g. Fortran comments). `set` directive ================== The value of a variable can be set during the preprocessing via the `set` directive. (Otherwise, variables can be also declared and defined via command line options.) The first argument is the name of the variable (unquoted), followed by an optional Python expression. If the Python expression is present, it must be separated by an equal sign from the variable name. If the Python expression and the equal sign are not present, the variable is set to `None`:: #:set DEBUG #:set LOG = 1 #:set LOGLEVEL = LOGLEVEL + 1 Note, that in the last example the variable `LOGLEVEL` must have been already defined in advance. The `set` directive also accepts assignments to variable tuples, provided the right hand side of the assignment is compatible with the variable tuple:: #:set VAR1, VAR2 = 1, 2 #:set (VAR1, VAR2) = 1, 2 The parantheses around the variable list (second example) are optional. The `set` directive can be also used in the inline form:: #{set X = 2}#print *, ${X}$ Similar to the line form, the separating equal sign is optional here as well. `del` directive =============== A variable (or macro) definition can be removed from the current scope by the `del` directive:: #:set X = 12 #! X available, with value 12 : #:del X #! X not available any more The variable name expression syntax is identical to the one used for the `set` directive, so that also variable tuples can be deleted:: #! Removes the variables X and Y from local scope #:del X, Y The variable passed to the ``del`` directive must exist and be erasable. So the example above would trigger an error, if the variables ``X`` and ``Y`` were not defined before. The `del` directive can also be used to delete macro definitions:: #:def echo(TXT) ${TXT}$ #:enddef @:echo(HELLO) #:del echo #! Following line throws an error as macro echo is not available any more @:echo(HELLO) The `del` directive can be also used in the inline form:: #{del X}# `if` directive ============== Conditional output can be generated using the `if` directive. The condition must be a Python expression, which can be converted to a `bool`. If the condition evaluates to `True`, the enclosed code is written to the output, otherwise it is ignored. :: print *, "Before" #:if DEBUG > 0 print *, "Debug code" #:endif print *, "After" would result in :: print *, "Before" print *, "Debug code" print *, "After" if the Python expression ``DEBUG > 0`` evaluates to `True`, otherwise in :: print *, "Before" print *, "After" For more complex scenarios ``elif`` and ``else`` branches can be used as well:: #:if DEBUG >= 2 print *, "Very detailed debug info" #:elif DEBUG >= 1 print *, "Less detailed debug info" #:else print *, "No debug info" #:endif The `if` directive is also available as inline directive:: print *, "COMPILATION MODE: #{if DEBUG > 0}#DEBUG#{else}#PRODUCTION#{endif}#" `for` directive =============== Fortran templates can be easily created by using the `for` directive. The following example creates a function for calculating the sine square for both single and double precision reals:: #:set real_kinds = ['sp', 'dp'] interface sin2 #:for rkind in real_kinds module procedure sin2_${rkind}$ #:endfor end interface sin2 #:for rkind in real_kinds function sin2_${rkind}$(xx) result(res) real(${rkind}$), intent(in) :: xx real(${rkind}$) :: res res = sin(xx) * sin(xx) end function sin2_${rkind}$ #:endfor The `for` directive expects a loop variable expression and an iterable separated by the ``in`` keyword. The code within the `for` directive is outputed for every iteration with the current value of the loop variable, which can be inserted using eval directives. The loop variable expression must be either a name or a list of names joined by comma (``,``). In the latter case, the iterable must consist of iterable items (e.g. tuples), which will be then unpacked into the loop variables. (The number of the loop variables and the number of the components of each iterated item must be identical.):: #:set kinds = ['sp', 'dp'] #:set names = ['real', 'dreal'] #! create kinds_names as [('sp', 'real'), ('dp', 'dreal')] #:set kinds_names = list(zip(kinds, names)) #! Access by indexing interface sin2 #:for kind_name in kinds_names module procedure sin2_${kind_name[1]}$ #:endfor end interface sin2 #! Unpacking in the loop header #:for kind, name in kinds_names function sin2_${name}$(xx) result(res) real(${kind}$), intent(in) :: xx real(${kind}$) :: res res = sin(xx) * sin(xx) end function sin2_${name}$ #:endfor The `for` directive can be used also in its inline form:: print *, "Numbers: #{for i in range(5)}#${i}$#{endfor}#" `def` directive =============== Parametrized macros can be defined with the `def` directive. This defines a regular callable in Python, which returns the rendered content of the macro body when called. The macro arguments are converted to local variables containing the actual arguments as values. The macro can be called from within an eval-directive, via the `call` and `block` control directives and via their abreviated form, the direct call. Given the macro definition :: #:def ASSERT(cond) #:if DEBUG > 0 if (.not. (${cond}$)) then print *, "Assert failed!" error stop end if #:endif #:enddef the following three calls :: #! call macro by evaluating a Python expression $:ASSERT('x > y') #! call macro by using the call directive (see below) #:call ASSERT x > y #:endcall ASSERT #! call macro by using the block directive (see below) #:block ASSERT x > y #:endblock ASSERT #! call macro by using the direct call directive (see below) @:ASSERT(x > y) would all yield :: if (.not. (x > y)) then print *, "Assert failed!" error stop end if if the variable `DEBUG` had a value greater than zero or an empty string otherwise. It is possible to declare default values for the positional arguments of a macro. If for a given positional argument such a value is provided, then default values must be provided for all following arguments as well. When the macro is called, missing positional arguments will be replaced by their default value:: #:def macro(X, Y=2, Z=3) X=${X}$, Y=${Y}$, Z=${Z}$ #:enddef macro $:macro(1) #! Returns "X=1, Y=2, Z=3" Similar to Python, it is also possible to define macros with a variable number of positional or keyword arguments using the ``*`` and ``**`` argument prefixes. The corresponding arguments will contain the unprocessed positional and keywords arguments as a list and a dictionary, respectively:: #:def macro(X, *VARPOS, **VARKW) pos: ${X}$ varpos: #{for ARG in VARPOS}#${ARG}$, #{endfor}# varkw: #{for KEYWORD in VARKW}#${KEYWORD}$->${VARKW[KEYWORD]}$, #{endfor}# #:enddef macro Calling the example macro above with :: $:macro(1, 2, 3, kw1=4, kw2=5) yields:: pos: 1 varpos: 2, 3, varkw: kw1->4, kw2->5, Scopes ------ Scopes in general follow the Python convention: Within the macro, all variables from the encompassing scope are available (as `DEBUG` in the example above), and additionally those which were passed as arguments. If a variable is defined within the macro, it will be only accessible within the macro. If a variable with the same name already exists in the encompassing scope, it will be shadowed by it for the time of the macro substitution. For example preprocessing the code snippet :: #:def macro(x) print *, "Local XY: ${x}$ ${y}$" #:set y = -2 print *, "Local XY: ${x}$ ${y}$" #:enddef #:set x = 1 #:set y = 2 print *, "Global XY: ${x}$ ${y}$" $:macro(-1) print *, "Global XY: ${x}$ ${y}$" would result in :: print *, "Global XY: 1 2" print *, "Local XY: -1 2" print *, "Local XY: -1 -2" print *, "Global XY: 1 2" For better readability, you can repeat the name of the macro (but not its argument list) at the corresponding enddef directive:: #:def ASSERT(cond) #:if DEBUG > 0 if (.not. (${cond}$)) then print *, "Assert failed!" error stop end if #:endif #:enddef ASSERT The `def` directive has no inline form. .. warning:: The content of macros is usually inserted via an eval directive and is accordingly subject to eventual line folding. Macros should, therefore, not contain any inline Fortran comments. (Comments starting at the beginning of the line preceded by optional whitespaces only are OK, though). Use preprocessor comments (``#!``) instead. `block` and `call` directives ============================= When a Python callable (regular Python function, macro etc.) needs a string argument of larger size (e.g. source code), it can be called using the `call` or the `block` directives to avoid extra quoting of the text argument and to enable passing of multiline arguments in a comfortable way:: #:def DEBUG_CODE(code) #:if DEBUG > 0 $:code #:endif #:enddef DEBUG_CODE #:block DEBUG_CODE if (a < b) then print *, "DEBUG: a is less than b" end if #:endblock DEBUG_CODE #:call DEBUG_CODE if (a < b) then print *, "DEBUG: a is less than b" end if #:endcall DEBUG_CODE The `block` and the `call` directives are equivalent. The two alternative forms exists in order to allow for more readable meta-code depending on the context. The `block` and `call` directives take the name of the callable as argument. The lines between the opening and closing directives will be rendered and then passed as positional *string* arguments to the callable. The name of the callable can be repeated in the `endblock` and `endcall` directives for enhanced readability:: #! This form is probably somewhat more natural to read #:block DEBUG_CODE if (a < b) then print *, "DEBUG: a (${a}$) is less than b (${b}$)" end if #:endblock DEBUG_CODE #:call DEBUG_CODE if (a < b) then print *, "DEBUG: a (${a}$) is less than b (${b}$)" end if #:endcall DEBUG_CODE If the callable needs more than one string arguments, the `contains` directive (for `block`) or the `nextarg` directive (for `call`) can be used to separate the arguments from each other:: #:def CHOOSE_CODE(debug_code, nondebug_code) #:if DEBUG > 0 $:debug_code #:else $:nondebug_code #:endif #:enddef CHOOSE_CODE #:block CHOOSE_CODE if (a < b) then print *, "DEBUG: a is less than b" end if #:contains print *, "No debugging" #:endcall CHOOSE_CODE #! This form is probably somewhat more natural to read #:call CHOOSE_CODE if (a < b) then print *, "DEBUG: a is less than b" end if #:nextarg print *, "No debugging" #:endcall CHOOSE_CODE The lines in the body of the `block` and `call` directives may contain directives themselves. However, any variable defined within the body of the `block` and `call` directives will be a local variable existing only during the evaluation of that branch of the directive (and not being available when the callable is called with the evaluated string as argument). The `contains` and `nextarg` directives may be followed by an optional argument name. In that case the text following will be passed as keyword argument to the callable. If the first argument should be also passed as keyword argument, it should be also preceded by a named `contains` or `nextarg` directive declared in the line immediately following the `block` or `call` directive. If an argument is passed as a keyword argument, all following arguments must be passed as keyword arguments as well:: #:block CHOOSE_CODE #:contains nondebug_code print *, "No debugging" #:contains debug_code if (a < b) then print *, "DEBUG: a is less than b" end if #:endblock CHOOSE_CODE #:call CHOOSE_CODE #:nextarg nondebug_code print *, "No debugging" #:nextarg debug_code if (a < b) then print *, "DEBUG: a is less than b" end if #:endcall CHOOSE_CODE Additional to passing the content of the `block` or `call` directives body as string argument, further arguments of arbitrary type can be passed by specifying them directly in the header of the directive. Among others, this can be very comfortable when the callable needs also non-string type of arguments:: #! Argument 'repeat' should be an integer, not string #:def REPEAT_CODE(code, repeat) #:for ind in range(repeat) $:code #:endfor #:enddef REPEAT_CODE #! Code block as positional argument and 3 as keyword argument "repeat" #:block REPEAT_CODE(repeat=3) this will be repeated 3 times #:block REPEAT_CODE #! Code block as positional argument and 3 as keyword argument "repeat" #:call REPEAT_CODE(repeat=3) this will be repeated 3 times #:endcall REPEAT_CODE The arguments must be specified between parantheses and are evaluated as Python expressions. The arguments specified in the directive (both, in the header and in the body) are passed to the callable in the following order: #. positional arguments in the header #. positional arguments in the body #. keyword arguments in the header #. keyword arguments in the body Callables without arguments can also be called with the `block` and `call` directives, provided the `endblock` and `endcall` directives immediately follows the opening directive. If there are empty lines between the opening and the closing directives, they will be interpreted as a positional argument:: #:def macro_noarg() NOARGS #:enddef macro_noarg #:def macro_arg1(arg1) ARG1:${arg1}$ #:enddef macro_arg1 #! Calling macro without arguments #:block macro_noarg #:endblock macro_noarg #! Calling macro without arguments #:call macro_noarg #:endcall macro_noarg #! Calling macro with one positional (empty) argument #! Note the empty line between block and endblock #:block macro_arg1 #:endblock macro_arg1 #! Calling macro with one positional (empty) argument #! Note the empty line between call and endcall #:call macro_arg1 #:endcall macro_arg1 The `block` and `call` directives can also be used in their inline form. As this easily leads to code being hard to read, it should be usually avoided:: ! Rather ugly print *, #{block CHOOSE_CODE}# a(:) #{contains}# size(a) #{endblock}# ! Rather ugly as well print *, #{call CHOOSE_CODE}# a(:) #{nextarg}# size(a) #{endcall}# ! This form is more readable print *, ${CHOOSE_CODE('a(:)', 'size(a)')}$ ! Alternatively, you may use a direct call (see next section) print *, @{CHOOSE_CODE(a(:), size(a))}@ If the callable only requires short text arguments, the more compact direct call directive should be used as an alternative (see next section). Direct call directive ===================== In order to enable compact (single line) calls while still maintaining code readability, the `block` and `call` directives have an alternative form, the direct call directive:: #:def ASSERT(cond) #:if DEBUG > 0 if (.not. (${cond}$)) then print *, "Assert failed!" error stop end if #:endif #:enddef ASSERT @:ASSERT(size(aa) >= size(bb)) The direct call directive starts with ``@:`` followed by the name of a Python callable and an opening parenthesis (``(``). Everything after that up to the closing parenthesis (``)``) is passed as *string argument* to the callable. The closing parenthesis may only be followed by whitespace characters. When the callable needs more than one argument, the arguments must be separated by a comma (``,``):: #:def ASSERT_EQUAL(received, expected) if (${received}$ /= ${expected}$) then print *, "ASSERT_EQUAL failed (${received}$ /= ${expected}$)!" error stop end if #:enddef ASSERT_EQUAL @:ASSERT_EQUAL(size(coords, dim=2), size(atomtypes)) .. note:: In order to be able to split the argument string of a direct call correctly, Fypp assumes that all provided arguments represent valid Fortran expressions with balanced quotes (``'`` or ``"``) and balanced brackets (``()``, ``[]`` and ``{}``) outside of quoted regions. The argument string is only split around commas which are outside of any quoted or bracketed regions. Arguments can be optionally enclosed within curly braces in order to avoid argument splitting at unwanted places or to improve readability. The outermost curly braces will be removed from the arguments before they are passed to the callable:: #! Passes "a**2 + b**2" and "c**2" as string arguments to ASSERT_EQUAL @:ASSERT_EQUAL({a**2 + b**2}, c**2) Keywords arguments can be passed by prefixing them with the keyword name and an equal sign:: @:ASSERT_EQUAL(expected=size(atomtypes), received=size(coords, dim=2)) @:ASSERT_EQUAL(expected=c**2, received={a**2 + b**2}) If the equal sign is followed immediately by an other equal sign, the argument will be recognized as positional and not as keyword argument. This exception allows for passing valid Fortran code containing the comparison operator (``==``) without the need for special bracketing. In other cases, however, bracketing may be needed to avoid recognition as keyword argument:: #! Passes string "a == b" as first positional argument @:ASSERT(a == b) #! Passes string "=b" as keyword argument "a" @:ASSERT(a={=b}) #! Passes string "b" as keyword argument "a" @:someMacro(a = b) #! Passes "a = b" as positional argument @:someMacro({a = b}) The direct call directive may contain continuation lines:: @:ASSERT_EQUAL(size(coords, dim=2), & & size(atomtypes)) The arguments are parsed for further inline eval directives (but not for any inline control or direct call directives), making variable substitutions in the arguments possible:: #:set MYSIZE = 2 @:ASSERT_EQUAL(size(coords, dim=2), ${MYSIZE}$) Whitespaces around the arguments of the direct call are stripped, but not the whitespaces within the optional curly braces around the argument:: #! Calls a macro without arguments @:macro_without_args() #! Calls a macro with no arguments (whitespace between () is stripped): @:macro_without_args( ) #! Calls a macro with empty string as argument @:macro_with_one_arg({}) #! Calls a macro with one space as argument @:macro_with_one_arg({ }) The direct call directive can also be used in its inline form:: #! Using CHOOSE_CODE() macro defined in previous section print *, @{CHOOSE_CODE(a(:), size(a))}@ `global` directive ================== Global variables are by default read-only in local scopes (e.g. within macros). This can be changed for selected variables by using the `global` directive:: #:def set_debug(value) #:global DEBUG #:set DEBUG = value #:enddef set_debug #:set DEBUG = 1 $:DEBUG $:set_debug(2) $:DEBUG In the example above, without the `global` directive, the `set` directive would have created a local variable within the macro, which had shadowed the global variable and was destroyed at the end of the macro execution. With the `global` directive the `set` refers to the variable in the global scope. The variable in the global scope does not need to exist yet, when the `global` directive is executed. It will be then created at the first `set` directive, or remain non-existing if no assignment is made in the current scope. A variable can only made global, if it was not created in the local scope yet. Therefore, the following code would throw an exception:: #:def set_debug(value) #! DEBUG variable created in local scope #:set DEBUG = value #! Invalid: variable DEBUG already exists in local scope #:global DEBUG #:enddef set_debug # Throws exception $:set_debug(2) `include` directive =================== The `include` directive allows you to collect your preprocessor macros and variable definitions in separate files and include them whenever needed. The include directive expects a quoted string with a file name:: #:include 'mydefs.fypp' If the file name is relative, it is interpreted relative to the folder where the processed file is located (or to the current folder, if Fypp reads from stdin). Further lookup paths can be added with the ``-I`` command line option. The `include` directive does not have an inline form. `mute` directive ================ Empty lines between Fypp definitions makes the code easier to read. However, being outside of Fypp-directives, those empty lines will be written unaltered to the output. This can be especially disturbing if various macro definition files are included, as the resulting output would eventually contain a lot of empty lines. With the `mute` directive, the output can be suspended. While everything is still processed as normal, no output is written for the code within the `mute` directive:: #:mute #:include "mydefs1.fypp" #:include "mydefs2.fypp" #:def test(x) print *, "TEST: ${x}$" #:enddef test #:endmute $:test('me') The example above would only produce :: print *, "TEST: me" as output without any newlines. The `mute` directive does not have an inline form. `stop` directive ================ The `stop` directive can be used to report an error and stop the preprocessor before all input has been consumed. This can be useful in cases, where some external conditions (e.g. user defined variables) do not meet certain criteria. The directive expects a Python expression, which will be converted to string and written to standard error. After writing the error message Fypp exits immediately with a non-zero exit code (see `Exit Codes`_):: #! Stop the code if DEBUGLEVEL is not positive #:if DEBUGLEVEL < 0 #:stop 'Wrong debug level {}!'.format(DEBUGLEVEL) #:endif There is no inline form of the `stop` directive. `assert` directive ================== The `assert` directive is a short form for the combination of an `if` and a `stop` directive. It evaluates a given expression and stops the code if the boolean value of the result is `False`. This can be very convenient, if you want to write robust macros containing sanity checks for their arguments:: #:def mymacro(RANK) #! Macro only works for RANK 1 and above #:assert RANK > 0 : #:enddef mymacro Given the macro definition above, the macro call :: $:mymacro(1) would pass the `assert` directive in the third line, while the call :: $:mymacro(0) would cause Fypp to stop at it. When the expression in an `assert` directive evaluates to `False`, Fypp reports the failed assertion (the condition, the file name and the line number) on standard error and terminates immediately with a non-zero exit code (see `Exit Codes`_). There is no inline form of the `assert` directive. Comment directive ================= Comment lines can be added by using the ``#!`` preprocessor directive. The comment line (including the newlines at their end) will be ignored by the prepropessor and will not appear in the output:: #! This will not show up in the output There is no inline form of the comment directive. **************** Various features **************** Multiline directives ==================== The line form of the control and eval directives can span arbitrary number of lines, if Fortran-style continuation characters are used:: #:if a > b & & or b > c & & or c > d $:somePythonFunction(param1, & ¶m2) The line break at the first line must be in the expression, not in the opening delimiter characters or in the directive name. Similar to Fortran, the continuation character at the beginning of each continuation line may be left away, but then all whitespaces at the beginning of the respective continuation line will be part of the expression. Inline directives must not contain any continuation lines. Line folding ============ The Fortran standard only allows source lines up to 132 characters. In order to emit standard conforming code, Fypp folds all lines in the output which it had manipulated before (all lines containing eval directives). Lines which were just copied to the output are left unaltered. The maximal line length can be chosen by the ``-l`` command line option. The indentation of the continuation lines can be tuned with the ``--indentation`` option, and the folding strategy can be selected by the ``-f`` option with following possibilities: * ``brute``: Continuation lines are indented relative to the beginning of the line, and each line is folded at the maximal line position. * ``simple``: Like ``brute``, but continuation lines are indented with respect of the indentation of the original line. * ``smart``: Like ``simple``, but Fypp tries to fold the line at a whitespace character in order to prevent split tokens. To prevent continuation lines becoming too short, it defaults to ``simple`` if no whitespace occurs in the last third of the line. The ``-F`` option can be used to turn off line folding. .. warning:: Fypp is not aware of the Fortran semantics of the lines it folds. Fypp applies the line folding mechanically (only considering the position of the whitespace characters). Lines containing eval directives and lines within macro definitions should, therefore, not contain any Fortran style comments (started by ``!``) *within* the line, as folding within the comment would result in invalid Fortran code. For comments within such lines, Fypps comment directive (``#!``) can be used instead:: #:def macro() print *, "DO NOT DO THIS!" ! Warning: Line may be folded within the comment print *, "This is OK." #! Preprocessor comment is safe as it will be stripped For comments starting at the beginning of the line (preceded by optional whitespace characters only) the folding is suppressed, though. This enables you to define macros with non-negligible comment lines (e.g. with source code documentation or OpenMP directives):: #:def macro(DTYPE) !> This functions calculates something (version ${DTYPE}$) !! \param xx Ingoing value !! \return Some calculated value. ${DTYPE}$ function calcSomething(xx) : end function calcSomething #:enddef macro Escaping ======== If you want to prevent Fypp to interpret something as a directive, put a backslash (``\``) between the first and second delimiter character. In case of inline directives, do it for both, the opening and the closing delimiter:: $\: 1 + 2 #\{if 1 > 2}\# @\:myMacro arg1 Fypp will not recognize the escaped strings as directives, but will remove the backslash between the delimiter characters in the output. If you put more than one backslash between the delimiters, only one will be removed. Line numbering markers ====================== In order to support compilers in emitting messages with correct line numbers with respect to the original source file, Fypp can put line number directives (a.k.a. line markers) in its output. This can be enabled by using the command line option ``-n``. Given a file ``test.fpp`` with the content :: program test #:if defined('MPI') use mpi #:else use openmpi #:endif : end program test the command :: fypp -n -DMPI test.fpp produces the output :: # 1 "test.fpp" 1 program test # 3 "test.fpp" use mpi # 7 "test.fpp" : end program test If during compilation of this output an error occurred in the line ``use mpi`` (e.g. the mpi module can not be found), the compiler would know that this line corresponds to line number 3 in the original file ``test.fpp`` and could emit an according error message. The line numbering directives can be fine tuned with the ``-N`` option, which accepts following mode arguments: * ``full``: Line numbering directives are emitted whenever lines are removed from the original source file or extra lines are added to it. * ``nocontlines``: Same as full, but line numbering directives are omitted before continuation lines. (Some compilers, like the NAG Fortran compiler, have difficulties with line numbering directives before continuation lines). Note: Due to a bug introduced in GFortran 5 (being also present in major versions 6), a workaround is needed for obtaining correct error messages when compiling preprocessed files with those compilers. Please use the command line option ``--line-marker-format 'gfortran5'`` in those cases. Scopes ====== Fypp uses a scope concept very similar to Pythons one. There is one global scope (like in Python modules), and temporary local scopes may be created in special cases (e.g. during macro calls). The global scope is the one, which Fypp normaly uses for defining objects. All imports specified on the command line are carried out in this scope And all definitions made by the `set` and `def` directives in the processed source file defines entities in that scope, unless they appear within a `block`, a `call` or a `def` directive. Addtional temporary local scopes are opened, whenever * a macro defined by the `def` directive is called, or * the body of the `block` or `call` directive is evaluated in order to render the text, which will be passed to the callable as argument. Any entity defined in a local scope is only visible within that scope and is unaccessible once the scope has been closed. For example the code snippet:: #:set toupper = lambda s: s.upper() #:call toupper #:set NUMBER = 9 here is the number ${NUMBER}$ #:endcall toupper $:defined('NUMBER') results after preprocessing in :: HERE IS THE NUMBER 9 False as the variable ``NUMBER`` defined in the local scope is destroyed, when the scope is closed (the `endcall` directive has been reached). Lookup rules ------------ When Fypp tries to resolve a name, the lookup rules depend on the scope, in which the query appears: * global scope (outside of any `def` or `call` directives): only the global scope is searched. * local scope (within the body of a `call` or `def` directive): first, the active local scope is searched. Then the scope embedding it (the scope which contains the directive) is searched. Then further embedding scopes are searched until finally also the global scope has been checked. The search is immediately terminated, if the name has been found in a scope. Note, that all variables outside of the active scope are read-only. If a variable with the same name is created in the active scope, it will shadow the original definition. Once the scope is closed, the variable regains it original value. For example:: #:set toupper = lambda s: s.upper() #:set X = 1 #:call toupper #:set X = 2 value ${X}$ #:endcall toupper value ${X}$ results in :: VALUE 2 value 1 Also note, that if a name can not be resolved in the active scope during a macro evaluation, the relevant embedding scope for the next lookup is the scope, where the macro has been defined (where the `def` directive occurs), and *not* the scope, from which the macro is being called. The following snippet demonstrates this:: #! GLOBAL SCOPE #:set toupper = lambda s: s.upper() #:call toupper #! LOCAL SCOPE 1 #:def macro1() #! LOCAL SCOPE 2A value of x: ${X}$ #:enddef macro1 #! LOCAL SCOPE 1 #:def macro2() #! LOCAL SCOPE 2B #:set X = 2 $:macro1() #:enddef macro2 #! LOCAL SCOPE 1 #:set X = 1 $:macro2() #:endcall #! GLOBAL SCOPE After processing the code above one obtains ``VALUE OF X: 1``. Although in the local scope 2B, from where the macro ``macro1()`` is called, the value of X is defined to be ``2``, the relevant scopes for the lookup of X during the macro evaluation are the local scope 2A of ``macro1()`` (where the eval-directive for X is located), the local scope 1 (where the `def` directive for ``macro1()`` occurs) and the global scope (which embeds local scope 1). Therefore, at the macro evaluation the value ``1`` will be substituted as this is the value of X in scope 1, and scope 1 is the first scope in the lookup order, which provides a value for X. Exit codes ========== When run as a standalone application, Fypp returns one of the following exit codes to the calling environment: * 0: Preprocessing finished successfully. * 1: Stopped due to an unexpected error. * 2: Explicitely requested stop encountered (`stop directive`_ or `assert directive`_). ******** Examples ******** Asserts and debug code ====================== In this example a simple "assert"-mechanism (as can be found in many programming languages) should be implemented, where run-time checks can be included or excluded depending on preprocessor variable definitions. Apart of single assert-like queries, we also want to include larger debug code pieces, which can be removed in the production code. First, we create an include file (``checks.fypp``) with the appropriate macros:: #:mute #! Enable debug feature if the preprocessor variable DEBUG has been defined #:set DEBUG = defined('DEBUG') #! Stops the code, if the condition passed to it is not fulfilled #! Only included in debug mode. #:def ASSERT(cond, msg=None) #:if DEBUG if (.not. (${cond}$)) then write(*,*) 'Run-time check failed' write(*,*) 'Condition: ${cond.replace("'", "''")}$' #:if msg is not None write(*,*) 'Message: ', ${msg}$ #:endif write(*,*) 'File: ${_FILE_}$' write(*,*) 'Line: ', ${_LINE_}$ stop end if #:endif #:enddef ASSERT #! Includes code if in debug mode. #:def DEBUG_CODE(code) #:if DEBUG $:code #:endif #:enddef DEBUG_CODE #:endmute Remarks: * All macro definitions are within a ``#:mute`` -- ``#:endmute`` pair in order to prevent the appearance of disturbing empty lines (the lines between the macro definitions) in the file which includes ``checks.fypp``. * The preprocessor variable ``DEBUG`` will determine, whether the checks and the debug code is left in the preprocessed code or not. * The content of both macros, ``ASSERT`` and ``DEBUG_CODE``, are only included if the variable ``DEBUG`` has been defined. * We also want to print out the failed condition for more verbose output. As the condition may contains apostrophes, we use Python's string replacement method to escape them. With the definitions above, we can use the functionality in any Fortran source after including ``checks.fypp``:: #:include 'checks.fypp' module testmod implicit none contains subroutine someFunction(ind, uplo) integer, intent(in) :: ind character, intent(in) :: uplo @:ASSERT(ind > 0, msg="Index must be positive") @:ASSERT(uplo == 'U' .or. uplo == 'L') ! Do something useful here ! : #:block DEBUG_CODE print *, 'We are in debug mode' print *, 'The value of ind is', ind #:endblock DEBUG_CODE end subroutine someFunction end module testmod Now, the file ``testmod.fpp`` can be preprocessed with Fypp. When the variable ``DEBUG`` is not set:: fypp testmod.fpp testmod.f90 the resulting routine will not contain the conditional code:: subroutine someFunction(ind, uplo) integer, intent(in) :: ind character, intent(in) :: uplo ! Do something useful here ! : end subroutine someFunction On the other hand, if the ``DEBUG`` variable is set:: fypp -DDEBUG testmod.fpp testmod.f90 the run-time checks and the debug code will be there:: subroutine someFunction(ind, uplo) integer, intent(in) :: ind character, intent(in) :: uplo if (.not. (ind > 0)) then write(*,*) 'Run-time check failed' write(*,*) 'Condition: ind > 0' write(*,*) 'Message: ', "Index must be positive" write(*,*) 'File: testmod.fpp' write(*,*) 'Line: ', 12 stop end if if (.not. (uplo == 'U' .or. uplo == 'L')) then write(*,*) 'Run-time check failed' write(*,*) 'Condition: uplo == ''U'' .or. uplo == ''L''' write(*,*) 'File: testmod.fpp' write(*,*) 'Line: ', 13 stop end if ! Do something useful here ! : print *, 'We are in debug mode' print *, 'The value of ind is', ind end subroutine someFunction Generic programming =================== The example below shows how to create a generic function ``maxRelError()``, which gives the maximal elementwise relative error for any pair of arrays with ranks from 0 (scalar) to 7 in single or double precision. The Fortran module (file ``errorcalc.fpp``) contains the interface ``maxRelError`` which maps to all the realizations with the different array ranks and precisions:: #:def ranksuffix(RANK) $:'' if RANK == 0 else '(' + ':' + ',:' * (RANK - 1) + ')' #:enddef ranksuffix #:set PRECISIONS = ['sp', 'dp'] #:set RANKS = range(0, 8) module errorcalc implicit none integer, parameter :: sp = kind(1.0) integer, parameter :: dp = kind(1.0d0) interface maxRelError #:for PREC in PRECISIONS #:for RANK in RANKS module procedure maxRelError_${RANK}$_${PREC}$ #:endfor #:endfor end interface maxRelError contains #:for PREC in PRECISIONS #:for RANK in RANKS function maxRelError_${RANK}$_${PREC}$(obtained, reference) result(res) real(${PREC}$), intent(in) :: obtained${ranksuffix(RANK)}$ real(${PREC}$), intent(in) :: reference${ranksuffix(RANK)}$ real(${PREC}$) :: res #:if RANK == 0 res = abs((obtained - reference) / reference) #:else res = maxval(abs((obtained - reference) / reference)) #:endif end function maxRelError_${RANK}$_${PREC}$ #:endfor #:endfor end module errorcalc The macro ``ranksuffix()`` defined at the beginning receives a rank as argument and returns a string, which is either the empty string (rank 0) or the appropriate number of dimension placeholder separated by commas and within parantheses (e.g. ``(:,:)`` for rank 2). The string expression is calculated as a Python expression, so that we can make use of the powerful string manipulation routines in Python and write it as a one-line routine. If we preprocess the Fortran source file ``errorcalc.fpp`` with Fypp:: fypp errorcalc.fpp errorcalc.f90 the resulting file ``errorcalc.f90`` will contain a module with the generic interface ``maxRelError()``:: interface maxRelError module procedure maxRelError_0_sp module procedure maxRelError_1_sp module procedure maxRelError_2_sp module procedure maxRelError_3_sp module procedure maxRelError_4_sp module procedure maxRelError_5_sp module procedure maxRelError_6_sp module procedure maxRelError_7_sp module procedure maxRelError_0_dp module procedure maxRelError_1_dp module procedure maxRelError_2_dp module procedure maxRelError_3_dp module procedure maxRelError_4_dp module procedure maxRelError_5_dp module procedure maxRelError_6_dp module procedure maxRelError_7_dp end interface maxRelError The interface maps to the appropriate functions:: function maxRelError_0_sp(obtained, reference) result(res) real(sp), intent(in) :: obtained real(sp), intent(in) :: reference real(sp) :: res res = abs((obtained - reference) / reference) end function maxRelError_0_sp function maxRelError_1_sp(obtained, reference) result(res) real(sp), intent(in) :: obtained(:) real(sp), intent(in) :: reference(:) real(sp) :: res res = maxval(abs((obtained - reference) / reference)) end function maxRelError_1_sp function maxRelError_2_sp(obtained, reference) result(res) real(sp), intent(in) :: obtained(:,:) real(sp), intent(in) :: reference(:,:) real(sp) :: res res = maxval(abs((obtained - reference) / reference)) end function maxRelError_2_sp : The function ``maxRelError()`` can be, therefore, invoked with a pair of arrays with various ranks or with a pair of scalars, both in single and in double precision, as required. If you prefer not to have preprocessor loops around long code blocks, the example above can be also written by defining a macro first and then calling the macro within the loop. The function definition would then look as follows:: contains #:def maxRelError_template(RANK, PREC) function maxRelError_${RANK}$_${PREC}$(obtained, reference) result(res) real(${PREC}$), intent(in) :: obtained${ranksuffix(RANK)}$ real(${PREC}$), intent(in) :: reference${ranksuffix(RANK)}$ real(${PREC}$) :: res #:if RANK == 0 res = abs((obtained - reference) / reference) #:else res = maxval(abs((obtained - reference) / reference)) #:endif end function maxRelError_${RANK}$_${PREC}$ #:enddef maxRelError_template #:for PREC in PRECISIONS #:for RANK in RANKS $:maxRelError_template(RANK, PREC) #:endfor #:endfor end module errorcalc *********************************** Integration into build environments *********************************** Fypp can be integrated into build environments like any other preprocessor. If your build environment is Python-based, you may consider to access its functionality directly via its API instead of calling it as an external script (see the `API documentation`_). Make ==== In traditional make based system you can define an appropriate preprocessor rule in your ``Makefile``:: .fpp.f90: fypp $(FYPPFLAGS) $< $@ or for GNU make:: %.f90: %.fpp fypp $(FYPPFLAGS) $< $@ Waf === For the `waf` build system the Fypp source tree contains extension modules in the folder ``tools/waf``. They use Fypps Python API, therefore, the ``fypp`` module must be accessible from Python. Using those waf modules, you can formulate a Fypp preprocessed Fortran build like the example below:: def options(opt): opt.load('compiler_fc') opt.load('fortran_fypp') def configure(conf): conf.load('compiler_fc') conf.load('fortran_fypp') def build(bld): sources = bld.path.ant_glob('*.fpp') bld( features='fypp fc fcprogram', source=sources, target='myprog' ) Check the documentation in the corresponding waf modules for further details. CMake ===== One possible way of invoking the Fypp preprocessor within the CMake build framework is demonstrated below (thanks to Jacopo Chevallard for providing the very first version of this example):: ### Pre-process: .fpp -> .f90 via Fypp # Create a list of the files to be preprocessed set(fppFiles file1.fpp file2.fpp file3.fpp) # Pre-process foreach(infileName IN LISTS fppFiles) # Generate output file name string(REGEX REPLACE ".fpp\$" ".f90" outfileName "${infileName}") # Create the full path for the new file set(outfile "${CMAKE_CURRENT_BINARY_DIR}/${outfileName}") # Generate input file name set(infile "${CMAKE_CURRENT_SOURCE_DIR}/${infileName}") # Custom command to do the processing add_custom-command( OUTPUT "${outfile}" COMMAND fypp "${infile}" "${outfile}" MAIN_DEPENDENCY "${infile}" VERBATIM) # Finally add output file to a list set(outFiles ${outFiles} "${outfile}") endforeach(infileName) ***************** API documentation ***************** Additional to its usage as a command line tool, Fypp can also be operated directly from Python. This can be especially practical, when Fypp is used in a Python driven build environment (e.g. waf, Scons). Below you find the detailed documentation of the API Fypp offers. fypp module =========== .. automodule:: fypp Fypp ==== .. autoclass:: Fypp :members: FyppOptions =========== .. autoclass:: FyppOptions :members: get_option_parser() =================== .. autofunction:: get_option_parser() FyppError ========= .. autoclass:: FyppError :members: ***** Notes ***** .. [#] I am indebted to pyratemps author Roland Koebler for some helpful discussions. fypp-3.1/docs/index.rst000066400000000000000000000001471403725613300151320ustar00rootroot00000000000000################## Fypp documentation ################## .. toctree:: :maxdepth: 2 fypp fypp-3.1/setup.py000066400000000000000000000025211403725613300140510ustar00rootroot00000000000000# -*- coding: utf-8 -*- from setuptools import setup from codecs import open from os import path here = path.abspath(path.dirname(__file__)) with open(path.join(here, 'README.rst'), encoding='utf-8') as f: long_description = f.read() setup( name='fypp', version='3.1', description='Python powered Fortran preprocessor', long_description=long_description, url='https://github.com/aradi/fypp', author='Bálint Aradi', author_email='aradi@uni-bremen.de', license='BSD', classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'Intended Audience :: Science/Research', 'Topic :: Software Development :: Code Generators', 'Topic :: Software Development :: Pre-processors', 'License :: OSI Approved :: BSD License', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.9', ], keywords='fortran metaprogramming pre-processor', package_dir={'': 'src'}, py_modules=['fypp'], entry_points={ 'console_scripts': [ 'fypp=fypp:run_fypp', ], }, ) fypp-3.1/src/000077500000000000000000000000001403725613300131265ustar00rootroot00000000000000fypp-3.1/src/fypp.py000077700000000000000000000000001403725613300163352../bin/fyppustar00rootroot00000000000000fypp-3.1/test/000077500000000000000000000000001403725613300133165ustar00rootroot00000000000000fypp-3.1/test/include/000077500000000000000000000000001403725613300147415ustar00rootroot00000000000000fypp-3.1/test/include/assert.inc000066400000000000000000000002331403725613300167330ustar00rootroot00000000000000#:mute #:def ASSERT_CODE(code) $:code #:enddef ASSERT_CODE #:def ASSERT() ${_THIS_FILE_}$:${_THIS_LINE_}$|${_FILE_}$:${_LINE_}$ #:enddef ASSERT #:endmute fypp-3.1/test/include/failingmacro.inc000066400000000000000000000001061403725613300200640ustar00rootroot00000000000000#:def failingmacro() before $:UNDEFINED_VARIABLE after #:enddef fypp-3.1/test/include/fypp1.inc000066400000000000000000000000671403725613300164760ustar00rootroot00000000000000INCL1 #:def incmacro(x) INCMACRO(${x}$) #:enddef INCL5 fypp-3.1/test/include/inimod2.py000066400000000000000000000000401403725613300166460ustar00rootroot00000000000000def get_version(): return 2 fypp-3.1/test/include/subfolder/000077500000000000000000000000001403725613300167265ustar00rootroot00000000000000fypp-3.1/test/include/subfolder/fypp2.inc000066400000000000000000000000061403725613300204550ustar00rootroot00000000000000FYPP2 fypp-3.1/test/include/subfolder/include_fypp1.inc000066400000000000000000000000261403725613300221610ustar00rootroot00000000000000#:include 'fypp1.inc' fypp-3.1/test/include/subfolder/include_fypp2.inc000066400000000000000000000000261403725613300221620ustar00rootroot00000000000000#:include 'fypp2.inc' fypp-3.1/test/inimod.py000066400000000000000000000000401403725613300151410ustar00rootroot00000000000000def get_version(): return 1 fypp-3.1/test/inimod2.py000066400000000000000000000000401403725613300152230ustar00rootroot00000000000000def get_version(): return 1 fypp-3.1/test/runtests.sh000077500000000000000000000014061403725613300155450ustar00rootroot00000000000000#!/bin/bash testdir="$(dirname $0)" if [ $# -gt 0 ]; then pythons=$* else pythons="python" fi root=".." if [ -z "$PYTHONPATH" ]; then export PYTHONPATH="$root/src" else export PYTHONPATH="$root/src:$PYTHONPATH" fi cd $testdir failed="0" failing_pythons="" for python in $pythons; do echo "Testing with interpreter '$python'" $python test_fypp.py exitcode=$? if [ $exitcode != 0 ]; then failed="$(($failed + 1))" if [ -z "$failing_pythons" ]; then failing_pythons=$python else failing_pythons="$failing_pythons, $python" fi fi done echo if [ $failed -gt 0 ]; then echo "Failing test runs: $failed" >&2 echo "Failing interpreter(s): $failing_pythons" >&2 exit 1 else echo "All test runs finished successfully" exit 0 fi fypp-3.1/test/test_fypp.py000066400000000000000000002302051403725613300157070ustar00rootroot00000000000000'''Unit tests for testing Fypp.''' import sys import platform import unittest import fypp def _linenum(linenr, fname=None, flag=None): if fname is None: fname = fypp.STRING return fypp.linenumdir_cpp(linenr, fname, flag) def _defvar(var, val): return '-D{0}={1}'.format(var, val) def _incdir(path): return '-I{0}'.format(path) def _linelen(linelen): return '-l{0}'.format(linelen) def _indentation(ind): return '--indentation={0}'.format(ind) def _folding(fold): return '-f{0}'.format(fold) def _moddir(path): return '-M{0}'.format(path) def _linenumbering(nummode): return '-N{0}'.format(nummode) def _linenum_gfortran5(): return '--line-marker-format=gfortran5' def _linenum_std(): return '--line-marker-format=std' def _importmodule(module): return '-m{0}'.format(module) _LINENUM_FLAG = '-n' _FIXED_FORMAT_FLAG = '--fixed-format' _NO_FOLDING_FLAG = '-F' _NEW_FILE = 1 _RETURN_TO_FILE = 2 # Various basic tests # # Each test consists of a tuple containing the test name and a tuple with the # arguments of the get_test_output_method() routine. # SIMPLE_TESTS = [ ('if_true', ([_defvar('TESTVAR', 1)], '#:if TESTVAR > 0\nTrue\n#:endif\n', 'True\n' ) ), ('if_false', ([_defvar('TESTVAR', 0)], '#:if TESTVAR > 0\nTrue\n#:endif\n', '' ) ), ('if_else_true', ([_defvar('TESTVAR', 1)], '#:if TESTVAR > 0\nTrue\n#:else\nFalse\n#:endif\n', 'True\n' ) ), ('if_else_false', ([_defvar('TESTVAR', 0)], '#:if TESTVAR > 0\nTrue\n#:else\nFalse\n#:endif\n', 'False\n' ) ), ('if_elif_true1', ([_defvar('TESTVAR', 1)], '#:if TESTVAR == 1\nTrue1\n#:elif TESTVAR == 2\nTrue2\n#:endif\n', 'True1\n' ) ), ('if_elif_true2', ([_defvar('TESTVAR', 2)], '#:if TESTVAR == 1\nTrue1\n#:elif TESTVAR == 2\nTrue2\n#:endif\n', 'True2\n' ) ), ('if_elif_false', ([_defvar('TESTVAR', 0)], '#:if TESTVAR == 1\nTrue1\n#:elif TESTVAR == 2\nTrue2\n#:endif\n', '' ) ), ('if_elif_else_true1', ([_defvar('TESTVAR', 1)], '#:if TESTVAR == 1\nTrue1\n#:elif TESTVAR == 2\nTrue2\n' '#:else\nFalse\n#:endif\n', 'True1\n' ) ), ('if_elif_else_true2', ([_defvar('TESTVAR', 2)], '#:if TESTVAR == 1\nTrue1\n#:elif TESTVAR == 2\nTrue2\n' '#:else\nFalse\n#:endif\n', 'True2\n' ) ), ('if_elif_else_false', ([_defvar('TESTVAR', 0)], '#:if TESTVAR == 1\nTrue1\n#:elif TESTVAR == 2\nTrue2\n' '#:else\nFalse\n#:endif\n', 'False\n' ) ), ('inline_if_true', ([_defvar('TESTVAR', 1)], '#{if TESTVAR > 0}#True#{endif}#Done', 'TrueDone' ) ), ('inline_if_false', ([_defvar('TESTVAR', 0)], '#{if TESTVAR > 0}#True#{endif}#Done', 'Done' ) ), ('inline_if_else_true', ([_defvar('TESTVAR', 1)], '#{if TESTVAR > 0}#True#{else}#False#{endif}#Done', 'TrueDone' ) ), ('inline_if_else_false', ([_defvar('TESTVAR', 0)], '#{if TESTVAR > 0}#True#{else}#False#{endif}#Done', 'FalseDone' ) ), ('inline_if_elif_true1', ([_defvar('TESTVAR', 1)], '#{if TESTVAR == 1}#True1#{elif TESTVAR == 2}#True2#{endif}#Done', 'True1Done' ) ), ('inline_if_elif_true2', ([_defvar('TESTVAR', 2)], '#{if TESTVAR == 1}#True1#{elif TESTVAR == 2}#True2#{endif}#Done', 'True2Done' ) ), ('inline_if_elif_false', ([_defvar('TESTVAR', 0)], '#{if TESTVAR == 1}#True1#{elif TESTVAR == 2}#True2#{endif}#Done', 'Done' ) ), ('inline_if_elif_else_true1', ([_defvar('TESTVAR', 1)], '#{if TESTVAR == 1}#True1#{elif TESTVAR == 2}#True2#{else}#False#{endif}#' 'Done', 'True1Done' ) ), ('inline_if_elif_else_true2', ([_defvar('TESTVAR', 2)], '#{if TESTVAR == 1}#True1#{elif TESTVAR == 2}#True2#{else}#False#{endif}#' 'Done', 'True2Done' ) ), ('inline_if_elif_else_false', ([_defvar('TESTVAR', 0)], '#{if TESTVAR == 1}#True1#{elif TESTVAR == 2}#True2#{else}#False#{endif}#' 'Done', 'FalseDone' ) ), ('linesub_eol', ([_defvar('TESTVAR', 1)], 'A\n$: TESTVAR + 1\nB\n', 'A\n2\nB\n' ) ), ('linesub_contlines', ([_defvar('TESTVAR', 1)], '$: TESTVAR & \n & + 1\n', '2\n' ) ), ('linesub_contlines2', ([_defvar('TESTVAR', 1)], '$: TEST& \n &VAR & \n & + 1\n', '2\n' ) ), ('linesub_contlines_contchar1', ([], '$: \'hello&\n world\'\n', 'hello world\n' ) ), ('linesub_contlines2_contchar1', ([], '$: \'hello&\n world&\n !\'\n', 'hello world !\n' ) ), ('exprsub', ([_defvar('TESTVAR', 1)], 'A${TESTVAR}$B${TESTVAR + 1}$C', 'A1B2C' ) ), ('exprsub_ignored_contlines', ([_defvar('TESTVAR', 1)], 'A${TEST&\n &VAR}$B${TESTVAR + 1}$C', 'A${TEST&\n &VAR}$B2C' ) ), ('macrosubs', ([], '#:def macro(var)\nMACRO|${var}$|\n#:enddef\n${macro(1)}$', 'MACRO|1|' ) ), ('macrosubs_named_enddef', ([], '#:def macro(var)\nMACRO|${var}$|\n#:enddef macro\n${macro(1)}$', 'MACRO|1|' ) ), ('macrodef_whitespace', ([], '#:def macro (var)\nMACRO|${var}$|\n#:enddef macro\n${macro(1)}$', 'MACRO|1|' ) ), ('macro_noargs', ([], '#:def macro()\nMACRO\n#:enddef\n${macro()}$', 'MACRO' ) ), ('recursive_macrosubs', ([], '#:def macro(var)\nMACRO|${var}$|\n#:enddef\n${macro(macro(1))}$', 'MACRO|MACRO|1||' ) ), ('macrosubs_extvarsubs', ([_defvar('TESTVAR', 1)], '#:def macro(var)\nMACRO|${var}$-${TESTVAR}$|\n#:enddef\n${macro(2)}$', 'MACRO|2-1|' ) ), ('macro_trailing_newlines', ([], '#:def macro()\nL1\n\n#:enddef\n$: macro()\n', 'L1\n\n', ) ), ('macro_trailing_newlines_inline', ([], '#:def macro()\nL1\n\n#:enddef\n|${macro()}$|', '|L1\n|', ) ), ('macro_call_named_arguments', ([], '#:def mymacro(A, B)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '$:mymacro(B=1, A=2)\n', 'A=2,B=1\n' ) ), ('macro_call_positional_and_named_arguments', ([], '#:def mymacro(A, B, C)\nA=${A}$,B=${B}$,C=${C}$\n#:enddef mymacro\n'\ '$:mymacro(1, C=3, B=2)\n', 'A=1,B=2,C=3\n' ) ), ('optarg_macro_call_all_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '#:call mymacro\n1\n#:nextarg\n2\n#:endcall\n', 'A=1,B=2\n' ) ), ('optarg_macro_block_all_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '#:block mymacro\n1\n#:contains\n2\n#:endblock\n', 'A=1,B=2\n' ) ), ('optarg_macro_call_missing_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '#:call mymacro\n1\n#:endcall\n', 'A=1,B=2\n' ) ), ('optarg_macro_block_missing_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '#:block mymacro\n1\n#:endblock\n', 'A=1,B=2\n' ) ), ('optarg_macro_eval_call_all_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '$:mymacro(1, 2)\n', 'A=1,B=2\n' ) ), ('optarg_macro_eval_call_missing_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '$:mymacro(1)\n', 'A=1,B=2\n' ) ), ('optarg_macro_direct_call_all_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '@:mymacro(1, 2)\n', 'A=1,B=2\n' ) ), ('optarg_macro_direct_call_all_args_inline', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '@{mymacro(1, 2)}@', 'A=1,B=2' ) ), ('optarg_macro_direct_call_missing_args', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '@:mymacro(1)\n', 'A=1,B=2\n' ) ), ('optarg_macro_direct_call_missing_args_inline', ([], '#:def mymacro(A, B=2)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '|@{mymacro(1)}@|', '|A=1,B=2|' ) ), ('optarg_macro_tuple_as_default', ([], '#:def macro(X, Y=2, Z=(1,2==3))\nX=${X}$,Y=${Y}$,Z=${Z[0]}$,${Z[1]}$\n'\ '#:enddef\n@:macro(1)\n', 'X=1,Y=2,Z=1,False\n' ) ), ('macro_vararg_no_varargs', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '$:macro(1, 2)\n', '|12[]|\n' ) ), ('macro_vararg_one_vararg', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '$:macro(1, 2, 3)\n', '|12[3]|\n' ) ), ('macro_vararg_two_varargs', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '$:macro(1, 2, 3, 4)\n', '|12[3, 4]|\n' ) ), ('macro_vararg_named_arguments_eval', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '$:macro(y=2, x=1)\n', '|12[]|\n' ) ), ('macro_vararg_named_arguments_directcall', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '@:macro(y=2, x=1)\n', '|12[]|\n' ) ), ('macro_vararg_named_arguments_call', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '#:call macro\n#:nextarg y\n2\n#:nextarg x\n1\n#:endcall\n', '|12[]|\n' ) ), ('macro_vararg_named_arguments_block', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '#:block macro\n#:contains y\n2\n#:contains x\n1\n#:endblock\n', '|12[]|\n' ) ), ('macro_vararg_named_arguments_inline_call', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '#{call macro}##{nextarg y}#2#{nextarg x}#1#{endcall}#', '|12[]|' ) ), ('macro_vararg_named_arguments_inline_block', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '#{block macro}##{contains y}#2#{contains x}#1#{endblock}#', '|12[]|' ) ), ('macro_vararg_mixed_arguments_eval', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '$:macro(1, z=3, y=2)\n', '|123[]|\n' ) ), ('macro_vararg_mixed_arguments_directcall', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '@:macro(1, z=3, y=2)\n', '|123[]|\n' ) ), ('macro_vararg_mixed_arguments_call', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#:call macro\n1\n#:nextarg z\n3\n#:nextarg y\n2\n#:endcall\n', '|123[]|\n' ) ), ('macro_vararg_mixed_arguments_block', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#:block macro\n1\n#:contains z\n3\n#:contains y\n2\n#:endblock\n', '|123[]|\n' ) ), ('macro_vararg_mixed_arguments_call2', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#:call macro\n#:nextarg\n1\n#:nextarg z\n3\n#:nextarg y\n2\n#:endcall\n', '|123[]|\n' ) ), ('macro_vararg_mixed_arguments_block2', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#:block macro\n'\ '#:contains\n1\n'\ '#:contains z\n3\n'\ '#:contains y\n'\ '2\n'\ '#:endblock\n', '|123[]|\n' ) ), ('macro_vararg_mixed_arguments_inline_call', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#{call macro}#1#{nextarg z}#3#{nextarg y}#2#{endcall}#', '|123[]|' ) ), ('macro_vararg_mixed_arguments_inline_block', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#{block macro}#1#{contains z}#3#{contains y}#2#{endblock}#', '|123[]|' ) ), ('macro_vararg_mixed_arguments_inline_call2', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#{call macro}##{nextarg}#1#{nextarg z}#3#{nextarg y}#2#{endcall}#', '|123[]|' ) ), ('macro_vararg_mixed_arguments_inline_block2', ([], '#:def macro(x, y, z, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '#{block macro}##{contains}#1#{contains z}#3#{contains y}#2#{endblock}#', '|123[]|' ) ), ('macro_varpos_varkw_with_keyword_arguments', ([], '#:def macro(x, y, *vararg, **varkw)\n'\ '|${x}$${y}$${varkw["z"]}$${vararg}$|\n'\ '#:enddef\n'\ '$:macro(1, 2, z=3)\n', '|123[]|\n' ) ), ('macro_varpos_varkw_with_pos_arguments', ([], '#:def macro(x, y, *vararg, **varkw)\n'\ '|${x}$${y}$${vararg}$|\n'\ '#:enddef\n'\ '$:macro(1, 2, 4, 5)\n', '|12[4, 5]|\n' ) ), ('macro_varpos_varkw_with_pos_and_kw_arguments', ([], '#:def macro(x, y, *vararg, **varkw)\n'\ '|${x}$${y}$${varkw["z"]}$${vararg}$|\n'\ '#:enddef\n'\ '$:macro(1, 2, 4, 5, z=3)\n', '|123[4, 5]|\n' ) ), ('for', ([], '#:for i in (1, 2, 3)\n${i}$\n#:endfor\n', '1\n2\n3\n' ) ), ('for_macro', ([], '#:def mymacro(val)\nVAL:${val}$\n#:enddef\n' '#:for i in (1, 2, 3)\n$: mymacro(i)\n#:endfor\n', 'VAL:1\nVAL:2\nVAL:3\n' ) ), ('inline_for', ([], '#{for i in (1, 2, 3)}#${i}$#{endfor}#Done\n', '123Done\n' ) ), ('inline_for_macro', ([], '#:def mymacro(val)\nVAL:${val}$\n#:enddef\n' '#{for i in (1, 2, 3)}#${mymacro(i)}$#{endfor}#Done\n', 'VAL:1VAL:2VAL:3Done\n' ) ), ('call_directive', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:call mymacro\nL1\nL2\nL3\n#:endcall\n', '|L1\nL2\nL3|\n', ) ), ('block_directive', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:block mymacro\nL1\nL2\nL3\n#:endblock\n', '|L1\nL2\nL3|\n', ) ), ('call_directive_named_endcall', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:call mymacro\nL1\nL2\nL3\n#:endcall mymacro\n', '|L1\nL2\nL3|\n', ) ), ('block_directive_named_endblock', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:block mymacro\nL1\nL2\nL3\n#:endblock mymacro\n', '|L1\nL2\nL3|\n', ) ), ('inine_call_directive_named_endcall', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#{call mymacro}#L1 L2 L3#{endcall mymacro}#', '|L1 L2 L3|', ) ), ('inine_block_directive_named_endblock', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#{block mymacro}#L1 L2 L3#{endblock mymacro}#', '|L1 L2 L3|', ) ), ('call_directive_quotation', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:call mymacro\n"""L1"""\nL2\nL3\n#:endcall\n', '|"""L1"""\nL2\nL3|\n', ) ), ('block_directive_quotation', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:block mymacro\n"""L1"""\nL2\nL3\n#:endblock\n', '|"""L1"""\nL2\nL3|\n', ) ), ('call_directive_backslash_escape1', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:call mymacro\nL1\\n\nL2\nL3\n#:endcall\n', '|L1\\n\nL2\nL3|\n', ) ), ('block_directive_backslash_escape1', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:block mymacro\nL1\\n\nL2\nL3\n#:endblock\n', '|L1\\n\nL2\nL3|\n', ) ), ('call_directive_backslash_escape2', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:call mymacro\nL1\\"a\\"\\n\nL2\nL3\n#:endcall\n', '|L1\\"a\\"\\n\nL2\nL3|\n', ) ), ('block_directive_backslash_escape2', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '#:block mymacro\nL1\\"a\\"\\n\nL2\nL3\n#:endblock\n', '|L1\\"a\\"\\n\nL2\nL3|\n', ) ), ('call_directive_2_args', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '#:call mymacro\n"""L1"""\nL2\n#:nextarg\nL3\n#:endcall\n', '|"""L1"""\nL2|L3|\n', ) ), ('block_directive_2_args', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '#:block mymacro\n"""L1"""\nL2\n#:contains\nL3\n#:endblock\n', '|"""L1"""\nL2|L3|\n', ) ), ('call_directive_2_args_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '#{call mymacro}#A1#{nextarg}#A2#{endcall}#', '|A1|A2|', ) ), ('block_directive_2_args_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '#{block mymacro}#A1#{contains}#A2#{endblock}#', '|A1|A2|', ) ), ('call_lambda_func', ([], '#:set convert = lambda s: s.lower()\n'\ '#:call convert\nHELLO\n#:endcall\n', 'hello\n', ) ), ('call_no_header_arg_no_body_arg', ([], '#:def macro0()\nNOARG\n#:enddef\n'\ '#:call macro0()\n#:endcall\n', 'NOARG\n', ) ), ('block_no_header_arg_no_body_arg', ([], '#:def macro0()\nNOARG\n#:enddef\n'\ '#:block macro0()\n#:endblock\n', 'NOARG\n', ) ), ('call_header_pos_arg_no_body_arg', ([], '#:def macro(arg)\n|${arg}$|\n#:enddef\n'\ '#:call macro("h1")\n#:endcall\n', '|h1|\n', ) ), ('block_header_pos_arg_no_body_arg', ([], '#:def macro(arg)\n|${arg}$|\n#:enddef\n'\ '#:block macro("h1")\n#:endblock\n', '|h1|\n', ) ), ('call_header_kwarg_no_body_arg', ([], '#:def macro(arg)\n|${arg}$|\n#:enddef\n'\ '#:call macro(arg="h1")\n#:endcall\n', '|h1|\n', ) ), ('block_header_kwarg_no_body_arg', ([], '#:def macro(arg)\n|${arg}$|\n#:enddef\n'\ '#:block macro(arg="h1")\n#:endblock\n', '|h1|\n', ) ), ('call_header_mixed_args_no_body_arg', ([], '#:def macro(arg, arg2)\n|${arg}$|${arg2}$|\n#:enddef\n'\ '#:call macro("h1", arg2="h2")\n#:endcall\n', '|h1|h2|\n', ) ), ('block_header_mixed_args_no_body_arg', ([], '#:def macro(arg, arg2)\n|${arg}$|${arg2}$|\n#:enddef\n'\ '#:block macro("h1", arg2="h2")\n#:endblock\n', '|h1|h2|\n', ) ), ('call_header_mixed_args_body_pos_arg', ([], '#:def macro(arg, arg2, arg3)\n|${arg}$|${arg2}$|${arg3}$|\n#:enddef\n'\ '#:call macro("h1", arg3="h3")\nB1\n#:endcall\n', '|h1|B1|h3|\n', ) ), ('block_header_mixed_args_body_pos_arg', ([], '#:def macro(arg, arg2, arg3)\n|${arg}$|${arg2}$|${arg3}$|\n#:enddef\n'\ '#:block macro("h1", arg3="h3")\nB1\n#:endblock\n', '|h1|B1|h3|\n', ) ), ('call_header_kwargs_body_pos_arg', ([], '#:def macro(arg, arg2, arg3)\n|${arg}$|${arg2}$|${arg3}$|\n#:enddef\n'\ '#:call macro(arg3="h3", arg2="h2")\nB1\n#:endcall\n', '|B1|h2|h3|\n', ) ), ('block_header_kwargs_body_pos_arg', ([], '#:def macro(arg, arg2, arg3)\n|${arg}$|${arg2}$|${arg3}$|\n#:enddef\n'\ '#:block macro(arg3="h3", arg2="h2")\nB1\n#:endblock\n', '|B1|h2|h3|\n', ) ), ('call_header_kwargs_body_kwarg', ([], '#:def macro(arg1, arg2, arg3)\n|${arg1}$|${arg2}$|${arg3}$|\n#:enddef\n'\ '#:call macro(arg1="h1", arg3="h3")\n#:nextarg arg2\nB1\n#:endcall\n', '|h1|B1|h3|\n', ) ), ('block_header_kwargs_body_kwarg', ([], '#:def macro(arg1, arg2, arg3)\n|${arg1}$|${arg2}$|${arg3}$|\n#:enddef\n'\ '#:block macro(arg1="h1", arg3="h3")\n#:contains arg2\nB1\n#:endblock\n', '|h1|B1|h3|\n', ) ), ('direct_call', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(a < b)\n', '|a < b|\n', ) ), ('direct_call_whitespace', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro (a < b)\n', '|a < b|\n', ) ), ('direct_call_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro(a < b)}@', '|a < b|', ) ), ('direct_call2', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(a < b )\n', '|a < b|\n', ) ), ('direct_call2_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro(a < b )}@', '|a < b|', ) ), ('direct_call3', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro( a < b)\n', '|a < b|\n', ) ), ('direct_call3_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro( a < b)}@', '|a < b|', ) ), ('direct_call4', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(a < b)\n', '|a < b|\n', ) ), ('direct_call4_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro(a < b)}@', '|a < b|', ) ), ('direct_call_contline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(a &\n &< b&\n &)\n', '|a < b|\n', ) ), ('direct_call_quotation', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro( """L1""" )\n', '|"""L1"""|\n', ) ), ('direct_call_quotation_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro( """L1""" )}@', '|"""L1"""|', ) ), ('direct_call_backslash_escape1', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(L1\\n)\n', '|L1\\n|\n', ) ), ('direct_call_backslash_escape1_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro(L1\\n)}@', '|L1\\n|', ) ), ('direct_call_backslash_escape2', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(L1\\"a\\"\\n)\n', '|L1\\"a\\"\\n|\n', ) ), ('direct_call_backslash_escape2_inline', ([], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@{mymacro(L1\\"a\\"\\n)}@', '|L1\\"a\\"\\n|', ) ), ('direct_call_2_args', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro("""L1""", L2)\n', '|"""L1"""|L2|\n', ) ), ('direct_call_2_args_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro("""L1""", L2)}@', '|"""L1"""|L2|', ) ), ('direct_call_2_args_escape1', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro("""L1"""","L2, L3)\n', '|"""L1"""","L2|L3|\n', ) ), ('direct_call_2_args_escape1_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro("""L1"""","L2, L3)}@', '|"""L1"""","L2|L3|', ) ), ('direct_call_2_args_escape2', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro((L1, L2), L3)\n', '|(L1, L2)|L3|\n', ) ), ('direct_call_2_args_escape2_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro((L1, L2), L3)}@', '|(L1, L2)|L3|', ) ), ('direct_call_2_args_escape3', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro({L1, L2}, L3)\n', '|L1, L2|L3|\n', ) ), ('direct_call_2_args_escape3_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro({L1, L2}, L3)}@', '|L1, L2|L3|', ) ), ('direct_call_2_args_escape4', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro([L1, L2], L3)\n', '|[L1, L2]|L3|\n', ) ), ('direct_call_2_args_escape4_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro([L1, L2], L3)}@', '|[L1, L2]|L3|', ) ), ('direct_call_2_args_escape5', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro("L1, L2", L3)\n', '|"L1, L2"|L3|\n', ) ), ('direct_call_2_args_escape5_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro("L1, L2", L3)}@', '|"L1, L2"|L3|', ) ), ('direct_call_2_args_escape6', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro(\'L1, L2\', L3)\n', '|\'L1, L2\'|L3|\n', ) ), ('direct_call_2_args_escape6_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro(\'L1, L2\', L3)}@', '|\'L1, L2\'|L3|', ) ), ('direct_call_2_args_escape7', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro(L1 ${2, 2}$, L3)\n', '|L1 (2, 2)|L3|\n', ) ), ('direct_call_2_args_escape7_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro(L1 ${2, 2}$, L3)}@', '|L1 (2, 2)|L3|', ) ), ('direct_call_2_args_escape8', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro({{L1, L2}}, L3)\n', '|{L1, L2}|L3|\n', ) ), ('direct_call_2_args_escape8_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro({{L1, L2}}, L3)}@', '|{L1, L2}|L3|', ) ), ('direct_call_kwarg', ([], '#:def mymacro(a)\n|${a}$|\n#:enddef\n'\ '@:mymacro(a = b)\n', '|b|\n', ) ), ('direct_call_kwarg_eq_operator', ([], '#:def mymacro(a)\n|${a}$|\n#:enddef\n'\ '@:mymacro(a == b)\n', '|a == b|\n', ) ), ('direct_call_kwarg_ptr_assignment', ([], '#:def mymacro(a)\n|${a}$|\n#:enddef\n'\ '@:mymacro(a => b)\n', '|> b|\n', ) ), ('direct_call_kwarg_escape', ([], '#:def mymacro(val1)\n|${val1}$|\n#:enddef\n'\ '@:mymacro({a = b})\n', '|a = b|\n', ) ), ('direct_call_varsubs', ([], '#:def mymacro(val1)\n|${val1}$|\n#:enddef\n'\ '@:mymacro(2x2=${2*2}$)\n', '|2x2=4|\n', ) ), ('direct_call_varsubs_inline', ([], '#:def mymacro(val1)\n|${val1}$|\n#:enddef\n'\ '@{mymacro(2x2=${2*2}$)}@', '|2x2=4|', ) ), ('direct_call_varsubs_2_args', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro(${2*1}$, ${2*2}$)\n', '|2|4|\n', ) ), ('direct_call_varsubs_2_args_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro(${2*1}$, ${2*2}$)}@', '|2|4|', ) ), ('direct_call_varsubs_2_args_escape', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro((${2*1}$, ${2*2}$), ${2*3}$)\n', '|(2, 4)|6|\n', ) ), ('direct_call_varsubs_2_args_escape_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro((${2*1}$, ${2*2}$), ${2*3}$)}@', '|(2, 4)|6|', ) ), ('direct_call_no_param', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '@:mymacro()\n', '||\n' ) ), ('direct_call_no_param_inline', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '@{mymacro()}@', '||' ) ), ('direct_call_no_param2', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '@:mymacro( )\n', '||\n' ) ), ('direct_call_no_param2_inline', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '@{mymacro( )}@', '||' ) ), ('call_no_param_inline', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '#{call mymacro}##{endcall}#\n', '||\n' ) ), ('block_no_param_inline', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '#{block mymacro}##{endblock}#\n', '||\n' ) ), ('call_no_param', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '#:call mymacro\n#:endcall\n', '||\n' ) ), ('block_no_param', ([], '#:def mymacro()\n||\n#:enddef mymacro\n'\ '#:block mymacro\n#:endblock\n', '||\n' ) ), ('call_empty_param_inline', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n'\ '#{call mymacro}# #{endcall}#\n', '| |\n' ) ), ('block_empty_param_inline', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n'\ '#{block mymacro}# #{endblock}#\n', '| |\n' ) ), ('call_empty_param', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n'\ '#:call mymacro\n\n#:endcall\n', '||\n' ) ), ('block_empty_param', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n'\ '#:block mymacro\n\n#:endblock\n', '||\n' ) ), ('call_empty_param_directcall', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n'\ '@:mymacro({})\n', '||\n' ) ), ('call_whitespace_param_directcall', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n'\ '@:mymacro({ })\n', '| |\n' ) ), ('comment_single', ([], ' #! Comment here\nDone\n', 'Done\n', ) ), ('comment_multiple', ([], ' #! Comment1\n#! Comment2\nDone\n', 'Done\n', ) ), ('set', ([], '#:set x = 2\n$: x\n', '2\n', ) ), ('set_no_rhs', ([], '#:set x\n$:x\n', '\n', ) ), ('set_equal_sign_nospace', ([], '#:set x=2\n$: x\n', '2\n', ) ), ('set_equal_sign_withspace', ([], '#:set x = 2\n$: x\n', '2\n', ) ), ('inline_set_equal_withspace', ([], '#{set x = 2}#${x}$Done\n', '2Done\n', ) ), ('inline_set_equal_nospace', ([], '#{set x=2}#${x}$Done\n', '2Done\n', ) ), ('set_function', ([], '$:setvar("x", 2)\n${x}$\nDone\n', "\n2\nDone\n", ) ), ('set_function_tuple', ([], '$:setvar("x, y", (2, 3))\n${x}$${y}$\nDone\n', "\n23\nDone\n", ) ), ('set_function_tuple2', ([], '$:setvar("(x, y)", (2, 3))\n${x}$${y}$\nDone\n', "\n23\nDone\n", ) ), ('set_function_multiple_args', ([], '$:setvar("x", 2, "y", 3)\n${x}$${y}$\nDone\n', "\n23\nDone\n", ) ), ('getvar_existing_value', ([_defvar('VAR', '\"VAL\"')], '$:getvar("VAR", "DEFAULT")\n', 'VAL\n', ) ), ('getvar_default_value', ([], '$:getvar("VAR", "DEFAULT")\n', 'DEFAULT\n', ) ), ('getvar_local_scope', ([], '#:set X = 1\n'\ '#:def test()\n$:getvar("X")\n#:set X = 2\n$:getvar("X")\n#:enddef\n'\ '$:test()\n', '1\n2\n', ) ), ('del_existing_variable', ([], '#:set X = 12\n$:defined("X")\n#:del X\n$:defined("X")\n', 'True\nFalse\n', ) ), ('del_variable_tuple', ([], '#:set X = 1\n#:set Y = 2\n${defined("X")}$${defined("Y")}$\n'\ '#:del X, Y\n${defined("X")}$${defined("Y")}$\n', 'TrueTrue\nFalseFalse\n', ) ), ('del_variable_local_scope', ([], '#:set echo = lambda s: s\n#:set X = 1\n'\ '#:call echo\n$:X\n#:set X = 2\n$:X\n$:defined("X")\n'\ '#:del X\n$:defined("X")\n#:endcall\n$:X\n', '1\n2\nTrue\nFalse\n1\n', ) ), ('del_macro', ([], '#:def mymacro(txt)\n|${txt}$|\n#:enddef mymacro\n$:defined("mymacro")\n'\ '$:mymacro("A")\n#:del mymacro\n$:defined("mymacro")\n', 'True\n|A|\nFalse\n', ) ), ('del_inline', ([], '#:set X = 12\n$:defined("X")\n#{del X}#${defined("X")}$\n', 'True\nFalse\n', ) ), ('del_inline_tuple', ([], '#:set X = 1\n#:set Y = 2\n${defined("X")}$${defined("Y")}$\n'\ '#{del X, Y}#${defined("X")}$${defined("Y")}$\n', 'TrueTrue\nFalseFalse\n', ) ), ('delvar_function', ([], '#:set X = 12\n$:defined("X")\n$:delvar("X")\n$:defined("X")\n', 'True\n\nFalse\n', ) ), ('delvar_function_tuple', ([], '#:set X = 1\n#:set Y = 2\n${defined("X")}$${defined("Y")}$\n'\ '$:delvar("X, Y")\n${defined("X")}$${defined("Y")}$\n', 'TrueTrue\n\nFalseFalse\n', ) ), ('delvar_function_multiple_args', ([], '#:set X = 1\n#:set Y = 2\n${defined("X")}$${defined("Y")}$\n'\ '$:delvar("X", "Y")\n${defined("X")}$${defined("Y")}$\n', 'TrueTrue\n\nFalseFalse\n', ) ), ('mute', ([], 'A\n#:mute\nB\n#:set VAR = 2\n#:endmute\nVAR=${VAR}$\n', 'A\nVAR=2\n' ) ), ('builtin_var_line', ([], '${_LINE_}$', '1' ) ), ('builtin_var_file', ([], '${_FILE_}$', fypp.STRING ) ), ('builtin_var_line_in_lineeval', ([], '$:_LINE_\n', '1\n' ) ), ('builtin_var_system', ([], '${_SYSTEM_}$', platform.system() ) ), ('builtin_var_machine', ([], '${_MACHINE_}$', platform.machine() ) ), ('escaped_control_inline', ([], r'A#\{if False}\#B#\{endif}\#', 'A#{if False}#B#{endif}#' ) ), ('escaped_control_line', ([], '#\\:if False\n', '#:if False\n' ) ), ('escaped_eval_inline', ([], r'A$\{1 + 1}\$', 'A${1 + 1}$' ) ), ('escaped_eval_line', ([], '$\\: 1 + 1\n', '$: 1 + 1\n' ) ), ('multi_escape', ([], r'$\\\{1 + 1}\\$', r'$\\{1 + 1}\$' ) ), ('escape_direct_call', ([], '@\\:assertTrue(x > y)\n', '@:assertTrue(x > y)\n' ) ), ('escape_direct_call_inline', ([], '@\\{assertTrue(x > y)}@', '@{assertTrue(x > y)}@' ) ), ('escape_comment', ([], 'A\n #\! Comment\n', 'A\n #! Comment\n', ) ), ('fold_lines', ([_linelen(10), _indentation(2), _folding('simple')], 'This line is not folded\nThis line ${1 + 1}$ is folded\n', 'This line is not folded\nThis line&\n & 2 is &\n &folded\n' ) ), ('prevent_comment_folding', ([_linelen(10), _indentation(2), _folding('simple')], '#:def macro()\n ! Should be not folded\nShould be folded\n#:enddef\n' '$:macro()\n', ' ! Should be not folded\nShould be&\n & folded\n' ) ), ('no_folding', ([_linelen(15), _indentation(4), _NO_FOLDING_FLAG], ' ${3}$456 89 123456 8', ' 3456 89 123456 8', ) ), ('brute_folding', ([_linelen(15), _indentation(4), _folding('brute')], ' ${3}$456 89 123456 8', ' 3456 89 1234&\n &56 8', ) ), ('simple_folding', ([_linelen(15), _indentation(4), _folding('simple')], ' ${3}$456 89 123456 8', ' 3456 89 1234&\n &56 8', ) ), ('smart_folding', ([_linelen(15), _indentation(4), _folding('smart')], ' ${3}$456 89 123456 8', ' 3456 89&\n & 123456&\n & 8', ) ), ('fixed_format_folding', ([_FIXED_FORMAT_FLAG], ' print *, ${\'aa\'}$, bb, cc, dd, ee, ff, gg, hh, ii, jj, kk, ll, ' 'mm, nn, oo, pp, qq, rr, ss, tt\n', ' print *, aa, bb, cc, dd, ee, ff, gg, hh, ii, jj, kk, ll, mm, nn, ' 'o\n &o, pp, qq, rr, ss, tt\n', ) ), ('tuple_assignment', ([], '#:set mytuple = (1, 2, 3)\n#:set a, b, c = mytuple\n${a}$${b}$${c}$\n', '123\n' ) ), ('tuple_assignment2', ([], '#:set a, b, c = (1, 2, 3)\n${a}$${b}$${c}$\n', '123\n' ) ), ('tuple_assignment3', ([], '#:set a, b, c = 1, 2, 3\n${a}$${b}$${c}$\n', '123\n' ) ), ('tuple_assignment_nospace', ([], '#:set a,b,c = (1, 2, 3)\n${a}$${b}$${c}$\n', '123\n' ) ), ('tuple_assignment_vartuple', ([], '#:set (a, b, c) = (1, 2, 3)\n${a}$${b}$${c}$\n', '123\n' ) ), ('tuple_assignment_vartuple2', ([], '#:set ( a, b, c ) = (1, 2, 3)\n${a}$${b}$${c}$\n', '123\n' ) ), ('inline_tuple_assignment', ([], '#{set a, b, c = 1, 2, 3}#${a}$${b}$${c}$\n', '123\n' ) ), ('inline_tuple_assignment_vartuple', ([], '#{set (a, b, c) = 1, 2, 3}#${a}$${b}$${c}$\n', '123\n' ) ), ('whitespace_but_no_param', ([], '#:if True\nOK\n#:endif \n', 'OK\n' ) ), ('whitespace_but_no_param2', ([], '#:if True\nOK\n#:endif \n \n', 'OK\n \n' ) ), ('whitespace_but_no_param_inline', ([], '#{if True}#OK#{endif }#', 'OK' ) ), ('for_loop_scope', ([], '#{for i in range(4)}##{set X = i}##{endfor}#${X}$${i}$\n', '33\n' ) ), ('macro_scope', ([], '#:set X = 3\n#:def setx()\n#:set X = -5\n#:enddef\n$:setx()\n$:X\n', '\n3\n' ) ), ('local_macro_local_scope', ([], '#:set echo = lambda s: s\n'\ '#:set X = 3\n#:call echo\n'\ '#:def mymacro()\nX:${X}$\n#:enddef\n'\ '#:set X = 2\n$:mymacro()\n#:endcall\n', 'X:2\n', ) ), ('local_macro_global_scope', ([], '#:set echo = lambda s: s\n'\ '#:set X = 3\n#:call echo\n' '#:def mymacro()\nX:${X}$\n#:enddef\n'\ '$:mymacro()\n#:endcall\n', 'X:3\n', ) ), ('scope_global_macro_called_from_local_scope', ([], '#:set echo = lambda s: s\n'\ '#:def printX()\nX:${X}$\n#:enddef\n#:set X = 1\n'\ '#:call echo\n#:set X = 2\n'\ '#:call echo\n#:set X = 3\n$:printX()\n'\ '#:endcall\n#:endcall\nX:${X}$\n', 'X:1\nX:1\n', ) ), ('scope_macro_lookup_locals_in_definition_scope', ([], '#:set X = 0\n'\ '#:def macro1()\n#:set X = 1\n'\ '#:def macro2()\n'\ '#:def macro3a()\nX3a:${X}$\n#:enddef macro3a\n'\ '#:def macro3b()\n#:set X = 3\n$:macro3a()\n#:enddef macro3b\n'\ '#:set X = 2\n$:macro3b()\nX2:${X}$\n'\ '#:enddef macro2\n$:macro2()\nX1:${X}$\n'\ '#:enddef macro1\n$:macro1()\nX0:${X}$\n', 'X3a:2\nX2:2\nX1:1\nX0:0\n', ) ), ('scope_macro_lookup_locals_above_definition_scope', ([], '#:set X = 0\n'\ '#:def macro1()\n#:set X = 1\n'\ '#:def macro2()\n'\ '#:def macro3a()\nX3a:${X}$\n#:enddef macro3a\n'\ '#:def macro3b()\n#:set X = 3\n$:macro3a()\n#:enddef macro3b\n'\ '$:macro3b()\nX2:${X}$\n'\ '#:enddef macro2\n$:macro2()\nX1:${X}$\n'\ '#:enddef macro1\n$:macro1()\nX0:${X}$\n', 'X3a:1\nX2:1\nX1:1\nX0:0\n', ) ), ('scope_macro_lookup_locals_global_scope', ([], '#:set X = 0\n'\ '#:def macro1()\n'\ '#:def macro2()\n'\ '#:def macro3a()\nX3a:${X}$\n#:enddef macro3a\n'\ '#:def macro3b()\n#:set X = 3\n$:macro3a()\n#:enddef macro3b\n'\ '$:macro3b()\nX2:${X}$\n'\ '#:enddef macro2\n$:macro2()\nX1:${X}$\n'\ '#:enddef macro1\n$:macro1()\nX0:${X}$\n', 'X3a:0\nX2:0\nX1:0\nX0:0\n', ) ), ('scope_generator_within_macro', ([], '#:def foo()\n#:set b = 21\n$:sum([b for i in range(2)])\n#:enddef\n'\ '$:foo()\n', '42\n' ) ), ('correct_predefined_var_injection', ([], '#:def ASSERT(cond)\n"${cond}$", ${_FILE_}$, ${_LINE_}$\n#:enddef\n'\ '@:ASSERT(2 < 3)\n', '"2 < 3", ' + fypp.STRING + ', 4\n' ) ), ('correct_line_numbering_in_if', ([], '#:if _LINE_ == 1\nOK\n#:endif\n', 'OK\n' ) ), ('correct_line_numbering_in_for', ([], '#:for line in [_LINE_]\n${line}$ - ${_LINE_}$\n#:endfor\n', '1 - 2\n' ) ), ('line_numbering_macro', ([], '#:def macro()\n${_THIS_LINE_}$,${_LINE_}$\n#:enddef macro\n'\ '${_THIS_LINE_}$,${_LINE_}$|${macro()}$\n', '4,4|2,4\n' ) ), ('line_numbering_argeval', ([], "#:set func = lambda s: str(_THIS_LINE_) + ',' + str(_LINE_) + '|' + s\n"\ "#:call func\n${_THIS_LINE_}$,${_LINE_}$\n#:endcall\n", '2,2|3,3\n' ) ), ('line_numbering_argeval_macrocall', ([_incdir('include')], "#:include 'assert.inc'\n"\ "#:call ASSERT_CODE\n@:ASSERT()\n#:endcall ASSERT_CODE\n", 'include/assert.inc:7|:3\n' ) ), ('line_numbering_eval_within_macro', ([], '#:def m1(A)\n${_LINE_}$\n#:enddef\n'\ '#:def m2(A)\n#:call m1\n${A}$\n#:endcall\n#:enddef\n'\ '$:m2(1)\n', '9\n' ) ), ('global_existing', ([], '#:set A = 1\n#:def macro()\n#:global A\n#:set A = 2\n#:enddef macro\n'\ '$:macro()\n$:A\n', '\n2\n' ) ), ('global_non_existing', ([], '#:def macro()\n#:global A\n#:set A = 2\n#:enddef macro\n'\ '$:defined("A")\n$:macro()\n$:A\n', 'False\n\n2\n' ) ), ('global_non_existing_evaldir', ([], '#:def macro()\n$:globalvar("A")\n#:set A = 2\n#:enddef macro\n'\ '$:defined("A")\n$:macro()\n$:A\n', 'False\n\n2\n' ) ), ('global_non_existing_evaldir_tuple', ([], '#:def macro()\n$:globalvar("A, B")\n#:set A = 2\n#:set B = 3\n'\ '#:enddef macro\n'\ '$:defined("A")\n$:defined("B")\n$:macro()\n$:A\n$:B\n', 'False\nFalse\n\n2\n3\n' ) ), ('global_non_existing_evaldir_arglist', ([], '#:def macro()\n$:globalvar("A", "B")\n#:set A = 2\n#:set B = 3\n'\ '#:enddef macro\n'\ '$:defined("A")\n$:defined("B")\n$:macro()\n$:A\n$:B\n', 'False\nFalse\n\n2\n3\n' ) ), ('global_in_global_scope', ([], '#:set A = 1\n#:global A\n$:A\n', '1\n' ) ), ('global_without_assignment', ([], '#:def macro()\n#:global A\n#:enddef macro\n'\ '$:defined("A")\n$:macro()\n$:defined("A")\n', 'False\n\nFalse\n' ) ), ] # Tests with line enumerations # # Each test consists of a tuple containing the test name and a tuple with the # arguments of the get_test_output_method() routine. # LINENUM_TESTS = [ # Explicit test for line number marker format ('explicit_str_linenum_test', ([_LINENUM_FLAG], '', '# 1 ""\n', ) ), # Explicit test for line number marker format (GFortran5 compatibility) ('explicit_str_linenum_test_gfortran5', ([_LINENUM_FLAG, _linenum_gfortran5()], '', '# 1 "" 1\n', ) ), # Explicit test for standard line number marker format ('explicit_str_linenum_test_standard', ([_LINENUM_FLAG, _linenum_std()], '', '#line 1 ""\n', ) ), ('trivial', ([_LINENUM_FLAG], 'Test\n', _linenum(0) + 'Test\n' ) ), ('if_true', ([_LINENUM_FLAG], '#:if 1 < 2\nTrue\n#:endif\nDone\n', _linenum(0) + _linenum(1) + 'True\n' + _linenum(3) + 'Done\n' ) ), ('if_false', ([_LINENUM_FLAG], '#:if 1 > 2\nTrue\n#:endif\nDone\n', _linenum(0) + _linenum(3) + 'Done\n' ) ), ('if_else_true', ([_LINENUM_FLAG], '#:if 1 < 2\nTrue\n#:else\nFalse\n#:endif\nDone\n', _linenum(0) + _linenum(1) + 'True\n' + _linenum(5) + 'Done\n' ) ), ('if_else_false', ([_LINENUM_FLAG], '#:if 1 > 2\nTrue\n#:else\nFalse\n#:endif\nDone\n', _linenum(0) + _linenum(3) + 'False\n' + _linenum(5) + 'Done\n' ) ), ('if_elif_true1', ([_LINENUM_FLAG], '#:if 1 == 1\nTrue1\n#:elif 1 == 2\nTrue2\n#:endif\nDone\n', _linenum(0) + _linenum(1) + 'True1\n' + _linenum(5) + 'Done\n' ) ), ('if_elif_true2', ([_LINENUM_FLAG], '#:if 2 == 1\nTrue1\n#:elif 2 == 2\nTrue2\n#:endif\nDone\n', _linenum(0) + _linenum(3) + 'True2\n' + _linenum(5) + 'Done\n' ) ), ('if_elif_false', ([_LINENUM_FLAG], '#:if 0 == 1\nTrue1\n#:elif 0 == 2\nTrue2\n#:endif\nDone\n', _linenum(0) + _linenum(5) + 'Done\n' ) ), ('if_elif_else_true1', ([_LINENUM_FLAG], '#:if 1 == 1\nTrue1\n#:elif 1 == 2\nTrue2\n' '#:else\nFalse\n#:endif\nDone\n', _linenum(0) + _linenum(1) + 'True1\n' + _linenum(7) + 'Done\n' ) ), ('if_elif_else_true2', ([_LINENUM_FLAG], '#:if 2 == 1\nTrue1\n#:elif 2 == 2\nTrue2\n' '#:else\nFalse\n#:endif\nDone\n', _linenum(0) + _linenum(3) + 'True2\n' + _linenum(7) + 'Done\n' ) ), ('if_elif_else_false', ([_LINENUM_FLAG], '#:if 0 == 1\nTrue1\n#:elif 0 == 2\nTrue2\n' '#:else\nFalse\n#:endif\nDone\n', _linenum(0) + _linenum(5) + 'False\n' + _linenum(7) + 'Done\n' ) ), ('inline_if_true', ([_LINENUM_FLAG], '#{if 1 < 2}#True#{endif}#Done\n', _linenum(0) + 'TrueDone\n' ) ), ('inline_if_false', ([_LINENUM_FLAG], '#{if 1 > 2}#True#{endif}#Done\n', _linenum(0) + 'Done\n' ) ), ('inline_if_else_true', ([_LINENUM_FLAG], '#{if 1 < 2}#True#{else}#False#{endif}#Done\n', _linenum(0) + 'TrueDone\n' ) ), ('inline_if_else_false', ([_LINENUM_FLAG], '#{if 1 > 2}#True#{else}#False#{endif}#Done\n', _linenum(0) + 'FalseDone\n' ) ), ('inline_if_elif_true1', ([_LINENUM_FLAG], '#{if 1 == 1}#True1#{elif 1 == 2}#True2#{endif}#Done\n', _linenum(0) + 'True1Done\n' ) ), ('inline_if_elif_true2', ([_LINENUM_FLAG], '#{if 2 == 1}#True1#{elif 2 == 2}#True2#{endif}#Done\n', _linenum(0) + 'True2Done\n' ) ), ('inline_if_elif_false', ([_LINENUM_FLAG], '#{if 0 == 1}#True1#{elif 0 == 2}#True2#{endif}#Done\n', _linenum(0) + 'Done\n' ) ), ('inline_if_elif_else_true1', ([_LINENUM_FLAG], '#{if 1 == 1}#True1#{elif 1 == 2}#True2#{else}#False#{endif}#Done\n', _linenum(0) + 'True1Done\n' ) ), ('inline_if_elif_else_true2', ([_LINENUM_FLAG], '#{if 2 == 1}#True1#{elif 2 == 2}#True2#{else}#False#{endif}#Done\n', _linenum(0) + 'True2Done\n' ) ), ('inline_if_elif_else_false', ([_LINENUM_FLAG], '#{if 0 == 1}#True1#{elif 0 == 2}#True2#{else}#False#{endif}#Done\n', _linenum(0) + 'FalseDone\n' ) ), ('linesub_oneline', ([_LINENUM_FLAG], 'A\n$: 1 + 1\nB\n', _linenum(0) + 'A\n2\nB\n' ) ), ('linesub_contlines', ([_LINENUM_FLAG, _defvar('TESTVAR', 1)], '$: TESTVAR & \n & + 1\nDone\n', _linenum(0) + '2\n' + _linenum(2) + 'Done\n' ) ), ('linesub_contlines2', ([_LINENUM_FLAG, _defvar('TESTVAR', 1)], '$: TEST& \n &VAR & \n & + 1\nDone\n', _linenum(0) + '2\n' + _linenum(3) + 'Done\n' ) ), ('exprsub_single_line', ([_LINENUM_FLAG, _defvar('TESTVAR', 1)], 'A${TESTVAR}$B${TESTVAR + 1}$C', _linenum(0) + 'A1B2C' ) ), ('exprsub_multi_line', ([_LINENUM_FLAG], '${"line1\\nline2"}$\nDone\n', _linenum(0) + 'line1\n' + _linenum(0) + 'line2\nDone\n' ) ), ('macrosubs', ([_LINENUM_FLAG], '#:def macro(var)\nMACRO|${var}$|\n#:enddef\n${macro(1)}$', _linenum(0) + _linenum(3) + 'MACRO|1|' ) ), ('recursive_macrosubs', ([_LINENUM_FLAG], '#:def macro(var)\nMACRO|${var}$|\n#:enddef\n${macro(macro(1))}$', _linenum(0) + _linenum(3) + 'MACRO|MACRO|1||' ) ), ('macrosubs_multiline', ([_LINENUM_FLAG], '#:def macro(c)\nMACRO1|${c}$|\nMACRO2|${c}$|\n#:enddef\n${macro(\'A\')}$' '\n', _linenum(0) + _linenum(4) + 'MACRO1|A|\n' + _linenum(4) + 'MACRO2|A|\n' ) ), ('recursive_macrosubs_multiline', ([_LINENUM_FLAG], '#:def f(c)\nLINE1|${c}$|\nLINE2|${c}$|\n#:enddef\n$: f(f("A"))\n', (_linenum(0) + _linenum(4) + 'LINE1|LINE1|A|\n' + _linenum(4) + 'LINE2|A||\n' + _linenum(4) + 'LINE2|LINE1|A|\n' + _linenum(4) + 'LINE2|A||\n') ) ), ('multiline_macrocall', ([_LINENUM_FLAG], '#:def macro(c)\nMACRO|${c}$|\n#:enddef\n$: mac& \n &ro(\'A\')\nDone\n', _linenum(0) + _linenum(3) + 'MACRO|A|\n' + _linenum(5) + 'Done\n' ) ), ('call_directive_2_args', ([_LINENUM_FLAG], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '#:call mymacro\nL1\nL2\n#:nextarg\nL3\n#:endcall\n', _linenum(0) + _linenum(3) + '|L1\n' + _linenum(3) + 'L2|L3|\n' + _linenum(9), ) ), ('for', ([_LINENUM_FLAG], '#:for i in (1, 2)\n${i}$\n#:endfor\nDone\n', (_linenum(0) + _linenum(1) + '1\n' + _linenum(1) + '2\n' + _linenum(3) + 'Done\n') ) ), ('inline_for', ([_LINENUM_FLAG], '#{for i in (1, 2)}#${i}$#{endfor}#Done\n', _linenum(0) + '12Done\n' ) ), ('set', ([_LINENUM_FLAG], '#:set x = 2\n$: x\n', _linenum(0) + _linenum(1) + '2\n', ) ), ('inline_set', ([_LINENUM_FLAG], '#{set x = 2}#${x}$Done\n', _linenum(0) + '2Done\n', ) ), ('comment_single', ([_LINENUM_FLAG], ' #! Comment here\nDone\n', _linenum(0) + _linenum(1) + 'Done\n' ) ), ('comment_multiple', ([_LINENUM_FLAG], ' #! Comment1\n#! Comment2\nDone\n', _linenum(0) + _linenum(2) + 'Done\n', ) ), ('mute', ([_LINENUM_FLAG], 'A\n#:mute\nB\n#:set VAR = 2\n#:endmute\nVAR=${VAR}$\n', _linenum(0) + 'A\n' + _linenum(5) + 'VAR=2\n' ) ), ('direct_call', ([_LINENUM_FLAG], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro( a < b )\n', _linenum(0) + _linenum(3) + '|a < b|\n', ) ), ('direct_call_contline', ([_LINENUM_FLAG], '#:def mymacro(val)\n|${val}$|\n#:enddef\n'\ '@:mymacro(a &\n &< b&\n &)\nDone\n', _linenum(0) + _linenum(3) + '|a < b|\n' + _linenum(6) + 'Done\n', ) ), ('assert_directive', ([_LINENUM_FLAG], '#:assert 1 < 2\nDone\n', _linenum(0) + _linenum(1) + 'Done\n', ) ), ('assert_directive_contline', ([_LINENUM_FLAG], '#:assert 1&\n& < 2\nDone\n', _linenum(0) + _linenum(2) + 'Done\n', ) ), ('smart_folding', ([_LINENUM_FLAG, _linelen(15), _indentation(4), _folding('smart')], ' ${3}$456 89 123456 8\nDone\n', _linenum(0) + ' 3456 89&\n' + _linenum(0) + ' & 123456&\n' + _linenum(0) + ' & 8\n' + 'Done\n' ) ), ('smart_folding_nocontlines', ([_LINENUM_FLAG, _linenumbering('nocontlines'), _linelen(15), _indentation(4), _folding('smart')], ' ${3}$456 89 123456 8\nDone\n', _linenum(0) + ' 3456 89&\n' + ' & 123456&\n' \ + ' & 8\n' + _linenum(1) + 'Done\n' ) ), ] # Tests with include files # # Each test consists of a tuple containing the test name and a tuple with the # arguments of the get_test_output_method() routine. # INCLUDE_TESTS = [ ('explicit_include', ([], '#:include "include/fypp1.inc"\n', 'INCL1\nINCL5\n' ) ), ('search_include', ([_incdir('include')], '#:include "fypp1.inc"\n', 'INCL1\nINCL5\n' ) ), ('nested_include_in_incpath', ([_incdir('include')], '#:include "subfolder/include_fypp1.inc"\n', 'INCL1\nINCL5\n' ) ), ('nested_include_in_folder_of_incfile', ([_incdir('include')], '#:include "subfolder/include_fypp2.inc"\n', 'FYPP2\n' ) ), ('search_include_linenum', ([_LINENUM_FLAG, _incdir('include')], '#:include "fypp1.inc"\n$: incmacro(1)\n', (_linenum(0) + _linenum(0, 'include/fypp1.inc', flag=_NEW_FILE) + 'INCL1\n' + _linenum(4, 'include/fypp1.inc') + 'INCL5\n' + _linenum(1, flag=_RETURN_TO_FILE) + 'INCMACRO(1)\n') ) ), ('nested_include_in_incpath_linenum', ([_LINENUM_FLAG, _incdir('include')], '#:include "subfolder/include_fypp1.inc"\n', (_linenum(0) + _linenum(0, 'include/subfolder/include_fypp1.inc', flag=_NEW_FILE) + _linenum(0, 'include/fypp1.inc', flag=_NEW_FILE) + 'INCL1\n' + _linenum(4, 'include/fypp1.inc') + 'INCL5\n' + _linenum(1, 'include/subfolder/include_fypp1.inc', flag=_RETURN_TO_FILE) + _linenum(1, flag=_RETURN_TO_FILE)) ) ), ('nested_include_in_folder_of_incfile2', ([_LINENUM_FLAG, _incdir('include')], '#:include "subfolder/include_fypp2.inc"\n', (_linenum(0) + _linenum(0, 'include/subfolder/include_fypp2.inc', flag=_NEW_FILE) + _linenum(0, 'include/subfolder/fypp2.inc', flag=_NEW_FILE) + 'FYPP2\n' + _linenum(1, 'include/subfolder/include_fypp2.inc', flag=_RETURN_TO_FILE) + _linenum(1, flag=_RETURN_TO_FILE)) ) ), ('muted_include', ([_incdir('include')], 'START\n#:mute\n#:include \'fypp1.inc\'\n#:endmute\nDONE\n', 'START\nDONE\n' ) ), ('muted_include_linenum', ([_LINENUM_FLAG, _incdir('include')], 'START\n#:mute\n#:include \'fypp1.inc\'\n#:endmute\nDONE\n', _linenum(0) + 'START\n' + _linenum(4) + 'DONE\n' ) ), ] # Tests triggering exceptions # # Each test consists of a tuple containing the test name and a tuple with the # arguments of the get_test_exception_method() routine. # EXCEPTION_TESTS = [ # # Parser errors # ('invalid_directive', ([], '#:invalid\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_macrodef', ([], '#:def alma[x]\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_for_decl', ([], '#:for i = 1, 2\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_include', ([], '#:include \n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('inline_include', ([], '#{include "test.h"}#\n', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('wrong_include_file', ([], '#:include "testfkjsdlfkjslf.h"\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_else', ([], '#:if 1 > 2\nA\n#:else True\nB\n#:endif\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('invalid_endif', ([], '#:if 1 > 2\nA\n#:else\nB\n#:endif INV\n', [(fypp.FyppFatalError, fypp.STRING, (4, 5))] ) ), ('invalid_endfor', ([], '#:for i in range(5)\n${i}$\n#:endfor INV\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('invalid_variable_assign', ([], '#:set A=\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_mute', ([], '#:mute TEST\n#:endmute\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_endmute', ([], '#:mute\n#:endmute INVALID\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('inline_mute', ([], '#{mute}#test#{endmute}#\n', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('inline_endmute', ([], '#:mute\ntest#{endmute}#\n', [(fypp.FyppFatalError, fypp.STRING, (1, 1))] ) ), ('setvar_with_equal', ([], '#:setvar x = 2\n$: x\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('inline_set_without_equal', ([], '#{set x 2}#${x}$Done\n', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('missing_del_name', ([], '#:del\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_del_name', ([], '#:del [a, b]\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('inline_def', ([], '#{def macro()}#TEST#{enddef}#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('invalid_direct_call_expr', ([], '#:def macro()\n#:enddef\n@:macro{}\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('invalid_direct_call_expr_inline', ([], '#:def macro()\n#:enddef\n@{macro{}}@\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('invalid_direct_call_expr2', ([], '#:def macro()\n#:enddef\n@:macro(\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('invalid_direct_call_expr2_inline', ([], '#:def macro()\n#:enddef\n@{macro(}@\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('direct_call_non_eval_dir', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@:mymacro(L1 #{if True}#2, 2#{endif}#)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 3))] ) ), ('direct_call_non_eval_dir_inline', ([], '#:def mymacro(val1, val2)\n|${val1}$|${val2}$|\n#:enddef\n'\ '@{mymacro(L1 #{if True}#2, 2#{endif}#)}@', [(fypp.FyppFatalError, fypp.STRING, (3, 3))] ) ), ('direct_call_unclosed quote', ([], '#:def mymacro(arg1)\n|${arg1}$|\n#:enddef\n'\ '@:mymacro("something)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, None, None)] ) ), ('direct_call_unclosed bracket', ([], '#:def mymacro(arg1)\n|${arg1}$|\n#:enddef\n'\ '@:mymacro({something)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, None, None)] ) ), ('direct_call_unbalanced bracket', ([], '#:def mymacro(arg1)\n|${arg1}$|\n#:enddef\n'\ '@:mymacro({(})\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, None, None)] ) ), ('missing_line_dir_content', ([], '#:\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('missing_line_dir_content2', ([], '#: \n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('missing_inline_dir_content', ([], '#{}#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('missing_inline_dir_content2', ([], '#{ }#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('set_setvar', ([], '#:setvar x 2\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('inline_setvar', ([], '#{setvar x 2}#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), # # Builder errors # ('line_if_inline_endif', ([], '#:if 1 < 2\nTrue\n#{endif}#\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('inline_if_line_endif', ([], '#{if 1 < 2}#True\n#:endif\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('line_if_inline_elif', ([], '#:if 1 < 2\nTrue\n#{elif 2 > 3}#\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('inline_if_line_elif', ([], '#{if 1 < 2}#True\n#:elif 2 > 3\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('line_if_inline_else', ([], '#:if 1 < 2\nTrue\n#{else}#\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('inline_if_line_else', ([], '#{if 1 < 2}#True\n#:else\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('loose_else', ([], 'A\n#:else\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('loose_inline_else', ([], 'A\n#{else}#\n', [(fypp.FyppFatalError, fypp.STRING, (1, 1))] ) ), ('loose_elif', ([], 'A\n#:elif 1 > 2\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('loose_inline_elif', ([], 'A\n#{elif 1 > 2}#\n', [(fypp.FyppFatalError, fypp.STRING, (1, 1))] ) ), ('loose_endif', ([], 'A\n#:endif\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('loose_inline_endif', ([], 'A\n#{endif}#\n', [(fypp.FyppFatalError, fypp.STRING, (1, 1))] ) ), ('mismatching_else', ([], '#:if 1 < 2\n#:for i in range(3)\n#:else\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('mismatching_elif', ([], '#:if 1 < 2\n#:for i in range(3)\n#:elif 1 > 2\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('mismatching_endif', ([], '#:if 1 < 2\n#:for i in range(3)\n#:endif\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('line_def_inline_enddef', ([], '#:def alma(x)\n#{enddef}#\n', [(fypp.FyppFatalError, fypp.STRING, (1, 1))] ) ), ('loose_enddef', ([], '#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('loose_inline_enddef', ([], '#{enddef}#\n', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('mismatching_enddef', ([], '#:def test(x)\n#{if 1 < 2}#\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('enddef_name_mismatch', ([], '#:def macro(var)\nMACRO|${var}$|\n#:enddef nonsense\n${macro(1)}$', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('endcall_name_mismatch', ([], '#:def macro(var)\nMACRO|${var}$|\n#:enddef\n'\ '#:call macro\n1\n#:endcall nonsense\n', [(fypp.FyppFatalError, fypp.STRING, (5, 6))] ) ), ('inline_endcall_name_mismatch', ([], '#:def macro(var)\nMACRO|${var}$|\n#:enddef\n'\ '#{call macro}#1#{endcall nonsense}#', [(fypp.FyppFatalError, fypp.STRING, (3, 3))] ) ), ('line_for_inline_endfor', ([], '#:for i in range(3)\nA\n#{endfor}#\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('inline_for_line_endfor', ([], '#{for i in range(3)}#Empty\n#:endfor\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2))] ) ), ('loose_endfor', ([], '#:endfor\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('loose_inline_endfor', ([], '#{endfor}#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('mismatching_endfor', ([], '#:for i in range(3)\n#{if 1 < 2}#\n#:endfor\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('loose_endmute', ([], '#:endmute\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('mismatching_endmute', ([], '#:mute\n#{if 1 < 2}#\n#:endmute\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('unclosed_directive', ([], '#:if 1 > 2\nA\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('missing_space_after_directive', ([], '#:if(1 > 2)\nA\n#:endif', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('missing_space_after_inline_directive', ([], '#{if(1 > 2)}#A#{endif}#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('mixing_block_and_endcall', ([], '#:def test(x)\n#:enddef\n#:block test\n1\n#:endcall\n', [(fypp.FyppFatalError, fypp.STRING, (4, 5))] ) ), ('mixing_call_and_endblock', ([], '#:def test(x)\n#:enddef\n#:call test\n1\n#:endblock\n', [(fypp.FyppFatalError, fypp.STRING, (4, 5))] ) ), ('mixing_call_and_contains', ([], '#:def test(x,y)\n#:enddef\n#:call test\n1\n#:contains\n2\n#:endcall\n', [(fypp.FyppFatalError, fypp.STRING, (4, 5))] ) ), ('mixing_block_and_nextarg', ([], '#:def test(x,y)\n#:enddef\n#:block test\n1\n#:nextarg\n2\n#:endblock\n', [(fypp.FyppFatalError, fypp.STRING, (4, 5))] ) ), # # Renderer errors # ('invalid_expression', ([], '${i}$', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('invalid_variable', ([], '#:set i 1.2.3\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_condition', ([], '#{if i >>> 3}##{endif}#', [(fypp.FyppFatalError, fypp.STRING, (0, 0))] ) ), ('invalid_iterator', ([], '#:for i in 1.2.3\nDummy\n#:endfor\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_macro_argument_expression', ([], '#:def alma(x))\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('tuple_macro_argument', ([], '#:def alma((x, y))\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))], ), ), ('repeated_keyword_argument', ([], '#:def mymacro(A, B)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '$:mymacro(A=1, A=2, B=3)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4))] ) ), ('pos_arg_after_keyword_arg', ([], '#:def mymacro(A, B)\nA=${A}$,B=${B}$\n#:enddef mymacro\n'\ '$:mymacro(B=4, 2)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4))] ) ), ('macrodef_pos_arg_after_keyword_arg', ([], '#:def mymacro(A, B=2, C)\nA=${A}$,B=${B},C=${C}$$\n#:enddef mymacro\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macrodef_pos_arg_after_var_arg', ([], '#:def mymacro(A, *B, C)\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ), ), ('macrodef_pos_arg_after_var_kwarg', ([], '#:def mymacro(A, **B, C)\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_macro_prefix', ([], '#:def __test(x)\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('reserved_macro_name', ([], '#:def defined(x)\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('macro_double_defined_arg', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${vararg}$|\n#:enddef\n'\ '$:macro(1, 2, x=1)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macro_invalid_argument_name', ([], '#:def macro(x, __y, *vararg)\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macro_invalid_varargument_name', ([], '#:def macro(x, y, *__vararg)\n#:enddef\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_variable_prefix', ([], '#:set __test = 2\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('reserved_variable_name', ([], '#:set _LINE_ = 2\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('macro_call_more_args', ([], '#:def test(x)\n${x}$\n#:enddef\n$: test(\'A\', 1)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macro_call_less_args', ([], '#:def test(x)\n${x}$\n#:enddef\n$: test()\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macro_invalid_keyword_arguments', ([], '#:def macro(x, y)\n|${x}$${y}$|\n#:enddef\n'\ '$:macro(1, 2, z=3)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macro_vararg_invalid_keyword_arguments', ([], '#:def macro(x, y, *vararg)\n|${x}$${y}$${z}$${vararg}$|\n#:enddef\n'\ '$:macro(1, 2, z=3)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('macro_kwarg_invalid_posarg', ([], '#:def macro(x, y, **varkw)\n|${x}$${y}$${varkw}$|\n#:enddef\n'\ '$:macro(1, 2, 3, z=3)\n', [(fypp.FyppFatalError, fypp.STRING, (3, 4)), (fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('short_line_length', ([_linelen(4)], '', [(fypp.FyppFatalError, None, None)] ) ), ('failing_macro_in_include', ([], '#:include "include/failingmacro.inc"\n$:failingmacro()\n', [(fypp.FyppFatalError, fypp.STRING, (1, 2)), (fypp.FyppFatalError, 'include/failingmacro.inc', (2, 3))] ) ), ('incompatible_tuple_assignment1', ([], '#:set a,b,c = (1, 2)\n${a}$${b}$${c}$\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('incompatible_tuple_assignment2', ([], '#:set a,b,c = (1, 2, 3, 4)\n${a}$${b}$${c}$\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('invalid_lhs_tuple1', ([], '#:set (a, b = (1, 2)\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('invalid_lhs_tuple2', ([], '#:set a, b) = (1, 2)\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('invalid_del_tuple1', ([], '#:del (a, b\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('invalid_del_tuple2', ([], '#:del a, b)\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('del_nonexisting_variable', ([], '#:del X\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ('local_macro_visibility', ([], '#:set echo = lambda s: s\n'\ '#:call echo\n' '#:def mymacro()\nX\n#:enddef\n'\ '#:endcall\n$:mymacro()\n', [(fypp.FyppFatalError, fypp.STRING, (6, 7))] ) ), # # Command line errors # ('def_error', (['-DVAR=1.2.2'], '', [(fypp.FyppFatalError, None, None)] ) ), ('missing_module', (['-mWhateverDummyKJFDKf'], '', [(fypp.FyppFatalError, None, None)] ) ), # # User requested stop # ('userstop', ([], '#:set A = 12\n#:if A > 10\n#:stop "Wrong A: {0}".format(A)\n#:endif\n', [(fypp.FyppStopRequest, fypp.STRING, (2, 3))] ) ), ('invalid_userstop_expr', ([], '#:set A = 12\n#:if A > 10\n#:stop "Wrong A: {0}".format(BA)\n#:endif\n', [(fypp.FyppFatalError, fypp.STRING, (2, 3))] ) ), ('invalid_inline_stop', ([], '#:set A = 1\n#:if A > 10\n#{stop "Wrong A: {0}".format(BA)}#\n#:endif\n', [(fypp.FyppFatalError, fypp.STRING, (2, 2))] ) ), ('assert', ([], '#:set A = 12\n#:assert A < 10\n', [(fypp.FyppStopRequest, fypp.STRING, (1, 2))] ) ), ('invalid_assert_expr', ([], '#:assert A < 10\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1))] ) ), ('invalid_inline_assert', ([], '#:set A = 12\n#{assert A < 10}#\n', [(fypp.FyppFatalError, fypp.STRING, (1, 1))] ) ), ('global_existing_in_local_scope', ([], '#:def macro()\n#:set A = 12\n#:global A\n#:enddef\n$:macro()\n', [(fypp.FyppFatalError, fypp.STRING, (4, 5)), (fypp.FyppFatalError, fypp.STRING, (2, 3)), (fypp.FyppFatalError, None, None)] ) ), ('setvar_func_odd_arguments', ([], '$:setvar("i", 1, "j")\n', [(fypp.FyppFatalError, fypp.STRING, (0, 1)), (fypp.FyppFatalError, None, None)] ) ), ] # Tests with module imports # # Each test consists of a tuple containing the test name and a tuple with the # arguments of the get_test_output_method() routine. # # NOTE: imports are global in Python, so all instances of Fypp following after # the tests below will see the imported modules Therefore, this tests should be # executed as last to minimize unwanted interactions between unit tests. Also, # no tests before these should import any modules. # IMPORT_TESTS = [ ('import_module', ([_importmodule('math')], '$:int(math.sqrt(4))\n', '2\n' ) ), ('import_module_current_dir', ([_importmodule('inimod')], '${inimod.get_version()}$', '1' ) ), ('import_module_modified_lookupdir', ([_moddir('include'), _importmodule('inimod2')], '${inimod2.get_version()}$', '2' ) ), ('import_subpackage', ([_importmodule('os.path')], '${os.path.isabs("a")}$', 'False' ) ), ] def _get_test_output_method(args, inp, out): '''Returns a test method for checking correctness of Fypp output. Args: args (list of str): Command-line arguments to pass to Fypp. inp (str): Input with Fypp directives. out (str): Expected output. Returns: method: Method to test equality of output with result delivered by Fypp. ''' def test_output(self): '''Tests whether Fypp result matches expected output.''' optparser = fypp.get_option_parser() options, leftover = optparser.parse_args(args) self.assertEqual(len(leftover), 0) tool = fypp.Fypp(options) result = tool.process_text(inp) self.assertEqual(out, result) return test_output def _get_test_exception_method(args, inp, exceptions): '''Returns a test method for checking correctness of thrown exception. Args: args (list of str): Command-line arguments to pass to Fypp. inp (str): Input with Fypp directives. exceptions (list of tuples): Each tuple contains an exception, a file name and a span (tuple or int). The tuples should be in reverse order (latest raised exception first). Returns: method: Method to test, whether Fypp throws the correct exception. ''' def test_exception(self): '''Tests whether Fypp throws the correct exception.''' optparser = fypp.get_option_parser() options, leftover = optparser.parse_args(args) self.assertEqual(len(leftover), 0) try: tool = fypp.Fypp(options) _ = tool.process_text(inp) except Exception as e: raised = e else: self.fail('No exception was raised') for exc, fname, span in exceptions: self.assertTrue(isinstance(raised, exc)) if fname is None: self.assertTrue(raised.fname is None) else: self.assertEqual(fname, raised.fname) if span is None: self.assertTrue(raised.span is None) else: self.assertEqual(span, raised.span) raised = raised.__cause__ self.assertTrue(not isinstance(raised, fypp.FyppError)) return test_exception def _test_needed(flag): return True class _TestContainer(unittest.TestCase): '''General test container class.''' @classmethod def add_test_methods(cls, tests, methodfactory): '''Adds tests to a test case. Args: tests (list of tuples): Tests to attach. testcase (TestCase): Class which the tests should be attached to. methodfactory (function): Functions which turns the tuples in tests into methods, which can be then attached to the test case. ''' already_added = set() for itest, test in enumerate(tests): name = test[0] if name in already_added: msg = "multiple occurrence of test name '{0}'".format(name) raise ValueError(msg) already_added.add(name) testargs = test[1] methodname = 'test_' + name if len(test) < 3: addtest = True else: addtest = _test_needed(test[2]) if addtest: setattr(cls, methodname, methodfactory(*testargs)) class SimpleTest(_TestContainer): pass SimpleTest.add_test_methods(SIMPLE_TESTS, _get_test_output_method) class LineNumberingTest(_TestContainer): pass LineNumberingTest.add_test_methods(LINENUM_TESTS, _get_test_output_method) class IncludeTest(_TestContainer): pass IncludeTest.add_test_methods(INCLUDE_TESTS, _get_test_output_method) class ExceptionTest(_TestContainer): pass ExceptionTest.add_test_methods(EXCEPTION_TESTS, _get_test_exception_method) class ImportTest(_TestContainer): pass ImportTest.add_test_methods(IMPORT_TESTS, _get_test_output_method) if __name__ == '__main__': unittest.main() fypp-3.1/tools/000077500000000000000000000000001403725613300134775ustar00rootroot00000000000000fypp-3.1/tools/waf/000077500000000000000000000000001403725613300142545ustar00rootroot00000000000000fypp-3.1/tools/waf/fypp_fortran.py000066400000000000000000000030731403725613300173420ustar00rootroot00000000000000#!/usr/bin/env python3 # encoding: utf-8 # Bálint Aradi, 2016-2021 '''Uses Fypp as Fortran preprocessor (.fpp -> .f90). Use this one (instead of fypp_preprocessor) if you want to preprocess Fortran sources with Fypp. You typically trigger the preprocessing via the 'fypp' feature:: def options(opt): opt.load('compiler_c') opt.load('compiler_fc') opt.load('fypp_fortran') def configure(conf): conf.load('compiler_c') conf.load('compiler_fc') conf.load('fypp_fortran') def build(bld): sources = bld.path.ant_glob('*.fpp') bld( features='fypp fc fcprogram', source=sources, target='myprog' ) Please check the documentation in the fypp_preprocessor module for the description of the uselib variables which may be passed to the task generator. ''' from waflib import TaskGen import fypp_preprocessor ################################################################################ # Configure ################################################################################ def configure(conf): fypp_preprocessor.configure(conf) ################################################################################ # Build ################################################################################ class fypp_fortran(fypp_preprocessor.fypp_preprocessor): ext_in = [ '.fpp' ] ext_out = [ '.f90' ] @TaskGen.extension('.fpp') def fypp_preprocess_fpp(self, node): 'Preprocess the .fpp files with Fypp.' f90node = node.change_ext('.f90') self.create_task('fypp_fortran', node, [ f90node ]) if 'fc' in self.features: self.source.append(f90node) fypp-3.1/tools/waf/fypp_preprocessor.py000066400000000000000000000144521403725613300204200ustar00rootroot00000000000000#!/usr/bin/env python3 # encoding: utf-8 # Bálint Aradi, 2016-2021 '''General module for using Fypp as preprocessor. This module implements the general framework for the Fypp preprocessor, but does not bind it to any task generator. If you want to use it to preprocessor Fortran source files (.fpp -> .f90), use the fypp_fortran module instead. Otherwise, you can generate your own binding as ususal:: def build(bld): bld(features='fypp', source=['test.fypp']) from waflib import TaskGen @TaskGen.extension('.fypp') def process(self, node): tsk = self.create_task('fypp_preprocessor', [node], node.change_ext('.out')) The preprocessor understands the following uselib attributes: * ``includes``: Directory/directories to search for include files * ``modules``: Python module(s) to import before preprocessing starts * ``defines``: Definition(s) to apply before preprocessing starts * ``inifiles``: Python file(s) to execute before preprosessing starts The example below demonstrates this:: def build(bld): bld(features='fypp', source=['trash.fypp'], includes='include', modules=['myfypp1', 'myfypp2'], defines='TEST=1 QUIET', inifiles='fyppini.py') ''' import re import os.path from waflib import Configure, Logs, Task, TaskGen, Tools, Errors try: import fypp except ImportError: fypp = None Tools.ccroot.USELIB_VARS['fypp'] = set([ 'DEFINES', 'INCLUDES', 'MODULES', 'INIFILES' ]) FYPP_INCPATH_ST = '-I%s' FYPP_DEFINES_ST = '-D%s' FYPP_LINENUM_FLAG = '-n' FYPP_MODULES_ST = '-m%s' FYPP_INIFILES_ST = '-i%s' class FyppPreprocError(Errors.WafError): pass ################################################################################ # Configure ################################################################################ def configure(conf): fypp_check(conf) fypp_add_user_flags(conf) @Configure.conf def fypp_add_user_flags(conf): '''Import user settings for Fypp.''' conf.add_os_flags('FYPP_FLAGS', dup=False) @Configure.conf def fypp_check(conf): '''Check for Fypp.''' conf.start_msg('Checking for fypp module') if fypp is None: conf.fatal('Python module \'fypp\' could not be imported.') version = fypp.VERSION version_regexp = re.compile(r'^(?P\d+)\.(?P\d+)'\ '(?:\.(?P\d+))?$') match = version_regexp.search(version) if not match: conf.fatal('cannot parse fypp version string') version = (match.group('major'), match.group('minor')) conf.env['FYPP_VERSION'] = version conf.end_msg('found (version %s.%s)' % version) ################################################################################ # Build ################################################################################ class fypp_preprocessor(Task.Task): def keyword(self): return 'Preprocessing' def run(self): argparser = fypp.get_option_parser() args = [FYPP_LINENUM_FLAG] args += self.env.FYPP_FLAGS args += [FYPP_DEFINES_ST % ss for ss in self.env['DEFINES']] args += [FYPP_INCPATH_ST % ss for ss in self.env['INCLUDES']] args += [FYPP_INIFILES_ST % ss for ss in self.env['INIFILES']] args += [FYPP_MODULES_ST % ss for ss in self.env['MODULES']] opts, leftover = argparser.parse_args(args) infile = self.inputs[0].abspath() outfile = self.outputs[0].abspath() if Logs.verbose: Logs.debug('runner: fypp.Fypp %r %r %r' % (args, infile, outfile)) tool = fypp.Fypp(opts) try: tool.process_file(infile, outfile) except fypp.FyppError as err: msg = ("%s [%s:%d]" % (err.msg, err.fname, err.span[0] + 1)) raise FyppPreprocError(msg) return 0 def scan(self): parser = FyppIncludeParser(self.generator.includes_nodes) nodes, names = parser.parse(self.inputs[0]) if Logs.verbose: Logs.debug('deps: deps for %r: %r; unresolved: %r' % (self.inputs, nodes, names)) return (nodes, names) TaskGen.feature('fypp')(Tools.ccroot.propagate_uselib_vars) TaskGen.feature('fypp')(Tools.ccroot.apply_incpaths) ################################################################################ # Helper routines ################################################################################ class FyppIncludeParser(object): '''Parser for include directives in files preprocessed by Fypp. It can not handle conditional includes. ''' # Include file pattern, opening and closing quoute must be replaced inside. INCLUDE_PATTERN = re.compile(r'^\s*#:include\s*(["\'])(?P.+?)\1', re.MULTILINE) def __init__(self, incpaths): '''Initializes the parser. :param quotes: Tuple containing the opening and closing quote sign. :type quotes: tuple ''' # Nodes still to be processed self._waiting = [] # Files we have already processed self._processed = set() # List of dependent nodes self._dependencies = [] # List of unresolved dependencies self._unresolved = set() # Paths to consider when checking for includes self._incpaths = incpaths def parse(self, node): '''Parser the includes in a given node. :return: Tuple with two elements: list of dependent nodes and list of unresolved depencies. ''' self._waiting = [ node, ] # self._waiting is eventually extended during _process() -> iterate while self._waiting: curnode = self._waiting.pop(0) self._process(curnode) return (self._dependencies, list(self._unresolved)) def _process(self, node): incfiles = self._get_include_files(node) for incfile in incfiles: if incfile in self._processed: continue self._processed.add(incfile) incnode = self._find_include_node(node, incfile) if incnode: self._dependencies.append(incnode) self._waiting.append(incnode) else: self._unresolved.add(incfile) def _get_include_files(self, node): txt = node.read() matches = self.INCLUDE_PATTERN.finditer(txt) incs = [ match.group('incfile') for match in matches ] return incs def _find_include_node(self, node, filename): for incpath in self._incpaths: incnode = incpath.find_resource(filename) if incnode: break else: incnode = node.parent.find_resource(filename) return incnode fypp-3.1/tox.ini000066400000000000000000000002711403725613300136520ustar00rootroot00000000000000[tox] envlist = py34, py35, py36, py37, py38, py39 [testenv] skip_missing_interpreters = true setenv = PYTHONPATH = {toxinidir}/src changedir=test commands=python test_fypp.py fypp-3.1/utils/000077500000000000000000000000001403725613300134775ustar00rootroot00000000000000fypp-3.1/utils/bump-version.py000077500000000000000000000040241403725613300165020ustar00rootroot00000000000000#!/usr/bin/env python3 import sys import re import os VERSION_PATTERN = r'\d+\.\d+(?:\.\d+)?(?:-\w+)?' FILES_PATTERNS = [ ('bin/fypp', r'^VERSION\s*=\s*([\'"]){}\1'.format(VERSION_PATTERN), "VERSION = '{version}'"), ('docs/fypp.rst', r'Fypp Version[ ]*{}.'.format(VERSION_PATTERN), 'Fypp Version {shortversion}.'), ('setup.py', r'version\s*=\s*([\'"]){}\1'.format(VERSION_PATTERN), "version='{version}'"), ('docs/conf.py', r'version\s*=\s*([\'"]){}\1'.format(VERSION_PATTERN), "version = '{shortversion}'"), ('docs/conf.py', r'release\s*=\s*([\'"]){}\1'.format(VERSION_PATTERN), "release = '{version}'"), ] if len(sys.argv) < 2: print("Missing version string") sys.exit(1) version = sys.argv[1] shortversion = '.'.join(version.split('.')[0:2]) match = re.match(VERSION_PATTERN, version) if match is None: print("Invalid version string") sys.exit(1) rootdir = os.path.join(os.path.dirname(sys.argv[0]), '..') for fname, regexp, repl in FILES_PATTERNS: fname = os.path.join(rootdir, fname) print("Replacments in '{}': ".format(fname), end='') fp = open(fname, 'r') txt = fp.read() fp.close() replacement = repl.format(version=version, shortversion=shortversion) newtxt, nsub = re.subn(regexp, replacement, txt, flags=re.MULTILINE) print(nsub) fp = open(fname, 'w') fp.write(newtxt) fp.close() # Replace version number in Change Log and adapt decoration below fname = os.path.join(rootdir, 'CHANGELOG.rst') print("Replacments in '{}': ".format(fname), end='') fp = open(fname, 'r') txt = fp.read() fp.close() decoration = '=' * len(version) newtxt, nsub = re.subn( '^Unreleased\s*\n=+', version + '\n' + decoration, txt, count=1, flags=re.MULTILINE) print(nsub) fp = open(fname, 'w') fp.write(newtxt) fp.close()