stringtemplate3-3.1/0000755000175000001440000000000010756132767013400 5ustar pinkusersstringtemplate3-3.1/PKG-INFO0000644000175000001440000000221010756132767014470 0ustar pinkusersMetadata-Version: 1.0 Name: stringtemplate3 Version: 3.1 Summary: A powerful template engine with strict model-view separation Home-page: http://www.stringtemplate.org/ Author: Benjamin Niemann Author-email: pink@odahoda.de License: BSD Description: ST (StringTemplate) is a template engine for generating source code, web pages, emails, or any other formatted text output. ST is particularly good at multi-targeted code generators, multiple site skins, and internationalization/localization. It evolved over years of effort developing jGuru.com. ST also generates this website and powers the ANTLR v3 code generator. Its distinguishing characteristic is that it strictly enforces model-view separation unlike other engines. Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Text Processing stringtemplate3-3.1/README.txt0000444000175000001440000000322510756132622015064 0ustar pinkusersStringTemplate 3.1 (Python) February 17, 2008 AUTHORS ======= Design & Java implementation: Terence Parr, parrt at cs usfca edu ANTLR project lead and supreme dictator for life University of San Francisco Python port: Marq Kole Python implementation up to V2.2 Copyright 2003-2006 Benjamin Niemann Updated Python implementation from V2.2 to V3. Copyright 2007 ABSTRACT ======== ST (StringTemplate) is a template engine (with ports for C# and Python) for generating source code, web pages, emails, or any other formatted text output. ST is particularly good at multi-targeted code generators, multiple site skins, and internationalization/localization. It evolved over years of effort developing jGuru.com. ST also generates this website and powers the ANTLR v3 code generator. Its distinguishing characteristic is that it strictly enforces model-view separation unlike other engines. DOCUMENTATION ============= The main website is: http://www.stringtemplate.org The documentation is in the wiki: http://www.antlr.org/wiki/display/ST/StringTemplate+3.1+Documentation Here are the 3.1 release notes: http://www.antlr.org/wiki/display/ST/StringTemplate+3.1+Release+Notes LICENSE ======= Per the license in LICENSE.txt, this software is not guaranteed to work and might even destroy all life on this planet. See the CHANGES.txt file. INSTALLATION ============ $ python setup.py install or just copy the stringtemplate3 directory to a location of your choice - it's a pure Python package. StringTemplate has a dependency on ANTLR V2. Download and install it from . stringtemplate3-3.1/stringtemplate3/0000755000175000001440000000000010756132767016525 5ustar pinkusersstringtemplate3-3.1/stringtemplate3/interfaces.py0000444000175000001440000001423310756132623021212 0ustar pinkusers # [The "BSD licence"] # Copyright (c) 2003-2006 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # from StringIO import StringIO from stringtemplate3.language import ( InterfaceLexer, InterfaceParser ) from stringtemplate3.utils import deprecated from stringtemplate3.errors import DEFAULT_ERROR_LISTENER class TemplateDefinition(object): """All the info we need to track for a template defined in an interface""" def __init__(self, name, formalArgs, optional): self.name = name self.formalArgs = formalArgs self.optional = optional class StringTemplateGroupInterface(object): """ A group interface is like a group without the template implementations; there are just template names/argument-lists like this: interface foo; class(name,fields); method(name,args,body); """ def __init__(self, file, errors=None, superInterface=None): """Create an interface from the input stream""" ## What is the group name self.name = None ## Maps template name to TemplateDefinition object self.templates = {} ## Are we derived from another group? Templates not found in this # group will be searched for in the superGroup recursively. self.superInterface = superInterface ## Where to report errors. All string templates in this group # use this error handler by default. if errors is not None: self.listener = errors else: self.listener = DEFAULT_ERROR_LISTENER self.parseInterface(file) @deprecated def getSuperInterface(self): return self.superInterface @deprecated def setSuperInterface(self, superInterface): self.superInterface = superInterface def parseInterface(self, r): try: lexer = InterfaceLexer.Lexer(r) parser = InterfaceParser.Parser(lexer) parser.groupInterface(self) except RuntimeError, exc: #FIXME: Exception name = self.name or "" self.error("problem parsing group "+name+": "+str(exc), exc) def defineTemplate(self, name, formalArgs, optional): d = TemplateDefinition(name, formalArgs, optional) self.templates[d.name] = d def getMissingTemplates(self, group): """ Return a list of all template names missing from group that are defined in this interface. Return null if all is well. """ missing = [] templates = self.templates.items() templates.sort() for name, d in templates: if not d.optional and not group.isDefined(d.name): missing.append(d.name) return missing or None def getMismatchedTemplates(self, group): """ Return a list of all template sigs that are present in the group, but that have wrong formal argument lists. Return null if all is well. """ mismatched = [] templates = self.templates.items() templates.sort() for name, d in templates: if group.isDefined(d.name): defST = group.getTemplateDefinition(d.name) formalArgs = defST.formalArguments ack = False if ( (d.formalArgs is not None and formalArgs is None) or (d.formalArgs is None and formalArgs is not None) or (len(d.formalArgs) != len(formalArgs)) ): ack = True if not ack: for argName in formalArgs.keys(): if d.formalArgs.get(argName, None) == None: ack = True break if ack: mismatched.append(self.getTemplateSignature(d)) return mismatched or None @deprecated def getName(self): return self.name @deprecated def setName(self, name): self.name = name def error(self, msg, exc=None): if self.listener is not None: self.listener.error(msg, exc) def toString(self): buf = StringIO() buf.write("interface ") buf.write(self.name) buf.write(";\n") templates = self.templates.items() templates.sort() for name, d in templates: buf.write( self.getTemplateSignature(d) ) buf.write(";\n") return buf.getvalue() __str__ = toString def getTemplateSignature(self, d): buf = StringIO() if d.optional: buf.write("optional ") buf.write(d.name) if d.formalArgs is not None: buf.write('(') args = d.formalArgs.keys() args.sort() buf.write(", ".join(args)) buf.write(')') else: buf.write("()") return buf.getvalue() stringtemplate3-3.1/stringtemplate3/language/0000755000175000001440000000000010756132767020310 5ustar pinkusersstringtemplate3-3.1/stringtemplate3/language/ASTExpr.py0000444000175000001440000007761210756132623022152 0ustar pinkusers from StringIO import StringIO import antlr from stringtemplate3.utils import deprecated from stringtemplate3.language.Expr import Expr from stringtemplate3.language import ActionEvaluator from stringtemplate3.language.StringTemplateAST import StringTemplateAST from stringtemplate3.language.CatIterator import CatList from stringtemplate3.language.FormalArgument import UNKNOWN_ARGS import stringtemplate3 class IllegalStateException(Exception): def __init__(self, *args): Exception.__init__(self, *args) def isiterable(o): if isinstance(o, (basestring, stringtemplate3.StringTemplate)): # don't consider strings and templates as iterables return False try: iter(o) except TypeError: return False else: return True def convertAnyCollectionToList(o): list_ = None if isinstance(o, list): list_ = o elif isinstance(o, tuple) or isinstance(o, set): list_ = list(o) elif isinstance(o, dict): list_ = o.values() elif isinstance(o, CatList): list_ = [] for item in o.lists(): list_.append(item) if not list_: return o return list_ def convertAnythingToList(o): list_ = None if isinstance(o, list): list_ = o elif isinstance(o, tuple) or isinstance(o, set): list_ = list(o) elif isinstance(o, dict): list_ = o.values() elif isinstance(o, CatList): list_ = [] for item in o.lists(): list_.append(item) if not list_: return [o] return list_ class ASTExpr(Expr): """ A single string template expression enclosed in $...; separator=...$ parsed into an AST chunk to be evaluated. """ DEFAULT_ATTRIBUTE_NAME = 'it' DEFAULT_ATTRIBUTE_KEY = 'ik' DEFAULT_ATTRIBUTE_NAME_DEPRECATED = 'attr' DEFAULT_INDEX_VARIABLE_NAME = 'i' DEFAULT_INDEX0_VARIABLE_NAME = 'i0' DEFAULT_MAP_VALUE_NAME = '_default_' DEFAULT_MAP_KEY_NAME = 'key' MAP_KEY_VALUE = None ## Using an expr option w/o value, makes options table hold EMPTY_OPTION # value for that key. EMPTY_OPTION = "empty expr option" defaultOptionValues = { "anchor": StringTemplateAST(ActionEvaluator.STRING, "true"), "wrap": StringTemplateAST(ActionEvaluator.STRING, "\n") } supportedOptions = set([ "anchor", "format", "null", "separator", "wrap" ]) def __init__(self, enclosingTemplate, exprTree, options): super(ASTExpr, self).__init__(enclosingTemplate) self.exprTree = exprTree ## store separator etc... self.options = options ## A cached value of wrap=expr from the <...> expression. # Computed in write(StringTemplate, StringTemplateWriter) and used # in writeAttribute. self.wrapString = None ## For null values in iterated attributes and single attributes that # are null, use this value instead of skipping. For single valued # attributes like it's a shorthand for # n/a # For iterated values , you get 0 for # for null list values. Works for template application like: # }; null="0"> also. self.nullValue = None ## A cached value of separator=expr from the <...> expression. # Computed in write(StringTemplate, StringTemplateWriter) and used # in writeAttribute. self.separatorString = None # A cached value of option format=expr self.formatString = None ## Return the tree interpreted when self template is written out. @deprecated def getAST(self): return self.exprTree def __str__(self): return str(self.exprTree) ## To write out the value of an ASTExpr, invoke the evaluator in eval.g # to walk the tree writing out the values. For efficiency, don't # compute a bunch of strings and then pack them together. Write out # directly. # # Compute separator and wrap expressions, save as strings so we don't # recompute for each value in a multi-valued attribute or expression. # # If they set anchor option, then inform the writer to push current # char position. def write(self, this, out): if not self.exprTree or not this or not out: return 0 out.pushIndentation(self.indentation) # handle options, anchor, wrap, separator... anchorAST = self.getOption("anchor") if anchorAST is not None: # any non-empty expr means true; check presence out.pushAnchorPoint() self.handleExprOptions(this) evaluator = ActionEvaluator.Walker() evaluator.initialize(this, self, out) n = 0 try: # eval and write out tree n = evaluator.action(self.exprTree) except antlr.RecognitionException, re: this.error('can\'t evaluate tree: ' + self.exprTree.toStringList(), re) out.popIndentation() if anchorAST is not None: out.popAnchorPoint() return n def handleExprOptions(self, this): """Grab and cache options; verify options are valid""" # make sure options don't use format / renderer. They are usually # strings which might invoke a string renderer etc... self.formatString = None wrapAST = self.getOption("wrap") if wrapAST is not None: self.wrapString = self.evaluateExpression(this, wrapAST) nullValueAST = self.getOption("null") if nullValueAST is not None: self.nullValue = self.evaluateExpression(this, nullValueAST) separatorAST = self.getOption("separator") if separatorAST is not None: self.separatorString = self.evaluateExpression(this, separatorAST) formatAST = self.getOption("format") if formatAST is not None: self.formatString = self.evaluateExpression(this, formatAST) if self.options is not None: for option in self.options.keys(): if option not in self.supportedOptions: this.warning("ignoring unsupported option: "+option) # ----------------------------------------------------------------------------- # HELP ROUTINES CALLED BY EVALUATOR TREE WALKER # ----------------------------------------------------------------------------- ## For treat the names, phones as lists # to be walked in lock step as n=names[i], p=phones[i]. # def applyTemplateToListOfAttributes(self, this, attributes, templateToApply): if (not attributes) or (not templateToApply): return None # do not apply if missing templates or empty values argumentContext = None # indicate it's an ST-created list results = stringtemplate3.STAttributeList() # convert all attributes to iterators even if just one value attributesList = [] for o in attributes: if o is not None: o = convertAnythingToList(o) attributesList.append(o) attributes = attributesList numAttributes = len(attributesList) # ensure arguments line up formalArgumentNames = templateToApply.formalArgumentKeys if not formalArgumentNames: this.error('missing arguments in anonymous template in context ' + this.enclosingInstanceStackString) return None if len(formalArgumentNames) != numAttributes: this.error('number of arguments ' + str(formalArgumentNames) + ' mismatch between attribute list and anonymous' + ' template in context ' + this.enclosingInstanceStackString) # truncate arg list to match smaller size shorterSize = min(len(formalArgumentNames), numAttributes) numAttributes = shorterSize formalArgumentNames = formalArgumentNames[:shorterSize] # keep walking while at least one attribute has values i = 0 # iteration number from 0 while True: argumentContext = {} # get a value for each attribute in list; put into arg context # to simulate template invocation of anonymous template numEmpty = 0 for a in range(numAttributes): if attributes[a]: argName = formalArgumentNames[a] argumentContext[argName] = attributes[a][0] attributes[a] = attributes[a][1:] else: numEmpty += 1 if numEmpty == numAttributes: # No attribute values given break argumentContext[self.DEFAULT_INDEX_VARIABLE_NAME] = i + 1 argumentContext[self.DEFAULT_INDEX0_VARIABLE_NAME] = i embedded = templateToApply.getInstanceOf() embedded.enclosingInstance = this embedded.argumentContext = argumentContext results.append(embedded) i += 1 return results def applyListOfAlternatingTemplates(self, this, attributeValue, templatesToApply): if not attributeValue or not templatesToApply or templatesToApply == []: # do not apply if missing templates or empty value return None embedded = None argumentContext = None if isiterable(attributeValue): # results can be treated list an attribute, indicate ST created list resultVector = stringtemplate3.STAttributeList() for i, ithValue in enumerate(attributeValue): if ithValue is None: if self.nullValue is None: continue ithValue = self.nullValue templateIndex = i % len(templatesToApply) # rotate through embedded = templatesToApply[templateIndex] # template to apply is an actual stringtemplate.StringTemplate (created in # eval.g), but that is used as the examplar. We must create # a new instance of the embedded template to apply each time # to get new attribute sets etc... args = embedded.argumentsAST embedded = embedded.getInstanceOf() # make new instance embedded.enclosingInstance = this embedded.argumentsAST = args argumentContext = {} formalArgs = embedded.formalArguments isAnonymous = embedded.name == stringtemplate3.ANONYMOUS_ST_NAME self.setSoleFormalArgumentToIthValue(embedded, argumentContext, ithValue) if isinstance(attributeValue, dict): argumentContext[self.DEFAULT_ATTRIBUTE_KEY] = ithValue # formalArgs might be UNKNOWN, which is non-empty but treated # like 'no formal args'. FIXME: Is this correct if not (isAnonymous and formalArgs is not None and len(formalArgs) > 0 and formalArgs != UNKNOWN_ARGS): argumentContext[self.DEFAULT_ATTRIBUTE_NAME] = { ithValue: attributeValue[ithValue] } argumentContext[self.DEFAULT_ATTRIBUTE_NAME_DEPRECATED] = { ithValue: attributeValue[ithValue] } else: argumentContext[self.DEFAULT_ATTRIBUTE_KEY] = None # formalArgs might be UNKNOWN, which is non-empty but treated # like 'no formal args'. FIXME: Is this correct if not (isAnonymous and formalArgs is not None and len(formalArgs) > 0 and formalArgs != UNKNOWN_ARGS): argumentContext[self.DEFAULT_ATTRIBUTE_NAME] = ithValue argumentContext[self.DEFAULT_ATTRIBUTE_NAME_DEPRECATED] = ithValue argumentContext[self.DEFAULT_INDEX_VARIABLE_NAME] = i+1 argumentContext[self.DEFAULT_INDEX0_VARIABLE_NAME] = i embedded.argumentContext = argumentContext self.evaluateArguments(embedded) #sys.stderr.write('i=' + str(i) + ': applyTemplate(' + # embedded.getName() + ', args=' + # str(argumentContext) + ' to attribute value ' + # str(ithValue) + '\n') resultVector.append(embedded) if not len(resultVector): resultVector = None return resultVector else: embedded = templatesToApply[0] #sys.stderr.write('setting attribute ' + # DEFAULT_ATTRIBUTE_NAME + # ' in arg context of ' + embedded.getName() + # ' to ' + str(attributeValue) + '\n') argumentContext = {} formalArgs = embedded.formalArguments args = embedded.argumentsAST self.setSoleFormalArgumentToIthValue(embedded, argumentContext, attributeValue) isAnonymous = embedded.name == stringtemplate3.ANONYMOUS_ST_NAME # if it's an anonymous template with a formal arg, don't set it/attr # formalArgs might be UNKNOWN, which is non-empty but treated # like 'no formal args'. FIXME: Is this correct if not (isAnonymous and formalArgs is not None and len(formalArgs) > 0 and formalArgs != UNKNOWN_ARGS): argumentContext[self.DEFAULT_ATTRIBUTE_NAME] = attributeValue argumentContext[self.DEFAULT_ATTRIBUTE_NAME_DEPRECATED] = attributeValue argumentContext[self.DEFAULT_INDEX_VARIABLE_NAME] = 1 argumentContext[self.DEFAULT_INDEX0_VARIABLE_NAME] = 0 embedded.argumentContext = argumentContext self.evaluateArguments(embedded) return embedded def setSoleFormalArgumentToIthValue(self, embedded, argumentContext, ithValue): formalArgs = embedded.formalArguments if formalArgs: soleArgName = None isAnonymous = (embedded.name == stringtemplate3.ANONYMOUS_ST_NAME) if len(formalArgs) == 1 or (isAnonymous and len(formalArgs) > 0): if isAnonymous and len(formalArgs) > 1: embedded.error('too many arguments on {...} template: ' + str(formalArgs)) # if exactly 1 arg or anonymous, give that the value of # "it" as a convenience like they said # $list:template(arg=it)$ argNames = formalArgs.keys() soleArgName = argNames[0] argumentContext[soleArgName] = ithValue ## Return o.getPropertyName() given o and propertyName. If o is # a stringtemplate then access it's attributes looking for propertyName # instead (don't check any of the enclosing scopes; look directly into # that object). Also try isXXX() for booleans. Allow HashMap, # Hashtable as special case (grab value for key). # def getObjectProperty(self, this, o, propertyName): if (not o) or (not propertyName): return None value = None # Special case: our automatically created Aggregates via # attribute name: "{obj.{prop1,prop2}}" if isinstance(o, stringtemplate3.Aggregate): #sys.stderr.write("[getObjectProperty] o = " + str(o) + '\n') value = o.get(propertyName, None) if value is None: # no property defined; if a map in this group # then there may be a default value value = o.get(self.DEFAULT_MAP_VALUE_NAME, None) # Or: if it's a dictionary then pull using key not the # property method. elif isinstance(o, dict): if propertyName == 'keys': value = o.keys() elif propertyName == 'values': value = o.values() else: value = o.get(propertyName, None) if value is None: value = o.get(self.DEFAULT_MAP_VALUE_NAME, None) if value is self.MAP_KEY_VALUE: value = propertyName return value # Special case: if it's a template, pull property from # it's attribute table. # TODO: TJP just asked himself why we can't do inherited attr here? elif isinstance(o, stringtemplate3.StringTemplate): attributes = o.attributes if attributes: if attributes.has_key(propertyName): # prevent KeyError... value = attributes[propertyName] else: value = None else: # use getPropertyName() lookup methodSuffix = propertyName[0].upper() + propertyName[1:] methodName = 'get' + methodSuffix m = None if hasattr(o, methodName): m = getattr(o, methodName) else: methodName = 'is' + methodSuffix try: m = getattr(o, methodName) except AttributeError, ae: # try for a visible field try: try: return getattr(o, propertyName) except AttributeError, ae2: this.error('Can\'t get property ' + propertyName + ' using method get/is' + methodSuffix + ' or direct field access from ' + o.__class__.__name__ + ' instance', ae2) except AttributeError, ae: this.error('Class ' + o.__class__.__name__ + ' has no such attribute: ' + propertyName + ' in template context ' + this.enclosingInstanceStackString, ae) if m is not None: try: value = m() except Exception, e: this.error('Can\'t get property ' + propertyName + ' using method get/is' + methodSuffix + ' or direct field access from ' + o.__class__.__name__ + ' instance', e) return value ## Normally StringTemplate tests presence or absence of attributes # for adherence to my principles of separation, but some people # disagree and want to change. # # For 2.0, if the object is a boolean, do something special. $if(boolean)$ # will actually test the value. Now, self breaks my rules of entanglement # listed in my paper, but it truly surprises programmers to have booleans # always True. Further, the key to isolating logic in the model is avoiding # operators (for which you need attribute values). But, no operator is # needed to use boolean values. Well, actually I guess "!" (not) is # an operator. Regardless, for practical reasons, I'm going to technically # violate my rules as I currently have them defined. Perhaps for a future # version of the paper I will refine the rules. # # Post 2.1, I added a check for non-null Iterators, Collections, ... # with size==0 to return false. TJP 5/1/2005 # def testAttributeTrue(self, a): if not a: return False if isinstance(a, bool): return a return True ## For now, we can only add two objects as strings; convert objects to # strings then cat. # def add(self, a, b): # a None value means don't do cat, just return other value if a is None: return b elif b is None: return a return unicode(a) + unicode(b) ## Call a string template with args and return result. Do not convert # to a string yet. It may need attributes that will be available after # self is inserted into another template. # def getTemplateInclude(self, enclosing, templateName, argumentsAST): group = enclosing.group embedded = group.getEmbeddedInstanceOf(templateName, enclosing) if not embedded: enclosing.error('cannot make embedded instance of ' + templateName + ' in template ' + enclosing.getName()) return None embedded.argumentsAST = argumentsAST self.evaluateArguments(embedded) return embedded ## How to spit out an object. If it's not a StringTemplate nor a sequence, # just do o.toString(). If it's a StringTemplate, do o.write(out). # If it's a sequence, do a write(out, o[i]) for all elements. # Note that if you do something weird like set the values of a # multivalued tag to be sequences, it will effectively flatten it. # # If this is an embedded template, you might have specified # a separator arg; used when is a sequence. # def writeAttribute(self, this, o, out): return self._write(this, o, out) def _write(self, this, o, out): """ Write o relative to self to out. John Snyders fixes here for formatString. Basically, any time you are about to write a value, check formatting. """ if o is None: if self.nullValue is None: return 0 o = self.nullValue n = 0 try: if isinstance(o, stringtemplate3.StringTemplate): # failsafe: perhaps enclosing instance not set # Or, it could be set to another context! This occurs # when you store a template instance as an attribute of more # than one template (like both a header file and C file when # generating C code). It must execute within the context of # the enclosing template. o.enclosingInstance = this # if this is found up the enclosing instance chain, then # infinite recursion if stringtemplate3.lintMode and \ stringtemplate3.StringTemplate.isRecursiveEnclosingInstance(o): # throw exception since sometimes eval keeps going # even after I ignore self write of o. raise IllegalStateException('infinite recursion to ' + o.templateDeclaratorString + ' referenced in ' + o.enclosingInstance.templateDeclaratorString + '; stack trace:\n' + o.enclosingInstanceStackTrace) else: # if we have a wrap string, then inform writer it # might need to wrap if self.wrapString is not None: n = out.writeWrapSeparator(self.wrapString) # check if formatting needs to be applied to the stToWrite if self.formatString is not None: renderer = this.getAttributeRenderer(str) if renderer is not None: # you pay a penalty for applying format option to a # template because the template must be written to # a temp StringWriter so it can be formatted before # being written to the real output. buf = StringIO sw = this.getGroup().getStringTemplateWriter(buf) o.write(sw) n = out.write(renderer.toString(buf.getvalue(), self.formatString)) return n n = o.write(out) return n if isiterable(o): if isinstance(o, dict): # for mapping we want to iterate over the values lst = o.values() else: lst = o seenPrevValue = False for iterValue in lst: if iterValue is None: iterValue = self.nullValue if iterValue is not None: if ( seenPrevValue and self.separatorString is not None ): n += out.writeSeparator(self.separatorString) seenPrevValue = True n += self._write(this, iterValue, out) else: renderer = this.getAttributeRenderer(o.__class__) if renderer is not None: v = renderer.toString(o, self.formatString) else: v = unicode(o) if self.wrapString is not None: n = out.write(v, self.wrapString) else: n = out.write(v) return n except IOError, io: this.error('problem writing object: ' + o, io) return n def evaluateExpression(self, this, expr): """ A expr is normally just a string literal, but is still an AST that we must evaluate. The expr can be any expression such as a template include or string cat expression etc... Evaluate with its own writer so that we can convert to string and then reuse, don't want to compute all the time; must precompute w/o writing to output buffer. """ if expr is None: return None if isinstance(expr, StringTemplateAST): # must evaluate, writing to a string so we can hang on to it buf = StringIO() sw = this.group.getStringTemplateWriter(buf) evaluator = ActionEvaluator.Walker() evaluator.initialize(this, self, sw) try: evaluator.action(expr) # eval tree except antlr.RecognitionException, re: this.error( "can't evaluate tree: "+self.exprTree.toStringList(), re ) return buf.getvalue() else: # just in case we expand in the future and it's something else return str(expr) ## Evaluate an argument list within the context of the enclosing # template but store the values in the context of self, the # new embedded template. For example, bold(item=item) means # that bold.item should get the value of enclosing.item. # def evaluateArguments(self, this): argumentsAST = this.argumentsAST if not argumentsAST or not argumentsAST.getFirstChild(): # return immediately if missing tree or no actual args return # Evaluate args in the context of the enclosing template, but we # need the predefined args like 'it', 'attr', and 'i' to be # available as well so we put a dummy ST between the enclosing # context and the embedded context. The dummy has the predefined # context as does the embedded. enclosing = this.enclosingInstance argContextST = stringtemplate3.StringTemplate( group=this.group, template="" ) argContextST.name = '' argContextST.enclosingInstance = enclosing argContextST.argumentContext = this.argumentContext eval_ = ActionEvaluator.Walker() eval_.initialize(argContextST, self, None) #sys.stderr.write('eval args: ' + argumentsAST.toStringList() + '\n') #sys.stderr.write('ctx is ' + this.getArgumentContext()) try: # using any initial argument context (such as when obj is set), # evaluate the arg list like bold(item=obj). Since we pass # in any existing arg context, that context gets filled with # new values. With bold(item=obj), context becomes: #:[obj=...],[item=...]}. ac = eval_.argList(argumentsAST, this, this.argumentContext) this.argumentContext = ac except antlr.RecognitionException, re: this.error('can\'t evaluate tree: ' + argumentsAST.toStringList(), re) ## Return the first attribute if multiple valued or the attribute # itself if single-valued. Used in # def first(self, attribute): if not attribute: return None f = attribute attribute = convertAnyCollectionToList(attribute) if attribute and isinstance(attribute, list): f = attribute[0] return f ## Return the everything but the first attribute if multiple valued # or null if single-valued. Used in . # def rest(self, attribute): if not attribute: return None theRest = attribute attribute = convertAnyCollectionToList(attribute) if not attribute: # if not even one value return None return None if isinstance(attribute, list): # ignore first value attribute = attribute[1:] if not attribute: # if not more than one value, return None return None # return suitably altered iterator theRest = attribute else: # rest of single-valued attribute is None theRest = None return theRest ## Return the last attribute if multiple valued or the attribute # itself if single-valued. Used in . This is pretty # slow as it iterates until the last element. Ultimately, I could # make a special case for a List or Vector. # def last(self, attribute): if not attribute: return None l = attribute attribute = convertAnyCollectionToList(attribute) if attribute and isinstance(attribute, list): l = attribute[-1] return l def strip(self, attribute): """Return an iterator that skips all null values.""" if attribute is None: yield None elif isinstance(attribute, basestring): # don't iterate over string yield attribute else: try: it = iter(attribute) except TypeError: # attribute is not iterable yield attribute else: for value in it: if value is not None: yield value def trunc(self, attribute): """ Return all but the last element. trunc(x)=null if x is single-valued. """ return None # not impl. def length(self, attribute): """ Return the length of a multiple valued attribute or 1 if it is a single attribute. If attribute is null return 0. Special case several common collections and primitive arrays for speed. This method by Kay Roepke. """ if attribute is None: return 0 if isinstance(attribute, (dict, list)): i = len(attribute) elif isinstance(attribute, basestring): # treat strings as atoms i = 1 else: try: it = iter(attribute) except TypeError: # can't iterate over it, we have at least one of something. i = 1 else: # count items i = sum(1 for _ in it) return i def getOption(self, name): value = None if self.options is not None: value = self.options.get(name, None) if value == self.EMPTY_OPTION: return self.defaultOptionValues.get(name, None) return value stringtemplate3-3.1/stringtemplate3/language/ActionEvaluator.py0000444000175000001440000007230110756132623023752 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "eval.g" -> "ActionEvaluator.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> from stringtemplate3.language.CatIterator import CatList from stringtemplate3.language.StringTemplateAST import StringTemplateAST from StringIO import StringIO class NameValuePair(object): def __init__(self): self.name = None self.value = None ### header action <<< ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE APPLY = 4 MULTI_APPLY = 5 ARGS = 6 INCLUDE = 7 CONDITIONAL = 8 VALUE = 9 TEMPLATE = 10 FUNCTION = 11 SINGLEVALUEARG = 12 LIST = 13 NOTHING = 14 SEMI = 15 LPAREN = 16 RPAREN = 17 LITERAL_elseif = 18 COMMA = 19 ID = 20 ASSIGN = 21 COLON = 22 NOT = 23 PLUS = 24 DOT = 25 LITERAL_first = 26 LITERAL_rest = 27 LITERAL_last = 28 LITERAL_length = 29 LITERAL_strip = 30 LITERAL_trunc = 31 LITERAL_super = 32 ANONYMOUS_TEMPLATE = 33 STRING = 34 INT = 35 LBRACK = 36 RBRACK = 37 DOTDOTDOT = 38 TEMPLATE_ARGS = 39 NESTED_ANONYMOUS_TEMPLATE = 40 ESC_CHAR = 41 WS = 42 WS_CHAR = 43 ### user code>>> ### user code<<< class Walker(antlr.TreeParser): # ctor .. def __init__(self, *args, **kwargs): antlr.TreeParser.__init__(self, *args, **kwargs) self.tokenNames = _tokenNames ### __init__ header action >>> self.this = None self.out = None self.chunk = None ### __init__ header action <<< ### user action >>> def initialize(self, this, chunk, out): self.this = this self.chunk = chunk self.out = out def reportError(self, e): self.this.error("eval tree parse error", e) ### user action <<< def action(self, _t): numCharsWritten = 0 action_AST_in = None if _t != antlr.ASTNULL: action_AST_in = _t e = None try: ## for error handling pass e=self.expr(_t) _t = self._retTree numCharsWritten = self.chunk.writeAttribute(self.this, e, self.out) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return numCharsWritten def expr(self, _t): value = None expr_AST_in = None if _t != antlr.ASTNULL: expr_AST_in = _t a = None b = None e = None try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [PLUS]: pass _t3 = _t tmp1_AST_in = _t self.match(_t,PLUS) _t = _t.getFirstChild() a=self.expr(_t) _t = self._retTree b=self.expr(_t) _t = self._retTree value = self.chunk.add(a,b) _t = _t3 _t = _t.getNextSibling() elif la1 and la1 in [APPLY,MULTI_APPLY]: pass value=self.templateApplication(_t) _t = self._retTree elif la1 and la1 in [ID,DOT,ANONYMOUS_TEMPLATE,STRING,INT]: pass value=self.attribute(_t) _t = self._retTree elif la1 and la1 in [INCLUDE]: pass value=self.templateInclude(_t) _t = self._retTree elif la1 and la1 in [FUNCTION]: pass value=self.function(_t) _t = self._retTree elif la1 and la1 in [LIST,NOTHING]: pass value=self.list(_t) _t = self._retTree elif la1 and la1 in [VALUE]: pass _t4 = _t tmp2_AST_in = _t self.match(_t,VALUE) _t = _t.getFirstChild() e=self.expr(_t) _t = self._retTree _t = _t4 _t = _t.getNextSibling() buf = StringIO() sw = self.this.group.getStringTemplateWriter(buf) n = self.chunk.writeAttribute(self.this, e, sw) if n > 0: value = buf.getvalue() else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value ###/** Apply template(s) to an attribute; can be applied to another apply ### * result. ### */ def templateApplication(self, _t): value = None templateApplication_AST_in = None if _t != antlr.ASTNULL: templateApplication_AST_in = _t anon = None a = None templatesToApply = [] attributes = [] try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [APPLY]: pass _t14 = _t tmp3_AST_in = _t self.match(_t,APPLY) _t = _t.getFirstChild() a=self.expr(_t) _t = self._retTree _cnt16= 0 while True: if not _t: _t = antlr.ASTNULL if (_t.getType()==TEMPLATE): pass self.template(_t, templatesToApply) _t = self._retTree else: break _cnt16 += 1 if _cnt16 < 1: raise antlr.NoViableAltException(_t) value = self.chunk.applyListOfAlternatingTemplates(self.this, \ a, templatesToApply) _t = _t14 _t = _t.getNextSibling() elif la1 and la1 in [MULTI_APPLY]: pass _t17 = _t tmp4_AST_in = _t self.match(_t,MULTI_APPLY) _t = _t.getFirstChild() _cnt19= 0 while True: if not _t: _t = antlr.ASTNULL if (_tokenSet_0.member(_t.getType())): pass a=self.expr(_t) _t = self._retTree attributes.append(a) else: break _cnt19 += 1 if _cnt19 < 1: raise antlr.NoViableAltException(_t) tmp5_AST_in = _t self.match(_t,COLON) _t = _t.getNextSibling() anon = _t self.match(_t,ANONYMOUS_TEMPLATE) _t = _t.getNextSibling() anonymous = anon.getStringTemplate() templatesToApply.append(anonymous) value = self.chunk.applyTemplateToListOfAttributes(self.this, \ attributes, anon.getStringTemplate()) _t = _t17 _t = _t.getNextSibling() else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value def attribute(self, _t): value = None attribute_AST_in = None if _t != antlr.ASTNULL: attribute_AST_in = _t prop = None i3 = None i = None s = None at = None obj = None propName = None e = None try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [DOT]: pass _t33 = _t tmp6_AST_in = _t self.match(_t,DOT) _t = _t.getFirstChild() obj=self.expr(_t) _t = self._retTree if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [ID]: pass prop = _t self.match(_t,ID) _t = _t.getNextSibling() propName = prop.getText() elif la1 and la1 in [VALUE]: pass _t35 = _t tmp7_AST_in = _t self.match(_t,VALUE) _t = _t.getFirstChild() e=self.expr(_t) _t = self._retTree _t = _t35 _t = _t.getNextSibling() if e: propName = str(e) else: raise antlr.NoViableAltException(_t) _t = _t33 _t = _t.getNextSibling() value = self.chunk.getObjectProperty(self.this, obj, propName) elif la1 and la1 in [ID]: pass i3 = _t self.match(_t,ID) _t = _t.getNextSibling() value = self.this.getAttribute(i3.getText()) elif la1 and la1 in [INT]: pass i = _t self.match(_t,INT) _t = _t.getNextSibling() value = int(i.getText()) elif la1 and la1 in [STRING]: pass s = _t self.match(_t,STRING) _t = _t.getNextSibling() value = s.getText() elif la1 and la1 in [ANONYMOUS_TEMPLATE]: pass at = _t self.match(_t,ANONYMOUS_TEMPLATE) _t = _t.getNextSibling() value = at.getText(); if at.getText(): from stringtemplate3.templates import StringTemplate valueST = StringTemplate( group=self.this.group, template=at.getText() ) valueST.enclosingInstance = self.this valueST.name = "" value = valueST else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value def templateInclude(self, _t): value = None templateInclude_AST_in = None if _t != antlr.ASTNULL: templateInclude_AST_in = _t id = None a1 = None a2 = None args = None name = "" n = None try: ## for error handling pass _t10 = _t tmp8_AST_in = _t self.match(_t,INCLUDE) _t = _t.getFirstChild() if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [ID]: pass id = _t self.match(_t,ID) _t = _t.getNextSibling() a1 = _t if not _t: raise antlr.MismatchedTokenException() _t = _t.getNextSibling() name=id.getText(); args=a1 elif la1 and la1 in [VALUE]: pass _t12 = _t tmp9_AST_in = _t self.match(_t,VALUE) _t = _t.getFirstChild() n=self.expr(_t) _t = self._retTree a2 = _t if not _t: raise antlr.MismatchedTokenException() _t = _t.getNextSibling() _t = _t12 _t = _t.getNextSibling() if n: name = str(n); args=a2 else: raise antlr.NoViableAltException(_t) _t = _t10 _t = _t.getNextSibling() if name: value = self.chunk.getTemplateInclude(self.this, name, args) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value def function(self, _t): value = None function_AST_in = None if _t != antlr.ASTNULL: function_AST_in = _t a = None try: ## for error handling pass _t21 = _t tmp10_AST_in = _t self.match(_t,FUNCTION) _t = _t.getFirstChild() if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [LITERAL_first]: pass tmp11_AST_in = _t self.match(_t,LITERAL_first) _t = _t.getNextSibling() a=self.singleFunctionArg(_t) _t = self._retTree value = self.chunk.first(a) elif la1 and la1 in [LITERAL_rest]: pass tmp12_AST_in = _t self.match(_t,LITERAL_rest) _t = _t.getNextSibling() a=self.singleFunctionArg(_t) _t = self._retTree value = self.chunk.rest(a) elif la1 and la1 in [LITERAL_last]: pass tmp13_AST_in = _t self.match(_t,LITERAL_last) _t = _t.getNextSibling() a=self.singleFunctionArg(_t) _t = self._retTree value = self.chunk.last(a) elif la1 and la1 in [LITERAL_length]: pass tmp14_AST_in = _t self.match(_t,LITERAL_length) _t = _t.getNextSibling() a=self.singleFunctionArg(_t) _t = self._retTree value = self.chunk.length(a) elif la1 and la1 in [LITERAL_strip]: pass tmp15_AST_in = _t self.match(_t,LITERAL_strip) _t = _t.getNextSibling() a=self.singleFunctionArg(_t) _t = self._retTree value = self.chunk.strip(a) elif la1 and la1 in [LITERAL_trunc]: pass tmp16_AST_in = _t self.match(_t,LITERAL_trunc) _t = _t.getNextSibling() a=self.singleFunctionArg(_t) _t = self._retTree value = self.chunk.trunc(a) else: raise antlr.NoViableAltException(_t) _t = _t21 _t = _t.getNextSibling() except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value ###/** create a new list of expressions as a new multi-value attribute */ def list(self, _t): value=None list_AST_in = None if _t != antlr.ASTNULL: list_AST_in = _t e = None elements = [] value = CatList(elements) try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [LIST]: pass _t6 = _t tmp17_AST_in = _t self.match(_t,LIST) _t = _t.getFirstChild() _cnt8= 0 while True: if not _t: _t = antlr.ASTNULL if (_tokenSet_0.member(_t.getType())): pass e=self.expr(_t) _t = self._retTree if e: from stringtemplate3.language.ASTExpr import convertAnythingToList e = convertAnythingToList(e) elements.append(e) else: break _cnt8 += 1 if _cnt8 < 1: raise antlr.NoViableAltException(_t) _t = _t6 _t = _t.getNextSibling() elif la1 and la1 in [NOTHING]: pass tmp18_AST_in = _t self.match(_t,NOTHING) _t = _t.getNextSibling() nullSingleton = [None] element.append(nullSingleton) else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value def template(self, _t, templatesToApply ): template_AST_in = None if _t != antlr.ASTNULL: template_AST_in = _t t = None args = None anon = None args2 = None argumentContext = {} n = None try: ## for error handling pass _t26 = _t tmp19_AST_in = _t self.match(_t,TEMPLATE) _t = _t.getFirstChild() if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [ID]: pass t = _t self.match(_t,ID) _t = _t.getNextSibling() args = _t if not _t: raise antlr.MismatchedTokenException() _t = _t.getNextSibling() templateName = t.getText() group = self.this.group embedded = group.getEmbeddedInstanceOf( templateName, self.this ) if embedded: embedded.argumentsAST = args templatesToApply.append(embedded) elif la1 and la1 in [ANONYMOUS_TEMPLATE]: pass anon = _t self.match(_t,ANONYMOUS_TEMPLATE) _t = _t.getNextSibling() anonymous = anon.getStringTemplate() # to properly see overridden templates, always set # anonymous' group to be self's group anonymous.group = self.this.group templatesToApply.append(anonymous) elif la1 and la1 in [VALUE]: pass _t28 = _t tmp20_AST_in = _t self.match(_t,VALUE) _t = _t.getFirstChild() n=self.expr(_t) _t = self._retTree args2 = _t if not _t: raise antlr.MismatchedTokenException() _t = _t.getNextSibling() embedded = None if n: templateName = str(n) group = self.this.group embedded = group.getEmbeddedInstanceOf( templateName, self.this ) if embedded: embedded.argumentsAST = args2 templatesToApply.append(embedded) _t = _t28 _t = _t.getNextSibling() else: raise antlr.NoViableAltException(_t) _t = _t26 _t = _t.getNextSibling() except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t def singleFunctionArg(self, _t): value = None singleFunctionArg_AST_in = None if _t != antlr.ASTNULL: singleFunctionArg_AST_in = _t try: ## for error handling pass _t24 = _t tmp21_AST_in = _t self.match(_t,SINGLEVALUEARG) _t = _t.getFirstChild() value=self.expr(_t) _t = self._retTree _t = _t24 _t = _t.getNextSibling() except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value def ifCondition(self, _t): value = False ifCondition_AST_in = None if _t != antlr.ASTNULL: ifCondition_AST_in = _t a = None try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [APPLY,MULTI_APPLY,INCLUDE,VALUE,FUNCTION,LIST,NOTHING,ID,PLUS,DOT,ANONYMOUS_TEMPLATE,STRING,INT]: pass a=self.ifAtom(_t) _t = self._retTree value = self.chunk.testAttributeTrue(a) elif la1 and la1 in [NOT]: pass _t30 = _t tmp22_AST_in = _t self.match(_t,NOT) _t = _t.getFirstChild() a=self.ifAtom(_t) _t = self._retTree _t = _t30 _t = _t.getNextSibling() value = not self.chunk.testAttributeTrue(a) else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value def ifAtom(self, _t): value = None ifAtom_AST_in = None if _t != antlr.ASTNULL: ifAtom_AST_in = _t try: ## for error handling pass value=self.expr(_t) _t = self._retTree except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return value ###/** self is assumed to be the enclosing context as foo(x=y) must find y in ### * the template that encloses the ref to foo(x=y). We must pass in ### * the embedded template (the one invoked) so we can check formal args ### * in rawSetArgumentAttribute. ### */ def argList(self, _t, embedded, initialContext ): argumentContext = None argList_AST_in = None if _t != antlr.ASTNULL: argList_AST_in = _t argumentContext = initialContext if not argumentContext: argumentContext = {} try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [ARGS]: pass _t37 = _t tmp23_AST_in = _t self.match(_t,ARGS) _t = _t.getFirstChild() while True: if not _t: _t = antlr.ASTNULL if (_t.getType()==ASSIGN or _t.getType()==DOTDOTDOT): pass self.argumentAssignment(_t, embedded, argumentContext) _t = self._retTree else: break _t = _t37 _t = _t.getNextSibling() elif la1 and la1 in [SINGLEVALUEARG]: pass self.singleTemplateArg(_t, embedded, argumentContext) _t = self._retTree else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t return argumentContext def argumentAssignment(self, _t, embedded, argumentContext ): argumentAssignment_AST_in = None if _t != antlr.ASTNULL: argumentAssignment_AST_in = _t arg = None e = None try: ## for error handling if not _t: _t = antlr.ASTNULL la1 = _t.getType() if False: pass elif la1 and la1 in [ASSIGN]: pass _t43 = _t tmp24_AST_in = _t self.match(_t,ASSIGN) _t = _t.getFirstChild() arg = _t self.match(_t,ID) _t = _t.getNextSibling() e=self.expr(_t) _t = self._retTree _t = _t43 _t = _t.getNextSibling() if e: self.this.rawSetArgumentAttribute(embedded, argumentContext, \ arg.getText(), e) elif la1 and la1 in [DOTDOTDOT]: pass tmp25_AST_in = _t self.match(_t,DOTDOTDOT) _t = _t.getNextSibling() embedded.passThroughAttributes = True else: raise antlr.NoViableAltException(_t) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t def singleTemplateArg(self, _t, embedded, argumentContext ): singleTemplateArg_AST_in = None if _t != antlr.ASTNULL: singleTemplateArg_AST_in = _t e = None try: ## for error handling pass _t41 = _t tmp26_AST_in = _t self.match(_t,SINGLEVALUEARG) _t = _t.getFirstChild() e=self.expr(_t) _t = self._retTree _t = _t41 _t = _t.getNextSibling() if e: soleArgName = None # find the sole defined formal argument for embedded error = False formalArgs = embedded.formalArguments if formalArgs: argNames = formalArgs.keys() if len(argNames) == 1: soleArgName = argNames[0] #sys.stderr.write("sole formal arg of " + # embedded.getName() + " is " + # soleArgName) else: error = True else: error = True if error: self.this.error("template " + embedded.name + " must have exactly one formal arg in" + " template context " + self.this.enclosingInstanceStackString) else: self.this.rawSetArgumentAttribute(embedded, \ argumentContext, soleArgName, e) except antlr.RecognitionException, ex: self.reportError(ex) if _t: _t = _t.getNextSibling() self._retTree = _t _tokenNames = [ "<0>", "EOF", "<2>", "NULL_TREE_LOOKAHEAD", "APPLY", "MULTI_APPLY", "ARGS", "INCLUDE", "\"if\"", "VALUE", "TEMPLATE", "FUNCTION", "SINGLEVALUEARG", "LIST", "NOTHING", "SEMI", "LPAREN", "RPAREN", "\"elseif\"", "COMMA", "ID", "ASSIGN", "COLON", "NOT", "PLUS", "DOT", "\"first\"", "\"rest\"", "\"last\"", "\"length\"", "\"strip\"", "\"trunc\"", "\"super\"", "ANONYMOUS_TEMPLATE", "STRING", "INT", "LBRACK", "RBRACK", "DOTDOTDOT", "TEMPLATE_ARGS", "NESTED_ANONYMOUS_TEMPLATE", "ESC_CHAR", "WS", "WS_CHAR" ] ### generate bit set def mk_tokenSet_0(): ### var1 data = [ 60180949680L, 0L] return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) stringtemplate3-3.1/stringtemplate3/language/ChunkToken.py0000444000175000001440000000152010756132623022716 0ustar pinkusers import antlr from stringtemplate3.utils import deprecated class ChunkToken(antlr.CommonToken): """ Tracks the various string and attribute chunks discovered by the lexer. Subclassed CommonToken so that I could pass the indentation to the parser, which will add it to the ASTExpr created for the $...$ attribute reference. """ def __init__(self, type = None, text = '', indentation = ''): antlr.CommonToken.__init__(self, type=type, text=text) self.indentation = indentation @deprecated def getIndentation(self): return self.indentation @deprecated def setIndentation(self, indentation): self.indentation = indentation def __str__(self): return (antlr.CommonToken.__str__(self) + " " % self.indentation ) stringtemplate3-3.1/stringtemplate3/language/GroupParser.py0000444000175000001440000004375310756132624023135 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "group.g" -> "GroupParser.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> from ASTExpr import * import stringtemplate3 import traceback ### header action <<< ### preamble action>>> ### preamble action <<< ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL_group = 4 ID = 5 COLON = 6 LITERAL_implements = 7 COMMA = 8 SEMI = 9 AT = 10 DOT = 11 LPAREN = 12 RPAREN = 13 DEFINED_TO_BE = 14 STRING = 15 BIGSTRING = 16 ASSIGN = 17 ANONYMOUS_TEMPLATE = 18 LBRACK = 19 RBRACK = 20 LITERAL_default = 21 STAR = 22 PLUS = 23 OPTIONAL = 24 SL_COMMENT = 25 ML_COMMENT = 26 NL = 27 WS = 28 ###/** Match a group of template definitions beginning ### * with a group name declaration. Templates are enclosed ### * in double-quotes or <<...>> quotes for multi-line templates. ### * Template names have arg lists that indicate the cardinality ### * of the attribute: present, optional, zero-or-more, one-or-more. ### * Here is a sample group file: ### ### group nfa; ### ### // an NFA has edges and states ### nfa(states,edges) ::= << ### digraph NFA { ### rankdir=LR; ### ### ### } ### >> ### ### state(name) ::= "node [shape = circle]; ;" ### ### */ class Parser(antlr.LLkParser): ### user action >>> def reportError(self, e): if self.group_: self.group_.error("template group parse error", e) else: sys.stderr.write("template group parse error: " + str(e) + '\n') traceback.print_exc() ### user action <<< def __init__(self, *args, **kwargs): antlr.LLkParser.__init__(self, *args, **kwargs) self.tokenNames = _tokenNames ### __init__ header action >>> self.group_ = None ### __init__ header action <<< def group(self, g ): name = None s = None i = None i2 = None self.group_ = g try: ## for error handling pass self.match(LITERAL_group) name = self.LT(1) self.match(ID) g.name = name.getText() la1 = self.LA(1) if False: pass elif la1 and la1 in [COLON]: pass self.match(COLON) s = self.LT(1) self.match(ID) g.superGroup = s.getText() elif la1 and la1 in [LITERAL_implements,SEMI]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) la1 = self.LA(1) if False: pass elif la1 and la1 in [LITERAL_implements]: pass self.match(LITERAL_implements) i = self.LT(1) self.match(ID) g.implementInterface(i.getText()) while True: if (self.LA(1)==COMMA): pass self.match(COMMA) i2 = self.LT(1) self.match(ID) g.implementInterface(i2.getText()) else: break elif la1 and la1 in [SEMI]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) self.match(SEMI) while True: if (self.LA(1)==ID or self.LA(1)==AT) and (self.LA(2)==ID or self.LA(2)==LPAREN or self.LA(2)==DEFINED_TO_BE) and (self.LA(3)==ID or self.LA(3)==DOT or self.LA(3)==RPAREN): pass self.template(g) elif (self.LA(1)==ID) and (self.LA(2)==DEFINED_TO_BE) and (self.LA(3)==LBRACK): pass self.mapdef(g) else: break self.match(EOF_TYPE) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_0) def template(self, g ): scope = None region = None name = None t = None bt = None alias = None target = None formalArgs = {} st = None ignore = False templateName = None line = self.LT(1).getLine() try: ## for error handling if (self.LA(1)==ID or self.LA(1)==AT) and (self.LA(2)==ID or self.LA(2)==LPAREN): pass la1 = self.LA(1) if False: pass elif la1 and la1 in [AT]: pass self.match(AT) scope = self.LT(1) self.match(ID) self.match(DOT) region = self.LT(1) self.match(ID) templateName = g.getMangledRegionName(scope.getText(),region.getText()) if g.isDefinedInThisGroup(templateName): g.error("group "+g.name+" line "+str(line)+": redefinition of template region: @"+ scope.getText()+"."+region.getText()) st = stringtemplate3.StringTemplate() # create bogus template to fill in else: err = False # @template.region() ::= "..." scopeST = g.lookupTemplate(scope.getText()) if scopeST is None: g.error("group "+g.name+" line "+str(line)+": reference to region within undefined template: "+ scope.getText()) err = True if not scopeST.containsRegionName(region.getText()): g.error("group "+g.name+" line "+str(line)+": template "+scope.getText()+" has no region called "+ region.getText()) err = True if err: st = stringtemplate3.StringTemplate() else: st = g.defineRegionTemplate( scope.getText(), region.getText(), None, stringtemplate3.REGION_EXPLICIT ) elif la1 and la1 in [ID]: pass name = self.LT(1) self.match(ID) templateName = name.getText() if g.isDefinedInThisGroup(templateName): g.error("redefinition of template: " + templateName) # create bogus template to fill in st = stringtemplate3.StringTemplate() else: st = g.defineTemplate(templateName, None) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) if st is not None: st.groupFileLine = line self.match(LPAREN) la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass self.args(st) elif la1 and la1 in [RPAREN]: pass st.defineEmptyFormalArgumentList() else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) self.match(RPAREN) self.match(DEFINED_TO_BE) la1 = self.LA(1) if False: pass elif la1 and la1 in [STRING]: pass t = self.LT(1) self.match(STRING) st.template = t.getText() elif la1 and la1 in [BIGSTRING]: pass bt = self.LT(1) self.match(BIGSTRING) st.template = bt.getText() else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) elif (self.LA(1)==ID) and (self.LA(2)==DEFINED_TO_BE): pass alias = self.LT(1) self.match(ID) self.match(DEFINED_TO_BE) target = self.LT(1) self.match(ID) g.defineTemplateAlias(alias.getText(), target.getText()) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_1) def mapdef(self, g ): name = None m = None try: ## for error handling pass name = self.LT(1) self.match(ID) self.match(DEFINED_TO_BE) m=self.map() if g.getMap(name.getText()): g.error("redefinition of map: " + name.getText()) elif g.isDefinedInThisGroup(name.getText()): g.error("redefinition of template as map: " + name.getText()) else: g.defineMap(name.getText(), m) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_1) def args(self, st ): try: ## for error handling pass self.arg(st) while True: if (self.LA(1)==COMMA): pass self.match(COMMA) self.arg(st) else: break except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_2) def arg(self, st ): name = None s = None bs = None defaultValue = None try: ## for error handling pass name = self.LT(1) self.match(ID) if (self.LA(1)==ASSIGN) and (self.LA(2)==STRING): pass self.match(ASSIGN) s = self.LT(1) self.match(STRING) defaultValue = stringtemplate3.StringTemplate( template="$_val_$" ) defaultValue["_val_"] = s.getText() defaultValue.defineFormalArgument("_val_") defaultValue.name = ("<" + st.name + "'s arg " + name.getText() + " default value subtemplate>") elif (self.LA(1)==ASSIGN) and (self.LA(2)==ANONYMOUS_TEMPLATE): pass self.match(ASSIGN) bs = self.LT(1) self.match(ANONYMOUS_TEMPLATE) defaultValue = stringtemplate3.StringTemplate( group=st.group, template=bs.getText() ) defaultValue.name = ("<" + st.name + "'s arg " + name.getText() + " default value subtemplate>") elif (self.LA(1)==COMMA or self.LA(1)==RPAREN): pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) st.defineFormalArgument(name.getText(), defaultValue) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_3) def map(self): mapping = {} try: ## for error handling pass self.match(LBRACK) self.mapPairs(mapping) self.match(RBRACK) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_1) return mapping def mapPairs(self, mapping ): try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [STRING]: pass self.keyValuePair(mapping) while True: if (self.LA(1)==COMMA) and (self.LA(2)==STRING): pass self.match(COMMA) self.keyValuePair(mapping) else: break la1 = self.LA(1) if False: pass elif la1 and la1 in [COMMA]: pass self.match(COMMA) self.defaultValuePair(mapping) elif la1 and la1 in [RBRACK]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) elif la1 and la1 in [LITERAL_default]: pass self.defaultValuePair(mapping) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_4) def keyValuePair(self, mapping ): key = None try: ## for error handling pass key = self.LT(1) self.match(STRING) self.match(COLON) v=self.keyValue() mapping[key.getText()] = v except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_5) def defaultValuePair(self, mapping ): try: ## for error handling pass self.match(LITERAL_default) self.match(COLON) v=self.keyValue() mapping[stringtemplate3.language.ASTExpr.DEFAULT_MAP_VALUE_NAME] = v except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_4) def keyValue(self): value = None s1 = None s2 = None k = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [STRING]: pass s1 = self.LT(1) self.match(STRING) value = stringtemplate3.StringTemplate( group=self.group_, template=s1.getText() ) elif la1 and la1 in [BIGSTRING]: pass s2 = self.LT(1) self.match(BIGSTRING) value = stringtemplate3.StringTemplate( group=self.group_, template=s2.getText() ) elif la1 and la1 in [ID]: pass k = self.LT(1) self.match(ID) if not k.getText() == "key" : raise antlr.SemanticException(" k.getText() == \"key\" ") value = stringtemplate3.language.ASTExpr.MAP_KEY_VALUE elif la1 and la1 in [COMMA,RBRACK]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_5) return value _tokenNames = [ "<0>", "EOF", "<2>", "NULL_TREE_LOOKAHEAD", "\"group\"", "ID", "COLON", "\"implements\"", "COMMA", "SEMI", "AT", "DOT", "LPAREN", "RPAREN", "DEFINED_TO_BE", "STRING", "BIGSTRING", "ASSIGN", "ANONYMOUS_TEMPLATE", "LBRACK", "RBRACK", "\"default\"", "STAR", "PLUS", "OPTIONAL", "SL_COMMENT", "ML_COMMENT", "NL", "WS" ] ### generate bit set def mk_tokenSet_0(): ### var1 data = [ 2L, 0L] return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): ### var1 data = [ 1058L, 0L] return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): ### var1 data = [ 8192L, 0L] return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) ### generate bit set def mk_tokenSet_3(): ### var1 data = [ 8448L, 0L] return data _tokenSet_3 = antlr.BitSet(mk_tokenSet_3()) ### generate bit set def mk_tokenSet_4(): ### var1 data = [ 1048576L, 0L] return data _tokenSet_4 = antlr.BitSet(mk_tokenSet_4()) ### generate bit set def mk_tokenSet_5(): ### var1 data = [ 1048832L, 0L] return data _tokenSet_5 = antlr.BitSet(mk_tokenSet_5()) stringtemplate3-3.1/stringtemplate3/language/ActionLexer.py0000444000175000001440000006412010756132623023067 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "action.g" -> "ActionLexer.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> from stringtemplate3.language.StringTemplateToken import StringTemplateToken import stringtemplate3 ### header action <<< ### preamble action >>> ### preamble action <<< ### >>>The Literals<<< literals = {} literals[u"super"] = 32 literals[u"if"] = 8 literals[u"first"] = 26 literals[u"last"] = 28 literals[u"rest"] = 27 literals[u"trunc"] = 31 literals[u"strip"] = 30 literals[u"length"] = 29 literals[u"elseif"] = 18 ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE APPLY = 4 MULTI_APPLY = 5 ARGS = 6 INCLUDE = 7 CONDITIONAL = 8 VALUE = 9 TEMPLATE = 10 FUNCTION = 11 SINGLEVALUEARG = 12 LIST = 13 NOTHING = 14 SEMI = 15 LPAREN = 16 RPAREN = 17 LITERAL_elseif = 18 COMMA = 19 ID = 20 ASSIGN = 21 COLON = 22 NOT = 23 PLUS = 24 DOT = 25 LITERAL_first = 26 LITERAL_rest = 27 LITERAL_last = 28 LITERAL_length = 29 LITERAL_strip = 30 LITERAL_trunc = 31 LITERAL_super = 32 ANONYMOUS_TEMPLATE = 33 STRING = 34 INT = 35 LBRACK = 36 RBRACK = 37 DOTDOTDOT = 38 TEMPLATE_ARGS = 39 NESTED_ANONYMOUS_TEMPLATE = 40 ESC_CHAR = 41 WS = 42 WS_CHAR = 43 class Lexer(antlr.CharScanner) : ### user action >>> ### user action <<< def __init__(self, *argv, **kwargs) : antlr.CharScanner.__init__(self, *argv, **kwargs) self.caseSensitiveLiterals = True self.setCaseSensitive(True) self.literals = literals def nextToken(self): while True: try: ### try again .. while True: _token = None _ttype = INVALID_TYPE self.resetText() try: ## for char stream error handling try: ##for lexical error handling la1 = self.LA(1) if False: pass elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz': pass self.mID(True) theRetToken = self._returnToken elif la1 and la1 in u'0123456789': pass self.mINT(True) theRetToken = self._returnToken elif la1 and la1 in u'"': pass self.mSTRING(True) theRetToken = self._returnToken elif la1 and la1 in u'{': pass self.mANONYMOUS_TEMPLATE(True) theRetToken = self._returnToken elif la1 and la1 in u'[': pass self.mLBRACK(True) theRetToken = self._returnToken elif la1 and la1 in u']': pass self.mRBRACK(True) theRetToken = self._returnToken elif la1 and la1 in u'(': pass self.mLPAREN(True) theRetToken = self._returnToken elif la1 and la1 in u')': pass self.mRPAREN(True) theRetToken = self._returnToken elif la1 and la1 in u',': pass self.mCOMMA(True) theRetToken = self._returnToken elif la1 and la1 in u'=': pass self.mASSIGN(True) theRetToken = self._returnToken elif la1 and la1 in u':': pass self.mCOLON(True) theRetToken = self._returnToken elif la1 and la1 in u'+': pass self.mPLUS(True) theRetToken = self._returnToken elif la1 and la1 in u';': pass self.mSEMI(True) theRetToken = self._returnToken elif la1 and la1 in u'!': pass self.mNOT(True) theRetToken = self._returnToken elif la1 and la1 in u'\t\n\r ': pass self.mWS(True) theRetToken = self._returnToken else: if (self.LA(1)==u'.') and (self.LA(2)==u'.'): pass self.mDOTDOTDOT(True) theRetToken = self._returnToken elif (self.LA(1)==u'.') and (True): pass self.mDOT(True) theRetToken = self._returnToken else: self.default(self.LA(1)) if not self._returnToken: raise antlr.TryAgain ### found SKIP token ### return token to caller return self._returnToken ### handle lexical errors .... except antlr.RecognitionException, e: raise antlr.TokenStreamRecognitionException(e) ### handle char stream errors ... except antlr.CharStreamException,cse: if isinstance(cse, antlr.CharStreamIOException): raise antlr.TokenStreamIOException(cse.io) else: raise antlr.TokenStreamException(str(cse)) except antlr.TryAgain: pass def mID(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ID _saveIndex = 0 pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'abcdefghijklmnopqrstuvwxyz': pass self.matchRange(u'a', u'z') elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ': pass self.matchRange(u'A', u'Z') elif la1 and la1 in u'_': pass self.match('_') else: self.raise_NoViableAlt(self.LA(1)) while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'abcdefghijklmnopqrstuvwxyz': pass self.matchRange(u'a', u'z') elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ': pass self.matchRange(u'A', u'Z') elif la1 and la1 in u'0123456789': pass self.matchRange(u'0', u'9') elif la1 and la1 in u'_': pass self.match('_') elif la1 and la1 in u'/': pass self.match('/') else: break ### option { testLiterals=true } _ttype = self.testLiteralsTable(_ttype) self.set_return_token(_createToken, _token, _ttype, _begin) def mINT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = INT _saveIndex = 0 pass _cnt63= 0 while True: if ((self.LA(1) >= u'0' and self.LA(1) <= u'9')): pass self.matchRange(u'0', u'9') else: break _cnt63 += 1 if _cnt63 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mSTRING(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = STRING _saveIndex = 0 pass _saveIndex = self.text.length() self.match('"') self.text.setLength(_saveIndex) while True: if (self.LA(1)==u'\\'): pass self.mESC_CHAR(False, True) elif (_tokenSet_0.member(self.LA(1))): pass self.matchNot('"') else: break _saveIndex = self.text.length() self.match('"') self.text.setLength(_saveIndex) self.set_return_token(_createToken, _token, _ttype, _begin) ###/** Match escape sequences, optionally translating them for strings, but not ### * for templates. Do \} only when in {...} templates. ### */ def mESC_CHAR(self, _createToken, doEscape ): _ttype = 0 _token = None _begin = self.text.length() _ttype = ESC_CHAR _saveIndex = 0 c = '\0' pass self.match('\\') if (self.LA(1)==u'n') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.match('n') if not self.inputState.guessing: if doEscape: self.text.setLength(_begin) ; self.text.append("\n") elif (self.LA(1)==u'r') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.match('r') if not self.inputState.guessing: if doEscape: self.text.setLength(_begin) ; self.text.append("\r") elif (self.LA(1)==u't') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.match('t') if not self.inputState.guessing: if doEscape: self.text.setLength(_begin) ; self.text.append("\t") elif (self.LA(1)==u'b') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.match('b') if not self.inputState.guessing: if doEscape: self.text.setLength(_begin) ; self.text.append("\b") elif (self.LA(1)==u'f') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.match('f') if not self.inputState.guessing: if doEscape: self.text.setLength(_begin) ; self.text.append("\f") elif ((self.LA(1) >= u'\u0003' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass c = self.LA(1) self.matchNot(antlr.EOF_CHAR) if not self.inputState.guessing: if doEscape: self.text.setLength(_begin) ; self.text.append(str(c)) else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mANONYMOUS_TEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ANONYMOUS_TEMPLATE _saveIndex = 0 args = None t = None pass _saveIndex = self.text.length() self.match('{') self.text.setLength(_saveIndex) synPredMatched70 = False if (_tokenSet_1.member(self.LA(1))) and (_tokenSet_2.member(self.LA(2))): _m70 = self.mark() synPredMatched70 = True self.inputState.guessing += 1 try: pass self.mTEMPLATE_ARGS(False) except antlr.RecognitionException, pe: synPredMatched70 = False self.rewind(_m70) self.inputState.guessing -= 1 if synPredMatched70: pass args=self.mTEMPLATE_ARGS(False) if (_tokenSet_3.member(self.LA(1))) and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass _saveIndex = self.text.length() self.mWS_CHAR(False) self.text.setLength(_saveIndex) elif ((self.LA(1) >= u'\u0003' and self.LA(1) <= u'\ufffe')) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) if not self.inputState.guessing: # create a special token to track args t = StringTemplateToken(ANONYMOUS_TEMPLATE, self.text.getString(_begin), args) _token = t elif ((self.LA(1) >= u'\u0003' and self.LA(1) <= u'\ufffe')) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) while True: if (self.LA(1)==u'\\') and (self.LA(2)==u'{'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('{') elif (self.LA(1)==u'\\') and (self.LA(2)==u'}'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('}') elif (self.LA(1)==u'\\') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.mESC_CHAR(False, False) elif (self.LA(1)==u'{'): pass self.mNESTED_ANONYMOUS_TEMPLATE(False) elif (_tokenSet_4.member(self.LA(1))): pass self.matchNot('}') else: break if not self.inputState.guessing: if t: t.setText(self.text.getString(_begin)) _saveIndex = self.text.length() self.match('}') self.text.setLength(_saveIndex) self.set_return_token(_createToken, _token, _ttype, _begin) def mTEMPLATE_ARGS(self, _createToken): args=[] _ttype = 0 _token = None _begin = self.text.length() _ttype = TEMPLATE_ARGS _saveIndex = 0 a = None a2 = None pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\t\n\r ': pass _saveIndex = self.text.length() self.mWS_CHAR(False) self.text.setLength(_saveIndex) elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.mID(True) self.text.setLength(_saveIndex) a = self._returnToken if not self.inputState.guessing: args.append(a.getText()) while True: if (_tokenSet_5.member(self.LA(1))) and (_tokenSet_6.member(self.LA(2))): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\t\n\r ': pass _saveIndex = self.text.length() self.mWS_CHAR(False) self.text.setLength(_saveIndex) elif la1 and la1 in u',': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match(',') self.text.setLength(_saveIndex) la1 = self.LA(1) if False: pass elif la1 and la1 in u'\t\n\r ': pass _saveIndex = self.text.length() self.mWS_CHAR(False) self.text.setLength(_saveIndex) elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.mID(True) self.text.setLength(_saveIndex) a2 = self._returnToken if not self.inputState.guessing: args.append(a2.getText()) else: break la1 = self.LA(1) if False: pass elif la1 and la1 in u'\t\n\r ': pass _saveIndex = self.text.length() self.mWS_CHAR(False) self.text.setLength(_saveIndex) elif la1 and la1 in u'|': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('|') self.text.setLength(_saveIndex) self.set_return_token(_createToken, _token, _ttype, _begin) return args def mWS_CHAR(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = WS_CHAR _saveIndex = 0 la1 = self.LA(1) if False: pass elif la1 and la1 in u' ': pass self.match(' ') elif la1 and la1 in u'\t': pass self.match('\t') elif la1 and la1 in u'\n\r': pass if (self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.match('\r') elif (self.LA(1)==u'\r') and (self.LA(2)==u'\n'): pass self.match('\r') self.match('\n') elif (self.LA(1)==u'\n'): pass self.match('\n') else: self.raise_NoViableAlt(self.LA(1)) if not self.inputState.guessing: self.newline() else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mNESTED_ANONYMOUS_TEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NESTED_ANONYMOUS_TEMPLATE _saveIndex = 0 pass self.match('{') while True: if (self.LA(1)==u'\\') and (self.LA(2)==u'{'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('{') elif (self.LA(1)==u'\\') and (self.LA(2)==u'}'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('}') elif (self.LA(1)==u'\\') and ((self.LA(2) >= u'\u0003' and self.LA(2) <= u'\ufffe')): pass self.mESC_CHAR(False, False) elif (self.LA(1)==u'{'): pass self.mNESTED_ANONYMOUS_TEMPLATE(False) elif (_tokenSet_4.member(self.LA(1))): pass self.matchNot('}') else: break self.match('}') self.set_return_token(_createToken, _token, _ttype, _begin) def mLBRACK(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LBRACK _saveIndex = 0 pass self.match('[') self.set_return_token(_createToken, _token, _ttype, _begin) def mRBRACK(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = RBRACK _saveIndex = 0 pass self.match(']') self.set_return_token(_createToken, _token, _ttype, _begin) def mLPAREN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LPAREN _saveIndex = 0 pass self.match('(') self.set_return_token(_createToken, _token, _ttype, _begin) def mRPAREN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = RPAREN _saveIndex = 0 pass self.match(')') self.set_return_token(_createToken, _token, _ttype, _begin) def mCOMMA(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COMMA _saveIndex = 0 pass self.match(',') self.set_return_token(_createToken, _token, _ttype, _begin) def mDOT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = DOT _saveIndex = 0 pass self.match('.') self.set_return_token(_createToken, _token, _ttype, _begin) def mASSIGN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ASSIGN _saveIndex = 0 pass self.match('=') self.set_return_token(_createToken, _token, _ttype, _begin) def mCOLON(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COLON _saveIndex = 0 pass self.match(':') self.set_return_token(_createToken, _token, _ttype, _begin) def mPLUS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = PLUS _saveIndex = 0 pass self.match('+') self.set_return_token(_createToken, _token, _ttype, _begin) def mSEMI(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SEMI _saveIndex = 0 pass self.match(';') self.set_return_token(_createToken, _token, _ttype, _begin) def mNOT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NOT _saveIndex = 0 pass self.match('!') self.set_return_token(_createToken, _token, _ttype, _begin) def mDOTDOTDOT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = DOTDOTDOT _saveIndex = 0 pass self.match("...") self.set_return_token(_createToken, _token, _ttype, _begin) def mWS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = WS _saveIndex = 0 pass la1 = self.LA(1) if False: pass elif la1 and la1 in u' ': pass self.match(' ') elif la1 and la1 in u'\t': pass self.match('\t') elif la1 and la1 in u'\n\r': pass if (self.LA(1)==u'\r') and (self.LA(2)==u'\n'): pass self.match('\r') self.match('\n') elif (self.LA(1)==u'\r') and (True): pass self.match('\r') elif (self.LA(1)==u'\n'): pass self.match('\n') else: self.raise_NoViableAlt(self.LA(1)) if not self.inputState.guessing: self.newline() else: self.raise_NoViableAlt(self.LA(1)) if not self.inputState.guessing: _ttype = SKIP self.set_return_token(_createToken, _token, _ttype, _begin) ### generate bit set def mk_tokenSet_0(): data = [0L] * 2048 ### init list data[0] =-17179869192L data[1] =-268435457L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): data = [0L] * 1025 ### init list data[0] =4294977024L data[1] =576460745995190270L return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): data = [0L] * 1025 ### init list data[0] =288107235144377856L data[1] =1729382250602037246L return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) ### generate bit set def mk_tokenSet_3(): data = [0L] * 1025 ### init list data[0] =4294977024L return data _tokenSet_3 = antlr.BitSet(mk_tokenSet_3()) ### generate bit set def mk_tokenSet_4(): data = [0L] * 2048 ### init list data[0] =-8L data[1] =-2882303761785552897L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_4 = antlr.BitSet(mk_tokenSet_4()) ### generate bit set def mk_tokenSet_5(): data = [0L] * 1025 ### init list data[0] =17596481021440L return data _tokenSet_5 = antlr.BitSet(mk_tokenSet_5()) ### generate bit set def mk_tokenSet_6(): data = [0L] * 1025 ### init list data[0] =17596481021440L data[1] =576460745995190270L return data _tokenSet_6 = antlr.BitSet(mk_tokenSet_6()) ### __main__ header action >>> if __name__ == '__main__' : import sys import antlr import ActionLexer ### create lexer - shall read from stdin try: for token in ActionLexer.Lexer(): print token except antlr.TokenStreamException, e: print "error: exception caught while lexing: ", e ### __main__ header action <<< stringtemplate3-3.1/stringtemplate3/language/StringTemplateAST.py0000444000175000001440000000071010756132624024160 0ustar pinkusers import antlr class StringTemplateAST(antlr.CommonAST): def __init__(self, type=None, text=None): super(StringTemplateAST, self).__init__() if type is not None: self.setType(type) if text is not None: self.setText(text) # track template for ANONYMOUS blocks self.st = None def getStringTemplate(self): return self.st def setStringTemplate(self, st): self.st = st stringtemplate3-3.1/stringtemplate3/language/AngleBracketTemplateLexer.py0000444000175000001440000012272710756132623025700 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "angle.bracket.template.g" -> "AngleBracketTemplateLexer.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> import stringtemplate3 import stringtemplate3.language.TemplateParser from stringtemplate3.language.ChunkToken import ChunkToken ### header action <<< ### preamble action >>> ### preamble action <<< ### >>>The Literals<<< literals = {} ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL = 4 NEWLINE = 5 ACTION = 6 IF = 7 ELSEIF = 8 ELSE = 9 ENDIF = 10 REGION_REF = 11 REGION_DEF = 12 NL = 13 EXPR = 14 TEMPLATE = 15 IF_EXPR = 16 ESC_CHAR = 17 ESC = 18 HEX = 19 SUBTEMPLATE = 20 NESTED_PARENS = 21 INDENT = 22 COMMENT = 23 ###/** Break up an input text stream into chunks of either plain text ### * or template actions in "<...>". Treat IF and ENDIF tokens ### * specially. ### */ class Lexer(antlr.CharScanner) : ### user action >>> def reportError(self, e): self.this.error("<...> chunk lexer error", e) def upcomingELSE(self, i): return self.LA(i) == '<' and \ self.LA(i+1) == 'e' and \ self.LA(i+2) == 'l' and \ self.LA(i+3) == 's' and \ self.LA(i+4) == 'e' and \ self.LA(i+5) == '>' def upcomingENDIF(self, i): return self.LA(i) == '<' and \ self.LA(i+1) == 'e' and \ self.LA(i+2) == 'n' and \ self.LA(i+3) == 'd' and \ self.LA(i+4) == 'i' and \ self.LA(i+5) == 'f' and \ self.LA(i+6) == '>' def upcomingAtEND(self, i): return self.LA(i) == '<' and \ self.LA(i+1) == '@' and \ self.LA(i+2) == 'e' and \ self.LA(i+3) == 'n' and \ self.LA(i+4) == 'd' and \ self.LA(i+5) == '>' def upcomingNewline(self, i): return (self.LA(i) == '\r' and self.LA(i+1) == '\n') or \ self.LA(i) == '\n' ### user action <<< def __init__(self, *argv, **kwargs) : antlr.CharScanner.__init__(self, *argv, **kwargs) self.caseSensitiveLiterals = True self.setCaseSensitive(True) self.literals = literals ### __init__ header action >>> self.currentIndent = None self.this = None ### __init__ header action <<< def nextToken(self): while True: try: ### try again .. while True: _token = None _ttype = INVALID_TYPE self.resetText() try: ## for char stream error handling try: ##for lexical error handling la1 = self.LA(1) if False: pass elif la1 and la1 in u'\n\r': pass self.mNEWLINE(True) theRetToken = self._returnToken elif la1 and la1 in u'<': pass self.mACTION(True) theRetToken = self._returnToken else: if ((_tokenSet_0.member(self.LA(1))) and ( self.LA(1) != '\r' and self.LA(1) != '\n' )): pass self.mLITERAL(True) theRetToken = self._returnToken else: self.default(self.LA(1)) if not self._returnToken: raise antlr.TryAgain ### found SKIP token ### option { testLiterals=true } self.testForLiteral(self._returnToken) ### return token to caller return self._returnToken ### handle lexical errors .... except antlr.RecognitionException, e: raise antlr.TokenStreamRecognitionException(e) ### handle char stream errors ... except antlr.CharStreamException,cse: if isinstance(cse, antlr.CharStreamIOException): raise antlr.TokenStreamIOException(cse.io) else: raise antlr.TokenStreamException(str(cse)) except antlr.TryAgain: pass def mLITERAL(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LITERAL _saveIndex = 0 ind = None if not self.LA(1) != '\r' and self.LA(1) != '\n' : raise antlr.SemanticException(" self.LA(1) != '\\r' and self.LA(1) != '\\n' ") pass _cnt5= 0 while True: loopStartIndex = self.text.length() col = self.getColumn() if (self.LA(1)==u'\\') and (self.LA(2)==u'<'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('<') elif (self.LA(1)==u'\\') and (self.LA(2)==u'>') and (True) and (True) and (True) and (True) and (True): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('>') elif (self.LA(1)==u'\\') and (self.LA(2)==u'\\') and (True) and (True) and (True) and (True) and (True): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('\\') elif (self.LA(1)==u'\\') and (_tokenSet_1.member(self.LA(2))) and (True) and (True) and (True) and (True) and (True): pass self.match('\\') self.match(_tokenSet_1) elif (self.LA(1)==u'\t' or self.LA(1)==u' ') and (True) and (True) and (True) and (True) and (True) and (True): pass self.mINDENT(True) ind = self._returnToken if col == 1 and self.LA(1) == '<': # store indent in ASTExpr not in a literal self.currentIndent = ind.getText() # reset length to wack text self.text.setLength(loopStartIndex) else: self.currentIndent = None elif (_tokenSet_0.member(self.LA(1))) and (True) and (True) and (True) and (True) and (True) and (True): pass self.match(_tokenSet_0) else: break _cnt5 += 1 if _cnt5 < 1: self.raise_NoViableAlt(self.LA(1)) if not len(self.text.getString(_begin)): _ttype = SKIP self.set_return_token(_createToken, _token, _ttype, _begin) def mINDENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = INDENT _saveIndex = 0 pass _cnt8= 0 while True: if (self.LA(1)==u' ') and (True) and (True) and (True) and (True) and (True) and (True): pass self.match(' ') elif (self.LA(1)==u'\t') and (True) and (True) and (True) and (True) and (True) and (True): pass self.match('\t') else: break _cnt8 += 1 if _cnt8 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mNEWLINE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NEWLINE _saveIndex = 0 pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() self.currentIndent = None self.set_return_token(_createToken, _token, _ttype, _begin) def mACTION(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ACTION _saveIndex = 0 startCol = self.getColumn() if (self.LA(1)==u'<') and (self.LA(2)==u'\\') and (_tokenSet_2.member(self.LA(3))) and (_tokenSet_3.member(self.LA(4))) and (True) and (True) and (True): pass buf = u"" uc = u"\000" _saveIndex = self.text.length() self.match('<') self.text.setLength(_saveIndex) _cnt13= 0 while True: if (self.LA(1)==u'\\'): pass uc=self.mESC_CHAR(False) buf += uc else: break _cnt13 += 1 if _cnt13 < 1: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) self.text.setLength(_begin) ; self.text.append(buf) _ttype = LITERAL elif (self.LA(1)==u'<') and (self.LA(2)==u'!') and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass self.mCOMMENT(False) _ttype = SKIP elif (self.LA(1)==u'<') and (_tokenSet_4.member(self.LA(2))) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass if (self.LA(1)==u'<') and (self.LA(2)==u'i') and (self.LA(3)==u'f') and (self.LA(4)==u' ' or self.LA(4)==u'(') and (_tokenSet_5.member(self.LA(5))) and ((self.LA(6) >= u'\u0001' and self.LA(6) <= u'\ufffe')) and ((self.LA(7) >= u'\u0001' and self.LA(7) <= u'\ufffe')): pass _saveIndex = self.text.length() self.match('<') self.text.setLength(_saveIndex) self.match("if") while True: if (self.LA(1)==u' '): pass _saveIndex = self.text.length() self.match(' ') self.text.setLength(_saveIndex) else: break self.match("(") self.mIF_EXPR(False) self.match(")") _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) _ttype = IF if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'<') and (self.LA(2)==u'e') and (self.LA(3)==u'n') and (self.LA(4)==u'd') and (self.LA(5)==u'i') and (self.LA(6)==u'f') and (self.LA(7)==u'>'): pass _saveIndex = self.text.length() self.match('<') self.text.setLength(_saveIndex) self.match("endif") _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) _ttype = ENDIF if ((self.LA(1)==u'\n' or self.LA(1)==u'\r') and (startCol==1)): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'<') and (self.LA(2)==u'e') and (self.LA(3)==u'l') and (self.LA(4)==u's') and (self.LA(5)==u'e') and (self.LA(6)==u'>') and (True): pass _saveIndex = self.text.length() self.match('<') self.text.setLength(_saveIndex) self.match("else") _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) _ttype = ELSE if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'<') and (self.LA(2)==u'@') and (_tokenSet_6.member(self.LA(3))) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and ((self.LA(5) >= u'\u0001' and self.LA(5) <= u'\ufffe')) and ((self.LA(6) >= u'\u0001' and self.LA(6) <= u'\ufffe')) and (True): pass _saveIndex = self.text.length() self.match('<') self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('@') self.text.setLength(_saveIndex) _cnt22= 0 while True: if (_tokenSet_6.member(self.LA(1))): pass self.match(_tokenSet_6) else: break _cnt22 += 1 if _cnt22 < 1: self.raise_NoViableAlt(self.LA(1)) la1 = self.LA(1) if False: pass elif la1 and la1 in u'(': pass _saveIndex = self.text.length() self.match("()") self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) _ttype = REGION_REF elif la1 and la1 in u'>': pass _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) _ttype = REGION_DEF t = self.text.getString(_begin) self.text.setLength(_begin) ; self.text.append(t+"::=") if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) atLeft = False _cnt29= 0 while True: if (((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True) and (not (self.upcomingAtEND(1) or (self.upcomingNewline(1) and self.upcomingAtEND(2))))): pass if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() atLeft = True elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass self.matchNot(antlr.EOF_CHAR) atLeft = False else: self.raise_NoViableAlt(self.LA(1)) else: break _cnt29 += 1 if _cnt29 < 1: self.raise_NoViableAlt(self.LA(1)) if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() atLeft = True elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) if (self.LA(1)==u'<') and (self.LA(2)==u'@'): pass _saveIndex = self.text.length() self.match("<@end>") self.text.setLength(_saveIndex) elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and (True): pass self.matchNot(antlr.EOF_CHAR) self.this.error("missing region "+t+" <@end> tag") else: self.raise_NoViableAlt(self.LA(1)) if ((self.LA(1)==u'\n' or self.LA(1)==u'\r') and (atLeft)): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() else: ## pass else: self.raise_NoViableAlt(self.LA(1)) elif (self.LA(1)==u'<') and (_tokenSet_4.member(self.LA(2))) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass _saveIndex = self.text.length() self.match('<') self.text.setLength(_saveIndex) self.mEXPR(False) _saveIndex = self.text.length() self.match('>') self.text.setLength(_saveIndex) else: self.raise_NoViableAlt(self.LA(1)) t = ChunkToken(_ttype, self.text.getString(_begin), self.currentIndent) _token = t else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mESC_CHAR(self, _createToken): uc='\u0000' _ttype = 0 _token = None _begin = self.text.length() _ttype = ESC_CHAR _saveIndex = 0 a = None b = None c = None d = None if (self.LA(1)==u'\\') and (self.LA(2)==u'n'): pass _saveIndex = self.text.length() self.match("\\n") self.text.setLength(_saveIndex) uc = '\n' elif (self.LA(1)==u'\\') and (self.LA(2)==u'r'): pass _saveIndex = self.text.length() self.match("\\r") self.text.setLength(_saveIndex) uc = '\r' elif (self.LA(1)==u'\\') and (self.LA(2)==u't'): pass _saveIndex = self.text.length() self.match("\\t") self.text.setLength(_saveIndex) uc = '\t' elif (self.LA(1)==u'\\') and (self.LA(2)==u' '): pass _saveIndex = self.text.length() self.match("\\ ") self.text.setLength(_saveIndex) uc = ' ' elif (self.LA(1)==u'\\') and (self.LA(2)==u'u'): pass _saveIndex = self.text.length() self.match("\\u") self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) a = self._returnToken _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) b = self._returnToken _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) c = self._returnToken _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) d = self._returnToken uc = unichr(int(a.getText()+b.getText()+c.getText()+d.getText(), 16)) else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) return uc def mCOMMENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COMMENT _saveIndex = 0 startCol = self.getColumn() pass self.match("') and (True) and (True) and (True) and (True) and (True)): break if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass self.mNL(False) self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass self.matchNot(antlr.EOF_CHAR) else: break self.match("!>") if ((self.LA(1)==u'\n' or self.LA(1)==u'\r') and ( startCol == 1 )): pass self.mNL(False) self.newline() else: ## pass self.set_return_token(_createToken, _token, _ttype, _begin) def mIF_EXPR(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = IF_EXPR _saveIndex = 0 pass _cnt54= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'\\': pass self.mESC(False) elif la1 and la1 in u'\n\r': pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() elif la1 and la1 in u'{': pass self.mSUBTEMPLATE(False) elif la1 and la1 in u'(': pass self.mNESTED_PARENS(False) else: if (_tokenSet_7.member(self.LA(1))): pass self.matchNot(')') else: break _cnt54 += 1 if _cnt54 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mNL(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NL _saveIndex = 0 if (self.LA(1)==u'\r') and (self.LA(2)==u'\n') and (True) and (True) and (True) and (True) and (True): pass self.match('\r') self.match('\n') elif (self.LA(1)==u'\r') and (True) and (True) and (True) and (True) and (True) and (True): pass self.match('\r') elif (self.LA(1)==u'\n'): pass self.match('\n') else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mEXPR(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = EXPR _saveIndex = 0 pass _cnt42= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'\\': pass self.mESC(False) elif la1 and la1 in u'\n\r': pass self.mNL(False) self.newline() elif la1 and la1 in u'{': pass self.mSUBTEMPLATE(False) else: if (self.LA(1)==u'+' or self.LA(1)==u'=') and (self.LA(2)==u'"' or self.LA(2)==u'<'): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'=': pass self.match('=') elif la1 and la1 in u'+': pass self.match('+') else: self.raise_NoViableAlt(self.LA(1)) self.mTEMPLATE(False) elif (self.LA(1)==u'+' or self.LA(1)==u'=') and (self.LA(2)==u'{'): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'=': pass self.match('=') elif la1 and la1 in u'+': pass self.match('+') else: self.raise_NoViableAlt(self.LA(1)) self.mSUBTEMPLATE(False) elif (self.LA(1)==u'+' or self.LA(1)==u'=') and (_tokenSet_8.member(self.LA(2))): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'=': pass self.match('=') elif la1 and la1 in u'+': pass self.match('+') else: self.raise_NoViableAlt(self.LA(1)) self.match(_tokenSet_8) elif (_tokenSet_9.member(self.LA(1))): pass self.matchNot('>') else: break _cnt42 += 1 if _cnt42 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mESC(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ESC _saveIndex = 0 pass self.match('\\') self.matchNot(antlr.EOF_CHAR) self.set_return_token(_createToken, _token, _ttype, _begin) def mSUBTEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SUBTEMPLATE _saveIndex = 0 pass self.match('{') while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'{': pass self.mSUBTEMPLATE(False) elif la1 and la1 in u'\\': pass self.mESC(False) else: if (_tokenSet_10.member(self.LA(1))): pass self.matchNot('}') else: break self.match('}') self.set_return_token(_createToken, _token, _ttype, _begin) def mTEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = TEMPLATE _saveIndex = 0 la1 = self.LA(1) if False: pass elif la1 and la1 in u'"': pass self.match('"') while True: if (self.LA(1)==u'\\'): pass self.mESC(False) elif (_tokenSet_11.member(self.LA(1))): pass self.matchNot('"') else: break self.match('"') elif la1 and la1 in u'<': pass self.match("<<") if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) while True: ### nongreedy exit test if ((self.LA(1)==u'>') and (self.LA(2)==u'>') and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True)): break if ((self.LA(1)==u'\r') and (self.LA(2)==u'\n') and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and ((self.LA(5) >= u'\u0001' and self.LA(5) <= u'\ufffe')) and (True) and (True) and ( self.LA(3) == '>' and self.LA(4) == '>' )): pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1)==u'\n') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True) and ( self.LA(2) == '>' and self.LA(3) == '>' )): pass _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass self.matchNot(antlr.EOF_CHAR) else: break self.match(">>") else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mNESTED_PARENS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NESTED_PARENS _saveIndex = 0 pass self.match('(') _cnt63= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'(': pass self.mNESTED_PARENS(False) elif la1 and la1 in u'\\': pass self.mESC(False) else: if (_tokenSet_12.member(self.LA(1))): pass self.matchNot(')') else: break _cnt63 += 1 if _cnt63 < 1: self.raise_NoViableAlt(self.LA(1)) self.match(')') self.set_return_token(_createToken, _token, _ttype, _begin) def mHEX(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = HEX _saveIndex = 0 la1 = self.LA(1) if False: pass elif la1 and la1 in u'0123456789': pass self.matchRange(u'0', u'9') elif la1 and la1 in u'ABCDEF': pass self.matchRange(u'A', u'F') elif la1 and la1 in u'abcdef': pass self.matchRange(u'a', u'f') else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) ### generate bit set def mk_tokenSet_0(): data = [0L] * 2048 ### init list data[0] =-1152921504606856194L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): data = [0L] * 2048 ### init list data[0] =-5764607523034234882L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): data = [0L] * 1025 ### init list data[0] =4294967296L data[1] =14707067533131776L return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) ### generate bit set def mk_tokenSet_3(): data = [0L] * 1025 ### init list data[0] =4899634919602388992L data[1] =541434314878L return data _tokenSet_3 = antlr.BitSet(mk_tokenSet_3()) ### generate bit set def mk_tokenSet_4(): data = [0L] * 2048 ### init list data[0] =-4611686018427387906L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_4 = antlr.BitSet(mk_tokenSet_4()) ### generate bit set def mk_tokenSet_5(): data = [0L] * 2048 ### init list data[0] =-2199023255554L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_5 = antlr.BitSet(mk_tokenSet_5()) ### generate bit set def mk_tokenSet_6(): data = [0L] * 2048 ### init list data[0] =-4611687117939015682L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_6 = antlr.BitSet(mk_tokenSet_6()) ### generate bit set def mk_tokenSet_7(): data = [0L] * 2048 ### init list data[0] =-3298534892546L data[1] =-576460752571858945L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_7 = antlr.BitSet(mk_tokenSet_7()) ### generate bit set def mk_tokenSet_8(): data = [0L] * 2048 ### init list data[0] =-1152921521786716162L data[1] =-576460752303423489L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_8 = antlr.BitSet(mk_tokenSet_8()) ### generate bit set def mk_tokenSet_9(): data = [0L] * 2048 ### init list data[0] =-6917537823734113282L data[1] =-576460752571858945L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_9 = antlr.BitSet(mk_tokenSet_9()) ### generate bit set def mk_tokenSet_10(): data = [0L] * 2048 ### init list data[0] =-2L data[1] =-2882303761785552897L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_10 = antlr.BitSet(mk_tokenSet_10()) ### generate bit set def mk_tokenSet_11(): data = [0L] * 2048 ### init list data[0] =-17179869186L data[1] =-268435457L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_11 = antlr.BitSet(mk_tokenSet_11()) ### generate bit set def mk_tokenSet_12(): data = [0L] * 2048 ### init list data[0] =-3298534883330L data[1] =-268435457L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_12 = antlr.BitSet(mk_tokenSet_12()) ### __main__ header action >>> if __name__ == '__main__' : import sys import antlr import AngleBracketTemplateLexer ### create lexer - shall read from stdin try: for token in AngleBracketTemplateLexer.Lexer(): print token except antlr.TokenStreamException, e: print "error: exception caught while lexing: ", e ### __main__ header action <<< stringtemplate3-3.1/stringtemplate3/language/TemplateParser.py0000444000175000001440000002317010756132624023603 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "template.g" -> "TemplateParser.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> import stringtemplate3 from stringtemplate3.language.ChunkToken import ChunkToken from stringtemplate3.language.StringRef import StringRef from stringtemplate3.language.NewlineRef import NewlineRef ### header action <<< ### preamble action>>> ### preamble action <<< ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL = 4 NEWLINE = 5 ACTION = 6 IF = 7 ELSEIF = 8 ELSE = 9 ENDIF = 10 REGION_REF = 11 REGION_DEF = 12 NL = 13 EXPR = 14 TEMPLATE = 15 IF_EXPR = 16 ESC_CHAR = 17 ESC = 18 HEX = 19 SUBTEMPLATE = 20 NESTED_PARENS = 21 INDENT = 22 COMMENT = 23 ###/** A parser used to break up a single template into chunks, text literals ### * and attribute expressions. ### */ class Parser(antlr.LLkParser): ### user action >>> def reportError(self, e): group = self.this.group if group == stringtemplate3.StringTemplate.defaultGroup: self.this.error("template parse error; template context is "+self.this.enclosingInstanceStackString, e) else: self.this.error("template parse error in group "+self.this.group.name+" line "+str(self.this.groupFileLine)+"; template context is "+self.this.enclosingInstanceStackString, e) ### user action <<< def __init__(self, *args, **kwargs): antlr.LLkParser.__init__(self, *args, **kwargs) self.tokenNames = _tokenNames ### __init__ header action >>> self.this = None ### __init__ header action <<< def template(self, this ): s = None nl = None try: ## for error handling pass while True: la1 = self.LA(1) if False: pass elif la1 and la1 in [LITERAL]: pass s = self.LT(1) self.match(LITERAL) this.addChunk(StringRef(this,s.getText())) elif la1 and la1 in [NEWLINE]: pass nl = self.LT(1) self.match(NEWLINE) if self.LA(1) != ELSE and self.LA(1) != ENDIF: this.addChunk(NewlineRef(this,nl.getText())) elif la1 and la1 in [ACTION,IF,REGION_REF,REGION_DEF]: pass self.action(this) else: break except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_0) def action(self, this ): a = None i = None ei = None rr = None rd = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [ACTION]: pass a = self.LT(1) self.match(ACTION) indent = a.indentation c = this.parseAction(a.getText()) c.indentation = indent this.addChunk(c) elif la1 and la1 in [IF]: pass i = self.LT(1) self.match(IF) c = this.parseAction(i.getText()) # create and precompile the subtemplate subtemplate = stringtemplate3.StringTemplate(group=this.group) subtemplate.enclosingInstance = this subtemplate.name = i.getText() + "_subtemplate" this.addChunk(c) self.template(subtemplate) if c: c.subtemplate = subtemplate while True: if (self.LA(1)==ELSEIF): pass ei = self.LT(1) self.match(ELSEIF) ec = this.parseAction(ei.getText()) # create and precompile the subtemplate elseIfSubtemplate = stringtemplate3.StringTemplate(group=this.group) elseIfSubtemplate.enclosingInstance = this elseIfSubtemplate.name = ei.getText()+"_subtemplate" self.template(elseIfSubtemplate) if c is not None: c.addElseIfSubtemplate(ec, elseIfSubtemplate) else: break la1 = self.LA(1) if False: pass elif la1 and la1 in [ELSE]: pass self.match(ELSE) # create and precompile the subtemplate elseSubtemplate = stringtemplate3.StringTemplate(group=this.group) elseSubtemplate.enclosingInstance = this elseSubtemplate.name = "else_subtemplate" self.template(elseSubtemplate) if c: c.elseSubtemplate = elseSubtemplate elif la1 and la1 in [ENDIF]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) self.match(ENDIF) elif la1 and la1 in [REGION_REF]: pass rr = self.LT(1) self.match(REGION_REF) # define implicit template and # convert <@r()> to regionName = rr.getText() mangledRef = None err = False # watch out for <@super.r()>; that does NOT def implicit region # convert to if regionName.startswith("super."): #System.out.println("super region ref "+regionName); regionRef = regionName[len("super."):len(regionName)] templateScope = this.group.getUnMangledTemplateName(this.name) scopeST = this.group.lookupTemplate(templateScope) if scopeST is None: this.group.error("reference to region within undefined template: "+ templateScope) err = True if not scopeST.containsRegionName(regionRef): this.group.error("template "+templateScope+" has no region called "+ regionRef) err = True else: mangledRef = this.group.getMangledRegionName(templateScope, regionRef) mangledRef = "super." + mangledRef else: regionST = this.group.defineImplicitRegionTemplate(this, regionName) mangledRef = regionST.name if not err: # treat as regular action: mangled template include indent = rr.indentation c = this.parseAction(mangledRef+"()") c.indentation = indent this.addChunk(c) elif la1 and la1 in [REGION_DEF]: pass rd = self.LT(1) self.match(REGION_DEF) combinedNameTemplateStr = rd.getText() indexOfDefSymbol = combinedNameTemplateStr.find("::=") if indexOfDefSymbol >= 1: regionName = combinedNameTemplateStr[0:indexOfDefSymbol] template = combinedNameTemplateStr[indexOfDefSymbol+3:len(combinedNameTemplateStr)] regionST = this.group.defineRegionTemplate( this, regionName, template, stringtemplate3.REGION_EMBEDDED ) # treat as regular action: mangled template include indent = rd.indentation c = this.parseAction(regionST.name + "()") c.indentation = indent this.addChunk(c) else: this.error("embedded region definition screwed up") else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_1) _tokenNames = [ "<0>", "EOF", "<2>", "NULL_TREE_LOOKAHEAD", "LITERAL", "NEWLINE", "ACTION", "IF", "ELSEIF", "ELSE", "ENDIF", "REGION_REF", "REGION_DEF", "NL", "EXPR", "TEMPLATE", "IF_EXPR", "ESC_CHAR", "ESC", "HEX", "SUBTEMPLATE", "NESTED_PARENS", "INDENT", "COMMENT" ] ### generate bit set def mk_tokenSet_0(): ### var1 data = [ 1792L, 0L] return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): ### var1 data = [ 8176L, 0L] return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) stringtemplate3-3.1/stringtemplate3/language/StringRef.py0000444000175000001440000000146710756132624022563 0ustar pinkusers from stringtemplate3.language.Expr import Expr class StringRef(Expr): """ Represents a chunk of just simple text to spit out; nothing to "evaluate" """ def __init__(self, enclosingTemplate, text): super(StringRef, self).__init__(enclosingTemplate) self.str = text def write(self, this, out): """ Just print out the string; no reference to self because this is a literal -- not sensitive to attribute values. These strings never wrap because they are not part of an <...> expression. <"foo"; wrap="\n"> should wrap though if necessary. """ if self.str is not None: return out.write(self.str) return 0 def __str__(self): if self.str: return self.str return '' stringtemplate3-3.1/stringtemplate3/language/NewlineRef.py0000444000175000001440000000055610756132624022714 0ustar pinkusers from stringtemplate3.language.StringRef import StringRef class NewlineRef(StringRef): """ Represents a newline. Separated so I can do smart things like not spitting out newlines when the only thing on a line is an attr expr. """ def __init__(self, enclosingTemplate, str): super(NewlineRef, self).__init__(enclosingTemplate, str) stringtemplate3-3.1/stringtemplate3/language/Expr.py0000444000175000001440000000255610756132624021576 0ustar pinkusersfrom stringtemplate3.utils import deprecated class Expr(object): """ A string template expression embedded within the template. A template is parsed into a tokenized vector of Expr objects and then executed after the user sticks in attribute values. This list of Expr objects represents a "program" for the StringTemplate evaluator. """ def __init__(self, enclosingTemplate): ## The StringTemplate object surrounding this expr self.enclosingTemplate = enclosingTemplate ## Anything spit out as a chunk (even plain text) must be indented # according to whitespace before the action that generated it. So, # plain text in the outermost template is never indented, but the # text and attribute references in a nested template will all be # indented by the amount seen directly in front of the attribute # reference that initiates construction of the nested template. self.indentation = '' def write(self, this, out): """How to write this node to output""" raise NotImplementedError @deprecated def getEnclosingTemplate(self): return self.enclosingTemplate @deprecated def getIndentation(self): return self.indentation @deprecated def setIndentation(self, indentation): self.indentation = indentation stringtemplate3-3.1/stringtemplate3/language/DefaultTemplateLexer.py0000444000175000001440000012471710756132624024744 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "template.g" -> "DefaultTemplateLexer.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> import stringtemplate3 from stringtemplate3.language.ChunkToken import ChunkToken from stringtemplate3.language.StringRef import StringRef from stringtemplate3.language.NewlineRef import NewlineRef ### header action <<< ### preamble action >>> ### preamble action <<< ### >>>The Literals<<< literals = {} ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL = 4 NEWLINE = 5 ACTION = 6 IF = 7 ELSEIF = 8 ELSE = 9 ENDIF = 10 REGION_REF = 11 REGION_DEF = 12 NL = 13 EXPR = 14 TEMPLATE = 15 IF_EXPR = 16 ESC_CHAR = 17 ESC = 18 HEX = 19 SUBTEMPLATE = 20 NESTED_PARENS = 21 INDENT = 22 COMMENT = 23 ###/** Break up an input text stream into chunks of either plain text ### * or template actions in "$...$". Treat IF and ENDIF tokens ### * specially. ### */ class Lexer(antlr.CharScanner) : ### user action >>> def reportError(self, e): self.this.error("$...$ chunk lexer error", e) def upcomingELSE(self, i): return self.LA(i) == '$' and \ self.LA(i+1) == 'e' and \ self.LA(i+2) == 'l' and \ self.LA(i+3) == 's' and \ self.LA(i+4) == 'e' and \ self.LA(i+5) == '$' def upcomingENDIF(self, i): return self.LA(i) == '$' and \ self.LA(i+1) == 'e' and \ self.LA(i+2) == 'n' and \ self.LA(i+3) == 'd' and \ self.LA(i+4) == 'i' and \ self.LA(i+5) == 'f' and \ self.LA(i+6) == '$' def upcomingAtEND(self, i): return self.LA(i) == '$' and \ self.LA(i+1) == '@' and \ self.LA(i+2) == 'e' and \ self.LA(i+3) == 'n' and \ self.LA(i+4) == 'd' and \ self.LA(i+5) == '$' def upcomingNewline(self, i): return (self.LA(i) == '\r' and self.LA(i+1) == '\n') or \ self.LA(i) == '\n' ### user action <<< def __init__(self, *argv, **kwargs) : antlr.CharScanner.__init__(self, *argv, **kwargs) self.caseSensitiveLiterals = True self.setCaseSensitive(True) self.literals = literals ### __init__ header action >>> self.currentIndent = None self.this = None ### __init__ header action <<< def nextToken(self): while True: try: ### try again .. while True: _token = None _ttype = INVALID_TYPE self.resetText() try: ## for char stream error handling try: ##for lexical error handling la1 = self.LA(1) if False: pass elif la1 and la1 in u'\n\r': pass self.mNEWLINE(True) theRetToken = self._returnToken elif la1 and la1 in u'$': pass self.mACTION(True) theRetToken = self._returnToken else: if ((_tokenSet_0.member(self.LA(1))) and ( self.LA(1) != '\r' and self.LA(1) != '\n' )): pass self.mLITERAL(True) theRetToken = self._returnToken else: self.default(self.LA(1)) if not self._returnToken: raise antlr.TryAgain ### found SKIP token ### option { testLiterals=true } self.testForLiteral(self._returnToken) ### return token to caller return self._returnToken ### handle lexical errors .... except antlr.RecognitionException, e: raise antlr.TokenStreamRecognitionException(e) ### handle char stream errors ... except antlr.CharStreamException,cse: if isinstance(cse, antlr.CharStreamIOException): raise antlr.TokenStreamIOException(cse.io) else: raise antlr.TokenStreamException(str(cse)) except antlr.TryAgain: pass def mLITERAL(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LITERAL _saveIndex = 0 ind = None if not self.LA(1) != '\r' and self.LA(1) != '\n' : raise antlr.SemanticException(" self.LA(1) != '\\r' and self.LA(1) != '\\n' ") pass _cnt11= 0 while True: loopStartIndex = self.text.length() col = self.getColumn() if (self.LA(1)==u'\\') and (self.LA(2)==u'$'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('$') elif (self.LA(1)==u'\\') and (self.LA(2)==u'\\') and (True) and (True) and (True) and (True) and (True): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('\\') elif (self.LA(1)==u'\\') and (_tokenSet_1.member(self.LA(2))) and (True) and (True) and (True) and (True) and (True): pass self.match('\\') self.matchNot('$') elif (self.LA(1)==u'\t' or self.LA(1)==u' ') and (True) and (True) and (True) and (True) and (True) and (True): pass self.mINDENT(True) ind = self._returnToken if col == 1 and self.LA(1) == '$': # store indent in ASTExpr not in a literal self.currentIndent = ind.getText() # reset length to wack text self.text.setLength(loopStartIndex) else: self.currentIndent = None elif (_tokenSet_0.member(self.LA(1))) and (True) and (True) and (True) and (True) and (True) and (True): pass self.match(_tokenSet_0) else: break _cnt11 += 1 if _cnt11 < 1: self.raise_NoViableAlt(self.LA(1)) if not len(self.text.getString(_begin)): _ttype = SKIP # pure indent? self.set_return_token(_createToken, _token, _ttype, _begin) def mINDENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = INDENT _saveIndex = 0 pass _cnt72= 0 while True: if (self.LA(1)==u' ') and (True) and (True) and (True) and (True) and (True) and (True): pass self.match(' ') elif (self.LA(1)==u'\t') and (True) and (True) and (True) and (True) and (True) and (True): pass self.match('\t') else: break _cnt72 += 1 if _cnt72 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mNEWLINE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NEWLINE _saveIndex = 0 pass self.mNL(False) self.newline() self.currentIndent = None self.set_return_token(_createToken, _token, _ttype, _begin) def mNL(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NL _saveIndex = 0 if (self.LA(1)==u'\r') and (self.LA(2)==u'\n') and (True) and (True) and (True) and (True) and (True): pass self.match('\r') self.match('\n') elif (self.LA(1)==u'\r') and (True) and (True) and (True) and (True) and (True) and (True): pass self.match('\r') elif (self.LA(1)==u'\n'): pass self.match('\n') else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mACTION(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ACTION _saveIndex = 0 startCol = self.getColumn() if (self.LA(1)==u'$') and (self.LA(2)==u'\\') and (_tokenSet_2.member(self.LA(3))) and (_tokenSet_3.member(self.LA(4))) and (True) and (True) and (True): pass buf = u"" uc = u"\000" _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _cnt15= 0 while True: if (self.LA(1)==u'\\'): pass uc=self.mESC_CHAR(False) buf += uc else: break _cnt15 += 1 if _cnt15 < 1: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) self.text.setLength(_begin) ; self.text.append(buf) _ttype = LITERAL elif (self.LA(1)==u'$') and (self.LA(2)==u'!') and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass self.mCOMMENT(False) _ttype = SKIP elif (self.LA(1)==u'$') and (_tokenSet_1.member(self.LA(2))) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass if (self.LA(1)==u'$') and (self.LA(2)==u'i') and (self.LA(3)==u'f') and (self.LA(4)==u' ' or self.LA(4)==u'(') and (_tokenSet_4.member(self.LA(5))) and ((self.LA(6) >= u'\u0001' and self.LA(6) <= u'\ufffe')) and ((self.LA(7) >= u'\u0001' and self.LA(7) <= u'\ufffe')): pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) self.match("if") while True: if (self.LA(1)==u' '): pass _saveIndex = self.text.length() self.match(' ') self.text.setLength(_saveIndex) else: break self.match("(") self.mIF_EXPR(False) self.match(")") _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _ttype = IF if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'$') and (self.LA(2)==u'e') and (self.LA(3)==u'l') and (self.LA(4)==u's') and (self.LA(5)==u'e') and (self.LA(6)==u'i') and (self.LA(7)==u'f'): pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) self.match("elseif") while True: if (self.LA(1)==u' '): pass _saveIndex = self.text.length() self.match(' ') self.text.setLength(_saveIndex) else: break self.match("(") self.mIF_EXPR(False) self.match(")") _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _ttype = ELSEIF if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'$') and (self.LA(2)==u'e') and (self.LA(3)==u'n') and (self.LA(4)==u'd') and (self.LA(5)==u'i') and (self.LA(6)==u'f') and (self.LA(7)==u'$'): pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) self.match("endif") _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _ttype = ENDIF if ((self.LA(1)==u'\n' or self.LA(1)==u'\r') and ( startCol == 1 )): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'$') and (self.LA(2)==u'e') and (self.LA(3)==u'l') and (self.LA(4)==u's') and (self.LA(5)==u'e') and (self.LA(6)==u'$') and (True): pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) self.match("else") _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _ttype = ELSE if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() else: ## pass elif (self.LA(1)==u'$') and (self.LA(2)==u'@') and (_tokenSet_5.member(self.LA(3))) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and ((self.LA(5) >= u'\u0001' and self.LA(5) <= u'\ufffe')) and ((self.LA(6) >= u'\u0001' and self.LA(6) <= u'\ufffe')) and (True): pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('@') self.text.setLength(_saveIndex) _cnt28= 0 while True: if (_tokenSet_5.member(self.LA(1))): pass self.match(_tokenSet_5) else: break _cnt28 += 1 if _cnt28 < 1: self.raise_NoViableAlt(self.LA(1)) la1 = self.LA(1) if False: pass elif la1 and la1 in u'(': pass _saveIndex = self.text.length() self.match("()") self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _ttype = REGION_REF elif la1 and la1 in u'$': pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) _ttype = REGION_DEF t = self.text.getString(_begin) self.text.setLength(_begin) ; self.text.append(t+"::=") if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) newline(); elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) atLeft = False _cnt35= 0 while True: if (((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True) and (not (self.upcomingAtEND(1) or (self.upcomingNewline(1) and self.upcomingAtEND(2))))): pass if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() atLeft = Frue elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass self.matchNot(antlr.EOF_CHAR) atLeft = False else: self.raise_NoViableAlt(self.LA(1)) else: break _cnt35 += 1 if _cnt35 < 1: self.raise_NoViableAlt(self.LA(1)) if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline()() atLeft = True elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and (True) and (True) and (True) and (True) and (True) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) if (self.LA(1)==u'$') and (self.LA(2)==u'@'): pass _saveIndex = self.text.length() self.match("$@end$") self.text.setLength(_saveIndex) elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and (True): pass self.matchNot(antlr.EOF_CHAR) self.this.error("missing region "+t+" $@end$ tag") else: self.raise_NoViableAlt(self.LA(1)) if ((self.LA(1)==u'\n' or self.LA(1)==u'\r') and (atLeft)): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() else: ## pass else: self.raise_NoViableAlt(self.LA(1)) elif (self.LA(1)==u'$') and (_tokenSet_1.member(self.LA(2))) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) self.mEXPR(False) _saveIndex = self.text.length() self.match('$') self.text.setLength(_saveIndex) else: self.raise_NoViableAlt(self.LA(1)) t = ChunkToken(_ttype, self.text.getString(_begin), self.currentIndent) _token = t; else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mESC_CHAR(self, _createToken): uc='\u0000' _ttype = 0 _token = None _begin = self.text.length() _ttype = ESC_CHAR _saveIndex = 0 a = None b = None c = None d = None if (self.LA(1)==u'\\') and (self.LA(2)==u'n'): pass _saveIndex = self.text.length() self.match("\\n") self.text.setLength(_saveIndex) uc = '\n' elif (self.LA(1)==u'\\') and (self.LA(2)==u'r'): pass _saveIndex = self.text.length() self.match("\\r") self.text.setLength(_saveIndex) uc = '\r' elif (self.LA(1)==u'\\') and (self.LA(2)==u't'): pass _saveIndex = self.text.length() self.match("\\t") self.text.setLength(_saveIndex) uc = '\t' elif (self.LA(1)==u'\\') and (self.LA(2)==u' '): pass _saveIndex = self.text.length() self.match("\\ ") self.text.setLength(_saveIndex) uc = ' ' elif (self.LA(1)==u'\\') and (self.LA(2)==u'u'): pass _saveIndex = self.text.length() self.match("\\u") self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) a = self._returnToken _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) b = self._returnToken _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) c = self._returnToken _saveIndex = self.text.length() self.mHEX(True) self.text.setLength(_saveIndex) d = self._returnToken uc = unichr(int(a.getText()+b.getText()+c.getText()+d.getText(), 16)) else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) return uc def mCOMMENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COMMENT _saveIndex = 0 startCol = self.getColumn() pass self.match("$!") while True: ### nongreedy exit test if ((self.LA(1)==u'!') and (self.LA(2)==u'$') and (True) and (True) and (True) and (True) and (True)): break if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass self.mNL(False) self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass self.matchNot(antlr.EOF_CHAR) else: break self.match("!$") if ((self.LA(1)==u'\n' or self.LA(1)==u'\r') and ( startCol == 1 )): pass self.mNL(False) self.newline() else: ## pass self.set_return_token(_createToken, _token, _ttype, _begin) def mIF_EXPR(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = IF_EXPR _saveIndex = 0 pass _cnt60= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'\\': pass self.mESC(False) elif la1 and la1 in u'\n\r': pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() elif la1 and la1 in u'{': pass self.mSUBTEMPLATE(False) elif la1 and la1 in u'(': pass self.mNESTED_PARENS(False) else: if (_tokenSet_6.member(self.LA(1))): pass self.matchNot(')') else: break _cnt60 += 1 if _cnt60 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mEXPR(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = EXPR _saveIndex = 0 pass _cnt48= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'\\': pass self.mESC(False) elif la1 and la1 in u'\n\r': pass self.mNL(False) self.newline() elif la1 and la1 in u'{': pass self.mSUBTEMPLATE(False) else: if (self.LA(1)==u'+' or self.LA(1)==u'=') and (self.LA(2)==u'"' or self.LA(2)==u'<'): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'=': pass self.match('=') elif la1 and la1 in u'+': pass self.match('+') else: self.raise_NoViableAlt(self.LA(1)) self.mTEMPLATE(False) elif (self.LA(1)==u'+' or self.LA(1)==u'=') and (self.LA(2)==u'{'): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'=': pass self.match('=') elif la1 and la1 in u'+': pass self.match('+') else: self.raise_NoViableAlt(self.LA(1)) self.mSUBTEMPLATE(False) elif (self.LA(1)==u'+' or self.LA(1)==u'=') and (_tokenSet_7.member(self.LA(2))): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'=': pass self.match('=') elif la1 and la1 in u'+': pass self.match('+') else: self.raise_NoViableAlt(self.LA(1)) self.match(_tokenSet_7) elif (_tokenSet_8.member(self.LA(1))): pass self.matchNot('$') else: break _cnt48 += 1 if _cnt48 < 1: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mESC(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ESC _saveIndex = 0 pass self.match('\\') self.matchNot(antlr.EOF_CHAR) self.set_return_token(_createToken, _token, _ttype, _begin) def mSUBTEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SUBTEMPLATE _saveIndex = 0 pass self.match('{') while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'{': pass self.mSUBTEMPLATE(False) elif la1 and la1 in u'\\': pass self.mESC(False) else: if (_tokenSet_9.member(self.LA(1))): pass self.matchNot('}') else: break self.match('}') self.set_return_token(_createToken, _token, _ttype, _begin) def mTEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = TEMPLATE _saveIndex = 0 la1 = self.LA(1) if False: pass elif la1 and la1 in u'"': pass self.match('"') while True: if (self.LA(1)==u'\\'): pass self.mESC(False) elif (_tokenSet_10.member(self.LA(1))): pass self.matchNot('"') else: break self.match('"') elif la1 and la1 in u'<': pass self.match("<<") if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True): pass else: self.raise_NoViableAlt(self.LA(1)) while True: ### nongreedy exit test if ((self.LA(1)==u'>') and (self.LA(2)==u'>') and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and (True) and (True) and (True) and (True)): break if ((self.LA(1)==u'\r') and (self.LA(2)==u'\n') and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and ((self.LA(5) >= u'\u0001' and self.LA(5) <= u'\ufffe')) and (True) and (True) and ( self.LA(3) == '>' and self.LA(4) == '>' )): pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1)==u'\n') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True) and ( self.LA(2) == '>' and self.LA(3) == '>' )): pass _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() elif ((self.LA(1) >= u'\u0001' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0001' and self.LA(2) <= u'\ufffe')) and ((self.LA(3) >= u'\u0001' and self.LA(3) <= u'\ufffe')) and ((self.LA(4) >= u'\u0001' and self.LA(4) <= u'\ufffe')) and (True) and (True) and (True): pass self.matchNot(antlr.EOF_CHAR) else: break self.match(">>") else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mNESTED_PARENS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NESTED_PARENS _saveIndex = 0 pass self.match('(') _cnt69= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'(': pass self.mNESTED_PARENS(False) elif la1 and la1 in u'\\': pass self.mESC(False) else: if (_tokenSet_11.member(self.LA(1))): pass self.matchNot(')') else: break _cnt69 += 1 if _cnt69 < 1: self.raise_NoViableAlt(self.LA(1)) self.match(')') self.set_return_token(_createToken, _token, _ttype, _begin) def mHEX(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = HEX _saveIndex = 0 la1 = self.LA(1) if False: pass elif la1 and la1 in u'0123456789': pass self.matchRange(u'0', u'9') elif la1 and la1 in u'ABCDEF': pass self.matchRange(u'A', u'F') elif la1 and la1 in u'abcdef': pass self.matchRange(u'a', u'f') else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) ### generate bit set def mk_tokenSet_0(): data = [0L] * 2048 ### init list data[0] =-68719485954L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): data = [0L] * 2048 ### init list data[0] =-68719476738L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): data = [0L] * 1025 ### init list data[0] =4294967296L data[1] =14707067533131776L return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) ### generate bit set def mk_tokenSet_3(): data = [0L] * 1025 ### init list data[0] =287948969894477824L data[1] =541434314878L return data _tokenSet_3 = antlr.BitSet(mk_tokenSet_3()) ### generate bit set def mk_tokenSet_4(): data = [0L] * 2048 ### init list data[0] =-2199023255554L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_4 = antlr.BitSet(mk_tokenSet_4()) ### generate bit set def mk_tokenSet_5(): data = [0L] * 2048 ### init list data[0] =-1168231104514L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_5 = antlr.BitSet(mk_tokenSet_5()) ### generate bit set def mk_tokenSet_6(): data = [0L] * 2048 ### init list data[0] =-3298534892546L data[1] =-576460752571858945L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_6 = antlr.BitSet(mk_tokenSet_6()) ### generate bit set def mk_tokenSet_7(): data = [0L] * 2048 ### init list data[0] =-1152921521786716162L data[1] =-576460752303423489L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_7 = antlr.BitSet(mk_tokenSet_7()) ### generate bit set def mk_tokenSet_8(): data = [0L] * 2048 ### init list data[0] =-2305851874026202114L data[1] =-576460752571858945L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_8 = antlr.BitSet(mk_tokenSet_8()) ### generate bit set def mk_tokenSet_9(): data = [0L] * 2048 ### init list data[0] =-2L data[1] =-2882303761785552897L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_9 = antlr.BitSet(mk_tokenSet_9()) ### generate bit set def mk_tokenSet_10(): data = [0L] * 2048 ### init list data[0] =-17179869186L data[1] =-268435457L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_10 = antlr.BitSet(mk_tokenSet_10()) ### generate bit set def mk_tokenSet_11(): data = [0L] * 2048 ### init list data[0] =-3298534883330L data[1] =-268435457L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_11 = antlr.BitSet(mk_tokenSet_11()) ### __main__ header action >>> if __name__ == '__main__' : import sys import antlr import DefaultTemplateLexer ### create lexer - shall read from stdin try: for token in DefaultTemplateLexer.Lexer(): print token except antlr.TokenStreamException, e: print "error: exception caught while lexing: ", e ### __main__ header action <<< stringtemplate3-3.1/stringtemplate3/language/__init__.py0000444000175000001440000000151510756132624022411 0ustar pinkusers from stringtemplate3.language.ASTExpr import * from stringtemplate3.language.ActionEvaluator import * from stringtemplate3.language.ActionLexer import * from stringtemplate3.language.ActionParser import * from stringtemplate3.language.AngleBracketTemplateLexer import * from stringtemplate3.language.ChunkToken import * from stringtemplate3.language.ConditionalExpr import * from stringtemplate3.language.DefaultTemplateLexer import * from stringtemplate3.language.Expr import * from stringtemplate3.language.FormalArgument import * from stringtemplate3.language.GroupLexer import * from stringtemplate3.language.GroupParser import * from stringtemplate3.language.NewlineRef import * from stringtemplate3.language.StringRef import * from stringtemplate3.language.StringTemplateAST import * from stringtemplate3.language.TemplateParser import * stringtemplate3-3.1/stringtemplate3/language/StringTemplateToken.py0000444000175000001440000000133210756132624024612 0ustar pinkusers import antlr class StringTemplateToken(antlr.CommonToken): def __init__(self, type = 0, text = '', args = None): super(StringTemplateToken, self).__init__(type=type, text=text) ## Track any args for anonymous templates like # then }> # The lexer in action.g returns a single token ANONYMOUS_TEMPLATE # and so I need to have it parse args in the lexer and make them # available for when I build the anonymous template. if args is None: args = [] self.args = args def __str__(self): return super(StringTemplateToken, self).__str__() + \ '; args=' + str(self.args) __repr__ = __str__ stringtemplate3-3.1/stringtemplate3/language/CatIterator.py0000444000175000001440000000214710756132623023074 0ustar pinkusers from StringIO import StringIO ## Given a list of lists, return the combined elements one by one. # class CatList(object): def __init__(self, lists): ## List of lists to cat together # self._lists = lists def __len__(self): k = 0 for list_ in self._lists: k += len(list_) return k def lists(self): for list_ in self._lists: for item in list_: yield item def __iter__(self): for list_ in self._lists: for item in list_: yield item ## The result of asking for the string of a CatList is the list of # items and so this is just the cat'd list of both items. This # is destructive in that the iterator cursors have moved to the end # after printing. def __str__(self): buf = StringIO() #buf.write('[') k = len(self) for item in self.lists(): buf.write(str(item)) k -= 1 #if k: # buf.write(', ') #buf.write(']') return buf.getvalue() __repr__ = __str__ stringtemplate3-3.1/stringtemplate3/language/InterfaceLexer.py0000444000175000001440000003202610756132624023553 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "interface.g" -> "InterfaceLexer.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> # # [The "BSD licence"] # Copyright (c) 2003-2004 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import sys import traceback from stringtemplate3.language.FormalArgument import FormalArgument ### header action <<< ### preamble action >>> ### preamble action <<< ### >>>The Literals<<< literals = {} literals[u"interface"] = 4 literals[u"optional"] = 7 ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL_interface = 4 ID = 5 SEMI = 6 LITERAL_optional = 7 LPAREN = 8 RPAREN = 9 COMMA = 10 COLON = 11 SL_COMMENT = 12 ML_COMMENT = 13 WS = 14 class Lexer(antlr.CharScanner) : ### user action >>> ### user action <<< def __init__(self, *argv, **kwargs) : antlr.CharScanner.__init__(self, *argv, **kwargs) self.caseSensitiveLiterals = True self.setCaseSensitive(True) self.literals = literals def nextToken(self): while True: try: ### try again .. while True: _token = None _ttype = INVALID_TYPE self.resetText() try: ## for char stream error handling try: ##for lexical error handling la1 = self.LA(1) if False: pass elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz': pass self.mID(True) theRetToken = self._returnToken elif la1 and la1 in u'(': pass self.mLPAREN(True) theRetToken = self._returnToken elif la1 and la1 in u')': pass self.mRPAREN(True) theRetToken = self._returnToken elif la1 and la1 in u',': pass self.mCOMMA(True) theRetToken = self._returnToken elif la1 and la1 in u';': pass self.mSEMI(True) theRetToken = self._returnToken elif la1 and la1 in u':': pass self.mCOLON(True) theRetToken = self._returnToken elif la1 and la1 in u'\t\n\u000c\r ': pass self.mWS(True) theRetToken = self._returnToken else: if (self.LA(1)==u'/') and (self.LA(2)==u'/'): pass self.mSL_COMMENT(True) theRetToken = self._returnToken elif (self.LA(1)==u'/') and (self.LA(2)==u'*'): pass self.mML_COMMENT(True) theRetToken = self._returnToken else: self.default(self.LA(1)) if not self._returnToken: raise antlr.TryAgain ### found SKIP token ### option { testLiterals=true } self.testForLiteral(self._returnToken) ### return token to caller return self._returnToken ### handle lexical errors .... except antlr.RecognitionException, e: raise antlr.TokenStreamRecognitionException(e) ### handle char stream errors ... except antlr.CharStreamException,cse: if isinstance(cse, antlr.CharStreamIOException): raise antlr.TokenStreamIOException(cse.io) else: raise antlr.TokenStreamException(str(cse)) except antlr.TryAgain: pass def mID(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ID _saveIndex = 0 pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'abcdefghijklmnopqrstuvwxyz': pass self.matchRange(u'a', u'z') elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ': pass self.matchRange(u'A', u'Z') elif la1 and la1 in u'_': pass self.match('_') else: self.raise_NoViableAlt(self.LA(1)) while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'abcdefghijklmnopqrstuvwxyz': pass self.matchRange(u'a', u'z') elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ': pass self.matchRange(u'A', u'Z') elif la1 and la1 in u'0123456789': pass self.matchRange(u'0', u'9') elif la1 and la1 in u'-': pass self.match('-') elif la1 and la1 in u'_': pass self.match('_') else: break self.set_return_token(_createToken, _token, _ttype, _begin) def mLPAREN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LPAREN _saveIndex = 0 pass self.match('(') self.set_return_token(_createToken, _token, _ttype, _begin) def mRPAREN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = RPAREN _saveIndex = 0 pass self.match(')') self.set_return_token(_createToken, _token, _ttype, _begin) def mCOMMA(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COMMA _saveIndex = 0 pass self.match(',') self.set_return_token(_createToken, _token, _ttype, _begin) def mSEMI(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SEMI _saveIndex = 0 pass self.match(';') self.set_return_token(_createToken, _token, _ttype, _begin) def mCOLON(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COLON _saveIndex = 0 pass self.match(':') self.set_return_token(_createToken, _token, _ttype, _begin) def mSL_COMMENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SL_COMMENT _saveIndex = 0 pass self.match("//") while True: if (_tokenSet_0.member(self.LA(1))): pass self.match(_tokenSet_0) else: break if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') else: ## pass _ttype = Token.SKIP; self.newline(); self.set_return_token(_createToken, _token, _ttype, _begin) def mML_COMMENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ML_COMMENT _saveIndex = 0 pass self.match("/*") while True: ### nongreedy exit test if ((self.LA(1)==u'*') and (self.LA(2)==u'/')): break if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline(); elif ((self.LA(1) >= u'\u0000' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass self.matchNot(antlr.EOF_CHAR) else: break self.match("*/") _ttype = Token.SKIP; self.set_return_token(_createToken, _token, _ttype, _begin) def mWS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = WS _saveIndex = 0 pass _cnt32= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u' ': pass self.match(' ') elif la1 and la1 in u'\t': pass self.match('\t') elif la1 and la1 in u'\u000c': pass self.match('\f') elif la1 and la1 in u'\n\r': pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline(); else: break _cnt32 += 1 if _cnt32 < 1: self.raise_NoViableAlt(self.LA(1)) _ttype = Token.SKIP; self.set_return_token(_createToken, _token, _ttype, _begin) ### generate bit set def mk_tokenSet_0(): data = [0L] * 2048 ### init list data[0] =-9217L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### __main__ header action >>> if __name__ == '__main__' : import sys import antlr import InterfaceLexer ### create lexer - shall read from stdin try: for token in InterfaceLexer.Lexer(): print token except antlr.TokenStreamException, e: print "error: exception caught while lexing: ", e ### __main__ header action <<< stringtemplate3-3.1/stringtemplate3/language/InterfaceParser.py0000444000175000001440000001541510756132624023733 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "interface.g" -> "InterfaceParser.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> # # [The "BSD licence"] # Copyright (c) 2003-2004 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import sys import traceback from stringtemplate3.language.FormalArgument import FormalArgument ### header action <<< ### preamble action>>> ### preamble action <<< ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL_interface = 4 ID = 5 SEMI = 6 LITERAL_optional = 7 LPAREN = 8 RPAREN = 9 COMMA = 10 COLON = 11 SL_COMMENT = 12 ML_COMMENT = 13 WS = 14 ###/** Match an ST group interface. Just a list of template names with args. ### * Here is a sample interface file: ### * ### * interface nfa; ### * nfa(states,edges); ### * optional state(name); ### */ class Parser(antlr.LLkParser): ### user action >>> def reportError(self, e): if self.group_: self.group_.error("template group interface parse error", e) else: sys.stderr.write("template group interface parse error: " + str(e) + '\n') traceback.print_exc() ### user action <<< def __init__(self, *args, **kwargs): antlr.LLkParser.__init__(self, *args, **kwargs) self.tokenNames = _tokenNames ### __init__ header action >>> self.groupI = None ### __init__ header action <<< def groupInterface(self, groupI ): name = None self.groupI = groupI try: ## for error handling pass self.match(LITERAL_interface) name = self.LT(1) self.match(ID) groupI.name = name.getText() self.match(SEMI) _cnt3= 0 while True: if (self.LA(1)==ID or self.LA(1)==LITERAL_optional): pass self.template(groupI) else: break _cnt3 += 1 if _cnt3 < 1: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_0) def template(self, groupI ): opt = None name = None formalArgs = {} # leave blank if no args templateName = None try: ## for error handling pass la1 = self.LA(1) if False: pass elif la1 and la1 in [LITERAL_optional]: pass opt = self.LT(1) self.match(LITERAL_optional) elif la1 and la1 in [ID]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) name = self.LT(1) self.match(ID) self.match(LPAREN) la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass formalArgs=self.args() elif la1 and la1 in [RPAREN]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) self.match(RPAREN) self.match(SEMI) templateName = name.getText() groupI.defineTemplate(templateName, formalArgs, opt != None) except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_1) def args(self): args={} a = None b = None try: ## for error handling pass a = self.LT(1) self.match(ID) args[a.getText()] = FormalArgument(a.getText()) while True: if (self.LA(1)==COMMA): pass self.match(COMMA) b = self.LT(1) self.match(ID) args[b.getText()] = FormalArgument(b.getText()) else: break except antlr.RecognitionException, ex: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_2) return args _tokenNames = [ "<0>", "EOF", "<2>", "NULL_TREE_LOOKAHEAD", "\"interface\"", "ID", "SEMI", "\"optional\"", "LPAREN", "RPAREN", "COMMA", "COLON", "SL_COMMENT", "ML_COMMENT", "WS" ] ### generate bit set def mk_tokenSet_0(): ### var1 data = [ 2L, 0L] return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): ### var1 data = [ 162L, 0L] return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): ### var1 data = [ 512L, 0L] return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) stringtemplate3-3.1/stringtemplate3/language/GroupLexer.py0000444000175000001440000005426610756132624022761 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "group.g" -> "GroupLexer.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> from ASTExpr import * import stringtemplate3 import traceback ### header action <<< ### preamble action >>> ### preamble action <<< ### >>>The Literals<<< literals = {} literals[u"default"] = 21 literals[u"group"] = 4 literals[u"implements"] = 7 ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE LITERAL_group = 4 ID = 5 COLON = 6 LITERAL_implements = 7 COMMA = 8 SEMI = 9 AT = 10 DOT = 11 LPAREN = 12 RPAREN = 13 DEFINED_TO_BE = 14 STRING = 15 BIGSTRING = 16 ASSIGN = 17 ANONYMOUS_TEMPLATE = 18 LBRACK = 19 RBRACK = 20 LITERAL_default = 21 STAR = 22 PLUS = 23 OPTIONAL = 24 SL_COMMENT = 25 ML_COMMENT = 26 NL = 27 WS = 28 class Lexer(antlr.CharScanner) : ### user action >>> ### user action <<< def __init__(self, *argv, **kwargs) : antlr.CharScanner.__init__(self, *argv, **kwargs) self.caseSensitiveLiterals = True self.setCaseSensitive(True) self.literals = literals def nextToken(self): while True: try: ### try again .. while True: _token = None _ttype = INVALID_TYPE self.resetText() try: ## for char stream error handling try: ##for lexical error handling la1 = self.LA(1) if False: pass elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz': pass self.mID(True) theRetToken = self._returnToken elif la1 and la1 in u'"': pass self.mSTRING(True) theRetToken = self._returnToken elif la1 and la1 in u'<': pass self.mBIGSTRING(True) theRetToken = self._returnToken elif la1 and la1 in u'{': pass self.mANONYMOUS_TEMPLATE(True) theRetToken = self._returnToken elif la1 and la1 in u'@': pass self.mAT(True) theRetToken = self._returnToken elif la1 and la1 in u'(': pass self.mLPAREN(True) theRetToken = self._returnToken elif la1 and la1 in u')': pass self.mRPAREN(True) theRetToken = self._returnToken elif la1 and la1 in u'[': pass self.mLBRACK(True) theRetToken = self._returnToken elif la1 and la1 in u']': pass self.mRBRACK(True) theRetToken = self._returnToken elif la1 and la1 in u',': pass self.mCOMMA(True) theRetToken = self._returnToken elif la1 and la1 in u'.': pass self.mDOT(True) theRetToken = self._returnToken elif la1 and la1 in u';': pass self.mSEMI(True) theRetToken = self._returnToken elif la1 and la1 in u'*': pass self.mSTAR(True) theRetToken = self._returnToken elif la1 and la1 in u'+': pass self.mPLUS(True) theRetToken = self._returnToken elif la1 and la1 in u'=': pass self.mASSIGN(True) theRetToken = self._returnToken elif la1 and la1 in u'?': pass self.mOPTIONAL(True) theRetToken = self._returnToken elif la1 and la1 in u'\t\n\u000c\r ': pass self.mWS(True) theRetToken = self._returnToken else: if (self.LA(1)==u':') and (self.LA(2)==u':'): pass self.mDEFINED_TO_BE(True) theRetToken = self._returnToken elif (self.LA(1)==u'/') and (self.LA(2)==u'/'): pass self.mSL_COMMENT(True) theRetToken = self._returnToken elif (self.LA(1)==u'/') and (self.LA(2)==u'*'): pass self.mML_COMMENT(True) theRetToken = self._returnToken elif (self.LA(1)==u':') and (True): pass self.mCOLON(True) theRetToken = self._returnToken else: self.default(self.LA(1)) if not self._returnToken: raise antlr.TryAgain ### found SKIP token ### return token to caller return self._returnToken ### handle lexical errors .... except antlr.RecognitionException, e: raise antlr.TokenStreamRecognitionException(e) ### handle char stream errors ... except antlr.CharStreamException,cse: if isinstance(cse, antlr.CharStreamIOException): raise antlr.TokenStreamIOException(cse.io) else: raise antlr.TokenStreamException(str(cse)) except antlr.TryAgain: pass def mID(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ID _saveIndex = 0 pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'abcdefghijklmnopqrstuvwxyz': pass self.matchRange(u'a', u'z') elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ': pass self.matchRange(u'A', u'Z') elif la1 and la1 in u'_': pass self.match('_') else: self.raise_NoViableAlt(self.LA(1)) while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u'abcdefghijklmnopqrstuvwxyz': pass self.matchRange(u'a', u'z') elif la1 and la1 in u'ABCDEFGHIJKLMNOPQRSTUVWXYZ': pass self.matchRange(u'A', u'Z') elif la1 and la1 in u'0123456789': pass self.matchRange(u'0', u'9') elif la1 and la1 in u'-': pass self.match('-') elif la1 and la1 in u'_': pass self.match('_') else: break ### option { testLiterals=true } _ttype = self.testLiteralsTable(_ttype) self.set_return_token(_createToken, _token, _ttype, _begin) def mSTRING(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = STRING _saveIndex = 0 pass _saveIndex = self.text.length() self.match('"') self.text.setLength(_saveIndex) while True: if (self.LA(1)==u'\\') and (self.LA(2)==u'"'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('"') elif (self.LA(1)==u'\\') and (_tokenSet_0.member(self.LA(2))): pass self.match('\\') self.matchNot('"') elif (_tokenSet_1.member(self.LA(1))): pass self.matchNot('"') else: break _saveIndex = self.text.length() self.match('"') self.text.setLength(_saveIndex) self.set_return_token(_createToken, _token, _ttype, _begin) def mBIGSTRING(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = BIGSTRING _saveIndex = 0 pass _saveIndex = self.text.length() self.match("<<") self.text.setLength(_saveIndex) if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass _saveIndex = self.text.length() self.mNL(False) self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1) >= u'\u0000' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass else: self.raise_NoViableAlt(self.LA(1)) while True: ### nongreedy exit test if ((self.LA(1)==u'>') and (self.LA(2)==u'>')): break if ((self.LA(1)==u'\r') and (self.LA(2)==u'\n') and ( self.LA(3) == '>' and self.LA(4) == '>' )): pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1)==u'\n') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')) and ( self.LA(2) == '>' and self.LA(3) == '>' )): pass _saveIndex = self.text.length() self.match('\n') self.text.setLength(_saveIndex) self.newline() elif ((self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')) and ( self.LA(2) == '>' and self.LA(3) == '>' )): pass _saveIndex = self.text.length() self.match('\r') self.text.setLength(_saveIndex) self.newline() elif (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass self.mNL(False) self.newline() elif (self.LA(1)==u'\\') and (self.LA(2)==u'>'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('>') elif ((self.LA(1) >= u'\u0000' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass self.matchNot(antlr.EOF_CHAR) else: break _saveIndex = self.text.length() self.match(">>") self.text.setLength(_saveIndex) self.set_return_token(_createToken, _token, _ttype, _begin) def mNL(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = NL _saveIndex = 0 if (self.LA(1)==u'\r') and (self.LA(2)==u'\n'): pass self.match('\r') self.match('\n') elif (self.LA(1)==u'\r') and (True): pass self.match('\r') elif (self.LA(1)==u'\n'): pass self.match('\n') else: self.raise_NoViableAlt(self.LA(1)) self.set_return_token(_createToken, _token, _ttype, _begin) def mANONYMOUS_TEMPLATE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ANONYMOUS_TEMPLATE _saveIndex = 0 args = None t = None pass _saveIndex = self.text.length() self.match('{') self.text.setLength(_saveIndex) while True: ### nongreedy exit test if ((self.LA(1)==u'}') and (True)): break if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass la1 = self.LA(1) if False: pass elif la1 and la1 in u'\r': pass self.match('\r') elif la1 and la1 in u'\n': pass else: self.raise_NoViableAlt(self.LA(1)) self.match('\n') self.newline() elif (self.LA(1)==u'\\') and (self.LA(2)==u'>'): pass _saveIndex = self.text.length() self.match('\\') self.text.setLength(_saveIndex) self.match('>') elif ((self.LA(1) >= u'\u0000' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass self.matchNot(antlr.EOF_CHAR) else: break _saveIndex = self.text.length() self.match('}') self.text.setLength(_saveIndex) self.set_return_token(_createToken, _token, _ttype, _begin) def mAT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = AT _saveIndex = 0 pass self.match('@') self.set_return_token(_createToken, _token, _ttype, _begin) def mLPAREN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LPAREN _saveIndex = 0 pass self.match('(') self.set_return_token(_createToken, _token, _ttype, _begin) def mRPAREN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = RPAREN _saveIndex = 0 pass self.match(')') self.set_return_token(_createToken, _token, _ttype, _begin) def mLBRACK(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = LBRACK _saveIndex = 0 pass self.match('[') self.set_return_token(_createToken, _token, _ttype, _begin) def mRBRACK(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = RBRACK _saveIndex = 0 pass self.match(']') self.set_return_token(_createToken, _token, _ttype, _begin) def mCOMMA(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COMMA _saveIndex = 0 pass self.match(',') self.set_return_token(_createToken, _token, _ttype, _begin) def mDOT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = DOT _saveIndex = 0 pass self.match('.') self.set_return_token(_createToken, _token, _ttype, _begin) def mDEFINED_TO_BE(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = DEFINED_TO_BE _saveIndex = 0 pass self.match("::=") self.set_return_token(_createToken, _token, _ttype, _begin) def mSEMI(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SEMI _saveIndex = 0 pass self.match(';') self.set_return_token(_createToken, _token, _ttype, _begin) def mCOLON(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = COLON _saveIndex = 0 pass self.match(':') self.set_return_token(_createToken, _token, _ttype, _begin) def mSTAR(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = STAR _saveIndex = 0 pass self.match('*') self.set_return_token(_createToken, _token, _ttype, _begin) def mPLUS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = PLUS _saveIndex = 0 pass self.match('+') self.set_return_token(_createToken, _token, _ttype, _begin) def mASSIGN(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ASSIGN _saveIndex = 0 pass self.match('=') self.set_return_token(_createToken, _token, _ttype, _begin) def mOPTIONAL(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = OPTIONAL _saveIndex = 0 pass self.match('?') self.set_return_token(_createToken, _token, _ttype, _begin) def mSL_COMMENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = SL_COMMENT _saveIndex = 0 pass self.match("//") while True: if (_tokenSet_2.member(self.LA(1))): pass self.match(_tokenSet_2) else: break if (self.LA(1)==u'\n' or self.LA(1)==u'\r'): pass self.mNL(False) else: ## pass _ttype = SKIP; self.newline() self.set_return_token(_createToken, _token, _ttype, _begin) def mML_COMMENT(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = ML_COMMENT _saveIndex = 0 pass self.match("/*") while True: ### nongreedy exit test if ((self.LA(1)==u'*') and (self.LA(2)==u'/')): break if (self.LA(1)==u'\n' or self.LA(1)==u'\r') and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass self.mNL(False) self.newline() elif ((self.LA(1) >= u'\u0000' and self.LA(1) <= u'\ufffe')) and ((self.LA(2) >= u'\u0000' and self.LA(2) <= u'\ufffe')): pass self.matchNot(antlr.EOF_CHAR) else: break self.match("*/") _ttype = SKIP self.set_return_token(_createToken, _token, _ttype, _begin) def mWS(self, _createToken): _ttype = 0 _token = None _begin = self.text.length() _ttype = WS _saveIndex = 0 pass _cnt66= 0 while True: la1 = self.LA(1) if False: pass elif la1 and la1 in u' ': pass self.match(' ') elif la1 and la1 in u'\t': pass self.match('\t') elif la1 and la1 in u'\u000c': pass self.match('\f') elif la1 and la1 in u'\n\r': pass self.mNL(False) self.newline() else: break _cnt66 += 1 if _cnt66 < 1: self.raise_NoViableAlt(self.LA(1)) _ttype = SKIP self.set_return_token(_createToken, _token, _ttype, _begin) ### generate bit set def mk_tokenSet_0(): data = [0L] * 2048 ### init list data[0] =-17179869185L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): data = [0L] * 2048 ### init list data[0] =-17179869185L data[1] =-268435457L for x in xrange(2, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): data = [0L] * 2048 ### init list data[0] =-9217L for x in xrange(1, 1023): data[x] = -1L data[1023] =9223372036854775807L return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) ### __main__ header action >>> if __name__ == '__main__' : import sys import antlr import GroupLexer ### create lexer - shall read from stdin try: for token in GroupLexer.Lexer(): print token except antlr.TokenStreamException, e: print "error: exception caught while lexing: ", e ### __main__ header action <<< stringtemplate3-3.1/stringtemplate3/language/ActionParser.py0000444000175000001440000013765210756132623023257 0ustar pinkusers### $ANTLR 2.7.7 (2006-11-01): "action.g" -> "ActionParser.py"$ ### import antlr and other modules .. import sys import antlr version = sys.version.split()[0] if version < '2.2.1': False = 0 if version < '2.3': True = not False ### header action >>> from stringtemplate3.language.StringTemplateToken import StringTemplateToken import stringtemplate3 ### header action <<< ### preamble action>>> ### preamble action <<< ### import antlr.Token from antlr import Token ### >>>The Known Token Types <<< SKIP = antlr.SKIP INVALID_TYPE = antlr.INVALID_TYPE EOF_TYPE = antlr.EOF_TYPE EOF = antlr.EOF NULL_TREE_LOOKAHEAD = antlr.NULL_TREE_LOOKAHEAD MIN_USER_TYPE = antlr.MIN_USER_TYPE APPLY = 4 MULTI_APPLY = 5 ARGS = 6 INCLUDE = 7 CONDITIONAL = 8 VALUE = 9 TEMPLATE = 10 FUNCTION = 11 SINGLEVALUEARG = 12 LIST = 13 NOTHING = 14 SEMI = 15 LPAREN = 16 RPAREN = 17 LITERAL_elseif = 18 COMMA = 19 ID = 20 ASSIGN = 21 COLON = 22 NOT = 23 PLUS = 24 DOT = 25 LITERAL_first = 26 LITERAL_rest = 27 LITERAL_last = 28 LITERAL_length = 29 LITERAL_strip = 30 LITERAL_trunc = 31 LITERAL_super = 32 ANONYMOUS_TEMPLATE = 33 STRING = 34 INT = 35 LBRACK = 36 RBRACK = 37 DOTDOTDOT = 38 TEMPLATE_ARGS = 39 NESTED_ANONYMOUS_TEMPLATE = 40 ESC_CHAR = 41 WS = 42 WS_CHAR = 43 ###/** Parse the individual attribute expressions */ class Parser(antlr.LLkParser): ### user action >>> def reportError(self, e): group = self.this.group if group == stringtemplate3.StringTemplate.defaultGroup: self.this.error("action parse error; template context is "+self.this.enclosingInstanceStackString, e) else: self.this.error("action parse error in group "+self.this.group.name+" line "+str(self.this.groupFileLine)+"; template context is "+self.this.enclosingInstanceStackString, e) ### user action <<< def __init__(self, *args, **kwargs): antlr.LLkParser.__init__(self, *args, **kwargs) self.tokenNames = _tokenNames self.buildTokenTypeASTClassMap() self.astFactory = antlr.ASTFactory(self.getTokenTypeToASTClassMap()) self.astFactory.setASTNodeClass(stringtemplate3.language.StringTemplateAST) ### __init__ header action >>> if len(args) > 1 and isinstance(args[1], stringtemplate3.StringTemplate): self.this = args[1] else: raise ValueError("ActionParser requires a StringTemplate instance") ### __init__ header action <<< def action(self): opts = None self.returnAST = None currentAST = antlr.ASTPair() action_AST = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [EOF,SEMI,LPAREN,COMMA,ID,COLON,PLUS,LITERAL_first,LITERAL_rest,LITERAL_last,LITERAL_length,LITERAL_strip,LITERAL_trunc,LITERAL_super,ANONYMOUS_TEMPLATE,STRING,INT,LBRACK]: pass self.templatesExpr() self.addASTChild(currentAST, self.returnAST) la1 = self.LA(1) if False: pass elif la1 and la1 in [SEMI]: pass self.match(SEMI) opts=self.optionList() self.addASTChild(currentAST, self.returnAST) elif la1 and la1 in [EOF]: pass else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) action_AST = currentAST.root elif la1 and la1 in [CONDITIONAL]: pass tmp2_AST = None tmp2_AST = self.astFactory.create(self.LT(1)) self.makeASTRoot(currentAST, tmp2_AST) self.match(CONDITIONAL) self.match(LPAREN) self.ifCondition() self.addASTChild(currentAST, self.returnAST) self.match(RPAREN) action_AST = currentAST.root elif la1 and la1 in [LITERAL_elseif]: pass self.match(LITERAL_elseif) self.match(LPAREN) self.ifCondition() self.addASTChild(currentAST, self.returnAST) self.match(RPAREN) action_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_0) else: raise ex self.returnAST = action_AST return opts def templatesExpr(self): self.returnAST = None currentAST = antlr.ASTPair() templatesExpr_AST = None c = None c_AST = None try: ## for error handling synPredMatched10 = False if (_tokenSet_1.member(self.LA(1))) and (_tokenSet_2.member(self.LA(2))): _m10 = self.mark() synPredMatched10 = True self.inputState.guessing += 1 try: pass self.parallelArrayTemplateApplication() except antlr.RecognitionException, pe: synPredMatched10 = False self.rewind(_m10) self.inputState.guessing -= 1 if synPredMatched10: pass self.parallelArrayTemplateApplication() self.addASTChild(currentAST, self.returnAST) templatesExpr_AST = currentAST.root elif (_tokenSet_3.member(self.LA(1))) and (_tokenSet_4.member(self.LA(2))): pass self.expr() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==COLON): pass c = self.LT(1) c_AST = self.astFactory.create(c) self.makeASTRoot(currentAST, c_AST) self.match(COLON) if not self.inputState.guessing: c_AST.setType(APPLY) self.template() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==COMMA): pass self.match(COMMA) self.template() self.addASTChild(currentAST, self.returnAST) else: break else: break templatesExpr_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_5) else: raise ex self.returnAST = templatesExpr_AST def optionList(self): opts = {} self.returnAST = None currentAST = antlr.ASTPair() optionList_AST = None try: ## for error handling pass self.option(opts) while True: if (self.LA(1)==COMMA): pass tmp9_AST = None tmp9_AST = self.astFactory.create(self.LT(1)) self.match(COMMA) self.option(opts) else: break except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_0) else: raise ex self.returnAST = optionList_AST return opts def ifCondition(self): self.returnAST = None currentAST = antlr.ASTPair() ifCondition_AST = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [LPAREN,RPAREN,ID,PLUS,LITERAL_first,LITERAL_rest,LITERAL_last,LITERAL_length,LITERAL_strip,LITERAL_trunc,LITERAL_super,ANONYMOUS_TEMPLATE,STRING,INT,LBRACK]: pass self.ifAtom() self.addASTChild(currentAST, self.returnAST) ifCondition_AST = currentAST.root elif la1 and la1 in [NOT]: pass tmp10_AST = None tmp10_AST = self.astFactory.create(self.LT(1)) self.makeASTRoot(currentAST, tmp10_AST) self.match(NOT) self.ifAtom() self.addASTChild(currentAST, self.returnAST) ifCondition_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_6) else: raise ex self.returnAST = ifCondition_AST def option(self, opts ): self.returnAST = None currentAST = antlr.ASTPair() option_AST = None i = None i_AST = None e_AST = None try: ## for error handling pass i = self.LT(1) i_AST = self.astFactory.create(i) self.addASTChild(currentAST, i_AST) self.match(ID) la1 = self.LA(1) if False: pass elif la1 and la1 in [ASSIGN]: pass tmp11_AST = None tmp11_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp11_AST) self.match(ASSIGN) self.expr() e_AST = self.returnAST self.addASTChild(currentAST, self.returnAST) if not self.inputState.guessing: v=e_AST elif la1 and la1 in [EOF,COMMA]: pass if not self.inputState.guessing: v = stringtemplate3.language.ASTExpr.EMPTY_OPTION else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) if not self.inputState.guessing: opts[i_AST.getText()] = v option_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_7) else: raise ex self.returnAST = option_AST def expr(self): self.returnAST = None currentAST = antlr.ASTPair() expr_AST = None try: ## for error handling pass self.primaryExpr() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==PLUS): pass tmp12_AST = None tmp12_AST = self.astFactory.create(self.LT(1)) self.makeASTRoot(currentAST, tmp12_AST) self.match(PLUS) self.primaryExpr() self.addASTChild(currentAST, self.returnAST) else: break expr_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_8) else: raise ex self.returnAST = expr_AST def parallelArrayTemplateApplication(self): self.returnAST = None currentAST = antlr.ASTPair() parallelArrayTemplateApplication_AST = None c = None c_AST = None try: ## for error handling pass self.expr() self.addASTChild(currentAST, self.returnAST) _cnt17= 0 while True: if (self.LA(1)==COMMA): pass self.match(COMMA) self.expr() self.addASTChild(currentAST, self.returnAST) else: break _cnt17 += 1 if _cnt17 < 1: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) c = self.LT(1) c_AST = self.astFactory.create(c) self.addASTChild(currentAST, c_AST) self.match(COLON) self.anonymousTemplate() self.addASTChild(currentAST, self.returnAST) if not self.inputState.guessing: parallelArrayTemplateApplication_AST = currentAST.root parallelArrayTemplateApplication_AST = \ antlr.make(self.astFactory.create(MULTI_APPLY,"MULTI_APPLY"), parallelArrayTemplateApplication_AST) currentAST.root = parallelArrayTemplateApplication_AST if (parallelArrayTemplateApplication_AST != None) and (parallelArrayTemplateApplication_AST.getFirstChild() != None): currentAST.child = parallelArrayTemplateApplication_AST.getFirstChild() else: currentAST.child = parallelArrayTemplateApplication_AST currentAST.advanceChildToEnd() parallelArrayTemplateApplication_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_5) else: raise ex self.returnAST = parallelArrayTemplateApplication_AST def template(self): self.returnAST = None currentAST = antlr.ASTPair() template_AST = None try: ## for error handling pass la1 = self.LA(1) if False: pass elif la1 and la1 in [LPAREN,ID,LITERAL_super]: pass self.namedTemplate() self.addASTChild(currentAST, self.returnAST) elif la1 and la1 in [ANONYMOUS_TEMPLATE]: pass self.anonymousTemplate() self.addASTChild(currentAST, self.returnAST) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) if not self.inputState.guessing: template_AST = currentAST.root template_AST = antlr.make(self.astFactory.create(TEMPLATE), template_AST) currentAST.root = template_AST if (template_AST != None) and (template_AST.getFirstChild() != None): currentAST.child = template_AST.getFirstChild() else: currentAST.child = template_AST currentAST.advanceChildToEnd() template_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_9) else: raise ex self.returnAST = template_AST def anonymousTemplate(self): self.returnAST = None currentAST = antlr.ASTPair() anonymousTemplate_AST = None t = None t_AST = None try: ## for error handling pass t = self.LT(1) t_AST = self.astFactory.create(t) self.addASTChild(currentAST, t_AST) self.match(ANONYMOUS_TEMPLATE) if not self.inputState.guessing: anonymous = stringtemplate3.StringTemplate() anonymous.group = self.this.group anonymous.enclosingInstance = self.this anonymous.template = t.getText() anonymous.defineFormalArgument(t.args) t_AST.setStringTemplate(anonymous) anonymousTemplate_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_9) else: raise ex self.returnAST = anonymousTemplate_AST def ifAtom(self): self.returnAST = None currentAST = antlr.ASTPair() ifAtom_AST = None try: ## for error handling pass self.expr() self.addASTChild(currentAST, self.returnAST) ifAtom_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_6) else: raise ex self.returnAST = ifAtom_AST def primaryExpr(self): self.returnAST = None currentAST = antlr.ASTPair() primaryExpr_AST = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [EOF,SEMI,RPAREN,COMMA,COLON,PLUS,RBRACK]: pass primaryExpr_AST = currentAST.root elif la1 and la1 in [LITERAL_first,LITERAL_rest,LITERAL_last,LITERAL_length,LITERAL_strip,LITERAL_trunc]: pass self.function() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==DOT): pass tmp14_AST = None tmp14_AST = self.astFactory.create(self.LT(1)) self.makeASTRoot(currentAST, tmp14_AST) self.match(DOT) la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass tmp15_AST = None tmp15_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp15_AST) self.match(ID) elif la1 and la1 in [LPAREN]: pass self.valueExpr() self.addASTChild(currentAST, self.returnAST) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) else: break primaryExpr_AST = currentAST.root elif la1 and la1 in [LBRACK]: pass self.list_() self.addASTChild(currentAST, self.returnAST) primaryExpr_AST = currentAST.root else: synPredMatched25 = False if (self.LA(1)==LPAREN or self.LA(1)==ID or self.LA(1)==LITERAL_super) and (_tokenSet_10.member(self.LA(2))): _m25 = self.mark() synPredMatched25 = True self.inputState.guessing += 1 try: pass self.templateInclude() except antlr.RecognitionException, pe: synPredMatched25 = False self.rewind(_m25) self.inputState.guessing -= 1 if synPredMatched25: pass self.templateInclude() self.addASTChild(currentAST, self.returnAST) primaryExpr_AST = currentAST.root elif (_tokenSet_11.member(self.LA(1))) and (_tokenSet_12.member(self.LA(2))): pass self.atom() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==DOT): pass tmp16_AST = None tmp16_AST = self.astFactory.create(self.LT(1)) self.makeASTRoot(currentAST, tmp16_AST) self.match(DOT) la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass tmp17_AST = None tmp17_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp17_AST) self.match(ID) elif la1 and la1 in [LPAREN]: pass self.valueExpr() self.addASTChild(currentAST, self.returnAST) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) else: break primaryExpr_AST = currentAST.root elif (self.LA(1)==LPAREN) and (_tokenSet_13.member(self.LA(2))): pass self.valueExpr() self.addASTChild(currentAST, self.returnAST) primaryExpr_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_14) else: raise ex self.returnAST = primaryExpr_AST def templateInclude(self): self.returnAST = None currentAST = antlr.ASTPair() templateInclude_AST = None id = None id_AST = None qid = None qid_AST = None try: ## for error handling pass la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass id = self.LT(1) id_AST = self.astFactory.create(id) self.addASTChild(currentAST, id_AST) self.match(ID) self.argList() self.addASTChild(currentAST, self.returnAST) elif la1 and la1 in [LITERAL_super]: pass self.match(LITERAL_super) self.match(DOT) qid = self.LT(1) qid_AST = self.astFactory.create(qid) self.addASTChild(currentAST, qid_AST) self.match(ID) if not self.inputState.guessing: qid_AST.setText("super."+qid_AST.getText()) self.argList() self.addASTChild(currentAST, self.returnAST) elif la1 and la1 in [LPAREN]: pass self.indirectTemplate() self.addASTChild(currentAST, self.returnAST) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) if not self.inputState.guessing: templateInclude_AST = currentAST.root templateInclude_AST = antlr.make(self.astFactory.create(INCLUDE,"include"), templateInclude_AST) currentAST.root = templateInclude_AST if (templateInclude_AST != None) and (templateInclude_AST.getFirstChild() != None): currentAST.child = templateInclude_AST.getFirstChild() else: currentAST.child = templateInclude_AST currentAST.advanceChildToEnd() templateInclude_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_14) else: raise ex self.returnAST = templateInclude_AST def atom(self): self.returnAST = None currentAST = antlr.ASTPair() atom_AST = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass tmp20_AST = None tmp20_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp20_AST) self.match(ID) atom_AST = currentAST.root elif la1 and la1 in [STRING]: pass tmp21_AST = None tmp21_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp21_AST) self.match(STRING) atom_AST = currentAST.root elif la1 and la1 in [INT]: pass tmp22_AST = None tmp22_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp22_AST) self.match(INT) atom_AST = currentAST.root elif la1 and la1 in [ANONYMOUS_TEMPLATE]: pass tmp23_AST = None tmp23_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp23_AST) self.match(ANONYMOUS_TEMPLATE) atom_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_12) else: raise ex self.returnAST = atom_AST def valueExpr(self): self.returnAST = None currentAST = antlr.ASTPair() valueExpr_AST = None eval = None eval_AST = None try: ## for error handling pass eval = self.LT(1) eval_AST = self.astFactory.create(eval) self.makeASTRoot(currentAST, eval_AST) self.match(LPAREN) self.templatesExpr() self.addASTChild(currentAST, self.returnAST) self.match(RPAREN) if not self.inputState.guessing: eval_AST.setType(VALUE) eval_AST.setText("value") valueExpr_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_12) else: raise ex self.returnAST = valueExpr_AST def function(self): self.returnAST = None currentAST = antlr.ASTPair() function_AST = None try: ## for error handling pass la1 = self.LA(1) if False: pass elif la1 and la1 in [LITERAL_first]: pass tmp25_AST = None tmp25_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp25_AST) self.match(LITERAL_first) elif la1 and la1 in [LITERAL_rest]: pass tmp26_AST = None tmp26_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp26_AST) self.match(LITERAL_rest) elif la1 and la1 in [LITERAL_last]: pass tmp27_AST = None tmp27_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp27_AST) self.match(LITERAL_last) elif la1 and la1 in [LITERAL_length]: pass tmp28_AST = None tmp28_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp28_AST) self.match(LITERAL_length) elif la1 and la1 in [LITERAL_strip]: pass tmp29_AST = None tmp29_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp29_AST) self.match(LITERAL_strip) elif la1 and la1 in [LITERAL_trunc]: pass tmp30_AST = None tmp30_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp30_AST) self.match(LITERAL_trunc) else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) self.singleArg() self.addASTChild(currentAST, self.returnAST) if not self.inputState.guessing: function_AST = currentAST.root function_AST = antlr.make(self.astFactory.create(FUNCTION), function_AST) currentAST.root = function_AST if (function_AST != None) and (function_AST.getFirstChild() != None): currentAST.child = function_AST.getFirstChild() else: currentAST.child = function_AST currentAST.advanceChildToEnd() function_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_12) else: raise ex self.returnAST = function_AST def list_(self): self.returnAST = None currentAST = antlr.ASTPair() list__AST = None lb = None lb_AST = None try: ## for error handling pass lb = self.LT(1) lb_AST = self.astFactory.create(lb) self.makeASTRoot(currentAST, lb_AST) self.match(LBRACK) if not self.inputState.guessing: lb_AST.setType(LIST) lb_AST.setText("value") self.listElement() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==COMMA): pass self.match(COMMA) self.listElement() self.addASTChild(currentAST, self.returnAST) else: break self.match(RBRACK) list__AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_14) else: raise ex self.returnAST = list__AST def nonAlternatingTemplateExpr(self): self.returnAST = None currentAST = antlr.ASTPair() nonAlternatingTemplateExpr_AST = None c = None c_AST = None try: ## for error handling pass self.expr() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==COLON): pass c = self.LT(1) c_AST = self.astFactory.create(c) self.makeASTRoot(currentAST, c_AST) self.match(COLON) if not self.inputState.guessing: c_AST.setType(APPLY) self.template() self.addASTChild(currentAST, self.returnAST) else: break nonAlternatingTemplateExpr_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_15) else: raise ex self.returnAST = nonAlternatingTemplateExpr_AST def singleArg(self): self.returnAST = None currentAST = antlr.ASTPair() singleArg_AST = None try: ## for error handling pass self.match(LPAREN) self.nonAlternatingTemplateExpr() self.addASTChild(currentAST, self.returnAST) self.match(RPAREN) if not self.inputState.guessing: singleArg_AST = currentAST.root singleArg_AST = antlr.make(self.astFactory.create(SINGLEVALUEARG,"SINGLEVALUEARG"), singleArg_AST) currentAST.root = singleArg_AST if (singleArg_AST != None) and (singleArg_AST.getFirstChild() != None): currentAST.child = singleArg_AST.getFirstChild() else: currentAST.child = singleArg_AST currentAST.advanceChildToEnd() singleArg_AST = currentAST.root except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_12) else: raise ex self.returnAST = singleArg_AST def namedTemplate(self): self.returnAST = None currentAST = antlr.ASTPair() namedTemplate_AST = None qid = None qid_AST = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass tmp35_AST = None tmp35_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp35_AST) self.match(ID) self.argList() self.addASTChild(currentAST, self.returnAST) namedTemplate_AST = currentAST.root elif la1 and la1 in [LITERAL_super]: pass self.match(LITERAL_super) self.match(DOT) qid = self.LT(1) qid_AST = self.astFactory.create(qid) self.addASTChild(currentAST, qid_AST) self.match(ID) if not self.inputState.guessing: qid_AST.setText("super."+qid_AST.getText()) self.argList() self.addASTChild(currentAST, self.returnAST) namedTemplate_AST = currentAST.root elif la1 and la1 in [LPAREN]: pass self.indirectTemplate() self.addASTChild(currentAST, self.returnAST) namedTemplate_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_9) else: raise ex self.returnAST = namedTemplate_AST def argList(self): self.returnAST = None currentAST = antlr.ASTPair() argList_AST = None try: ## for error handling if (self.LA(1)==LPAREN) and (self.LA(2)==RPAREN): pass self.match(LPAREN) self.match(RPAREN) if not self.inputState.guessing: argList_AST = currentAST.root argList_AST = self.astFactory.create(ARGS,"ARGS") currentAST.root = argList_AST if (argList_AST != None) and (argList_AST.getFirstChild() != None): currentAST.child = argList_AST.getFirstChild() else: currentAST.child = argList_AST currentAST.advanceChildToEnd() else: synPredMatched52 = False if (self.LA(1)==LPAREN) and (_tokenSet_16.member(self.LA(2))): _m52 = self.mark() synPredMatched52 = True self.inputState.guessing += 1 try: pass self.singleArg() except antlr.RecognitionException, pe: synPredMatched52 = False self.rewind(_m52) self.inputState.guessing -= 1 if synPredMatched52: pass self.singleArg() self.addASTChild(currentAST, self.returnAST) argList_AST = currentAST.root elif (self.LA(1)==LPAREN) and (self.LA(2)==ID or self.LA(2)==DOTDOTDOT): pass self.match(LPAREN) self.argumentAssignment() self.addASTChild(currentAST, self.returnAST) while True: if (self.LA(1)==COMMA): pass self.match(COMMA) self.argumentAssignment() self.addASTChild(currentAST, self.returnAST) else: break self.match(RPAREN) if not self.inputState.guessing: argList_AST = currentAST.root argList_AST = antlr.make(self.astFactory.create(ARGS,"ARGS"), argList_AST) currentAST.root = argList_AST if (argList_AST != None) and (argList_AST.getFirstChild() != None): currentAST.child = argList_AST.getFirstChild() else: currentAST.child = argList_AST currentAST.advanceChildToEnd() argList_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_14) else: raise ex self.returnAST = argList_AST ###/** Match (foo)() and (foo+".terse")() */ def indirectTemplate(self): self.returnAST = None currentAST = antlr.ASTPair() indirectTemplate_AST = None e_AST = None args_AST = None try: ## for error handling pass tmp43_AST = None tmp43_AST = self.astFactory.create(self.LT(1)) self.match(LPAREN) self.templatesExpr() e_AST = self.returnAST tmp44_AST = None tmp44_AST = self.astFactory.create(self.LT(1)) self.match(RPAREN) self.argList() args_AST = self.returnAST if not self.inputState.guessing: indirectTemplate_AST = currentAST.root indirectTemplate_AST = antlr.make(self.astFactory.create(VALUE,"value"), e_AST, args_AST) currentAST.root = indirectTemplate_AST if (indirectTemplate_AST != None) and (indirectTemplate_AST.getFirstChild() != None): currentAST.child = indirectTemplate_AST.getFirstChild() else: currentAST.child = indirectTemplate_AST currentAST.advanceChildToEnd() except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_14) else: raise ex self.returnAST = indirectTemplate_AST def listElement(self): self.returnAST = None currentAST = antlr.ASTPair() listElement_AST = None try: ## for error handling if (_tokenSet_17.member(self.LA(1))) and (_tokenSet_4.member(self.LA(2))): pass self.expr() self.addASTChild(currentAST, self.returnAST) listElement_AST = currentAST.root elif (self.LA(1)==COMMA or self.LA(1)==RBRACK) and (_tokenSet_18.member(self.LA(2))): pass if not self.inputState.guessing: listElement_AST = currentAST.root listElement_AST = self.astFactory.create(NOTHING,"NOTHING") currentAST.root = listElement_AST if (listElement_AST != None) and (listElement_AST.getFirstChild() != None): currentAST.child = listElement_AST.getFirstChild() else: currentAST.child = listElement_AST currentAST.advanceChildToEnd() listElement_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_19) else: raise ex self.returnAST = listElement_AST def argumentAssignment(self): self.returnAST = None currentAST = antlr.ASTPair() argumentAssignment_AST = None try: ## for error handling la1 = self.LA(1) if False: pass elif la1 and la1 in [ID]: pass tmp45_AST = None tmp45_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp45_AST) self.match(ID) tmp46_AST = None tmp46_AST = self.astFactory.create(self.LT(1)) self.makeASTRoot(currentAST, tmp46_AST) self.match(ASSIGN) self.nonAlternatingTemplateExpr() self.addASTChild(currentAST, self.returnAST) argumentAssignment_AST = currentAST.root elif la1 and la1 in [DOTDOTDOT]: pass tmp47_AST = None tmp47_AST = self.astFactory.create(self.LT(1)) self.addASTChild(currentAST, tmp47_AST) self.match(DOTDOTDOT) argumentAssignment_AST = currentAST.root else: raise antlr.NoViableAltException(self.LT(1), self.getFilename()) except antlr.RecognitionException, ex: if not self.inputState.guessing: self.reportError(ex) self.consume() self.consumeUntil(_tokenSet_15) else: raise ex self.returnAST = argumentAssignment_AST def buildTokenTypeASTClassMap(self): self.tokenTypeToASTClassMap = None _tokenNames = [ "<0>", "EOF", "<2>", "NULL_TREE_LOOKAHEAD", "APPLY", "MULTI_APPLY", "ARGS", "INCLUDE", "\"if\"", "VALUE", "TEMPLATE", "FUNCTION", "SINGLEVALUEARG", "LIST", "NOTHING", "SEMI", "LPAREN", "RPAREN", "\"elseif\"", "COMMA", "ID", "ASSIGN", "COLON", "NOT", "PLUS", "DOT", "\"first\"", "\"rest\"", "\"last\"", "\"length\"", "\"strip\"", "\"trunc\"", "\"super\"", "ANONYMOUS_TEMPLATE", "STRING", "INT", "LBRACK", "RBRACK", "DOTDOTDOT", "TEMPLATE_ARGS", "NESTED_ANONYMOUS_TEMPLATE", "ESC_CHAR", "WS", "WS_CHAR" ] ### generate bit set def mk_tokenSet_0(): ### var1 data = [ 2L, 0L] return data _tokenSet_0 = antlr.BitSet(mk_tokenSet_0()) ### generate bit set def mk_tokenSet_1(): ### var1 data = [ 137390260224L, 0L] return data _tokenSet_1 = antlr.BitSet(mk_tokenSet_1()) ### generate bit set def mk_tokenSet_2(): ### var1 data = [ 274867093504L, 0L] return data _tokenSet_2 = antlr.BitSet(mk_tokenSet_2()) ### generate bit set def mk_tokenSet_3(): ### var1 data = [ 137394094082L, 0L] return data _tokenSet_3 = antlr.BitSet(mk_tokenSet_3()) ### generate bit set def mk_tokenSet_4(): ### var1 data = [ 274867126274L, 0L] return data _tokenSet_4 = antlr.BitSet(mk_tokenSet_4()) ### generate bit set def mk_tokenSet_5(): ### var1 data = [ 163842L, 0L] return data _tokenSet_5 = antlr.BitSet(mk_tokenSet_5()) ### generate bit set def mk_tokenSet_6(): ### var1 data = [ 131072L, 0L] return data _tokenSet_6 = antlr.BitSet(mk_tokenSet_6()) ### generate bit set def mk_tokenSet_7(): ### var1 data = [ 524290L, 0L] return data _tokenSet_7 = antlr.BitSet(mk_tokenSet_7()) ### generate bit set def mk_tokenSet_8(): ### var1 data = [ 137443835906L, 0L] return data _tokenSet_8 = antlr.BitSet(mk_tokenSet_8()) ### generate bit set def mk_tokenSet_9(): ### var1 data = [ 4882434L, 0L] return data _tokenSet_9 = antlr.BitSet(mk_tokenSet_9()) ### generate bit set def mk_tokenSet_10(): ### var1 data = [ 137428140032L, 0L] return data _tokenSet_10 = antlr.BitSet(mk_tokenSet_10()) ### generate bit set def mk_tokenSet_11(): ### var1 data = [ 60130590720L, 0L] return data _tokenSet_11 = antlr.BitSet(mk_tokenSet_11()) ### generate bit set def mk_tokenSet_12(): ### var1 data = [ 137494167554L, 0L] return data _tokenSet_12 = antlr.BitSet(mk_tokenSet_12()) ### generate bit set def mk_tokenSet_13(): ### var1 data = [ 137394585600L, 0L] return data _tokenSet_13 = antlr.BitSet(mk_tokenSet_13()) ### generate bit set def mk_tokenSet_14(): ### var1 data = [ 137460613122L, 0L] return data _tokenSet_14 = antlr.BitSet(mk_tokenSet_14()) ### generate bit set def mk_tokenSet_15(): ### var1 data = [ 655360L, 0L] return data _tokenSet_15 = antlr.BitSet(mk_tokenSet_15()) ### generate bit set def mk_tokenSet_16(): ### var1 data = [ 137394061312L, 0L] return data _tokenSet_16 = antlr.BitSet(mk_tokenSet_16()) ### generate bit set def mk_tokenSet_17(): ### var1 data = [ 274829213696L, 0L] return data _tokenSet_17 = antlr.BitSet(mk_tokenSet_17()) ### generate bit set def mk_tokenSet_18(): ### var1 data = [ 274833571842L, 0L] return data _tokenSet_18 = antlr.BitSet(mk_tokenSet_18()) ### generate bit set def mk_tokenSet_19(): ### var1 data = [ 137439477760L, 0L] return data _tokenSet_19 = antlr.BitSet(mk_tokenSet_19()) stringtemplate3-3.1/stringtemplate3/language/ConditionalExpr.py0000444000175000001440000000774010756132623023761 0ustar pinkusers import antlr from stringtemplate3.language import ASTExpr from stringtemplate3.language import ActionEvaluator from stringtemplate3.utils import deprecated class ElseIfClauseData(object): def __init__(self, expr, st): self.expr = expr self.st = st class ConditionalExpr(ASTExpr): """A conditional reference to an embedded subtemplate.""" def __init__(self, enclosingTemplate, tree): super(ConditionalExpr, self).__init__(enclosingTemplate, tree, None) self.subtemplate = None self.elseIfSubtemplates = None self.elseSubtemplate = None @deprecated def setSubtemplate(self, subtemplate): self.subtemplate = subtemplate @deprecated def getSubtemplate(self): return self.subtemplate @deprecated def setElseSubtemplate(self, elseSubtemplate): self.elseSubtemplate = elseSubtemplate @deprecated def getElseSubtemplate(self): return self.elseSubtemplate def addElseIfSubtemplate(self, conditionalTree, subtemplate): if self.elseIfSubtemplates is None: self.elseIfSubtemplates = [] d = ElseIfClauseData(conditionalTree, subtemplate) self.elseIfSubtemplates.append(d) def write(self, this, out): """ To write out the value of a condition expr, invoke the evaluator in eval.g to walk the condition tree computing the boolean value. If result is true, then write subtemplate. """ if self.exprTree is None or this is None or out is None: return 0 evaluator = ActionEvaluator.Walker() evaluator.initialize(this, self, out) n = 0 try: testedTrue = False # get conditional from tree and compute result cond = self.exprTree.getFirstChild() # eval and write out tree. In case the condition refers to an # undefined attribute, we catch the KeyError exception and proceed # with a False value. try: includeSubtemplate = evaluator.ifCondition(cond) except KeyError, ke: includeSubtemplate = False if includeSubtemplate: n = self.writeSubTemplate(this, out, self.subtemplate) testedTrue = True elif ( self.elseIfSubtemplates is not None and len(self.elseIfSubtemplates) > 0 ): for elseIfClause in self.elseIfSubtemplates: try: includeSubtemplate = evaluator.ifCondition(elseIfClause.expr.exprTree) except KeyError, ke: includeSubtemplate = False if includeSubtemplate: n = self.writeSubTemplate(this, out, elseIfClause.st) testedTrue = True break if not testedTrue and self.elseSubtemplate is not None: # evaluate ELSE clause if present and IF/ELSEIF conditions # failed n = self.writeSubTemplate(this, out, self.elseSubtemplate) except antlr.RecognitionException, re: this.error( "can't evaluate tree: " + self.exprTree.toStringList(), re ) return n def writeSubTemplate(self, this, out, subtemplate): # To evaluate the IF chunk, make a new instance whose enclosingInstance # points at 'this' so get attribute works. Otherwise, enclosingInstance # points at the template used to make the precompiled code. We need a # new template instance every time we exec this chunk to get the new # "enclosing instance" pointer. s = subtemplate.getInstanceOf() s.enclosingInstance = this # make sure we evaluate in context of enclosing template's # group so polymorphism works. :) s.group = this.group s.nativeGroup = this.nativeGroup return s.write(out) stringtemplate3-3.1/stringtemplate3/language/FormalArgument.py0000444000175000001440000000446110756132624023600 0ustar pinkusers # the following represent bit positions emulating a cardinality bitset. OPTIONAL = 1 # a? REQUIRED = 2 # a ZERO_OR_MORE = 4 # a* ONE_OR_MORE = 8 # a+ suffixes = [ None, "?", "", None, "*", None, None, None, "+" ] ## When template arguments are not available such as when the user # uses "new StringTemplate(...)", then the list of formal arguments # must be distinguished from the case where a template can specify # args and there just aren't any such as the t() template above. UNKNOWN_ARGS = { '': -1 } def getCardinalityName(cardinality): return { OPTIONAL: 'optional', REQUIRED: 'exactly one', ZERO_OR_MORE: 'zero-or-more', ONE_OR_MORE: 'one-or-more' }.get(cardinality, 'unknown') class FormalArgument(object): """ Represents the name of a formal argument defined in a template: group test; test(a,b) : "$a$ $b$" t() : "blort" Each template has a set of these formal arguments or uses a placeholder object: UNKNOWN (indicating that no arguments were specified such as when a template is loaded from a file.st). Note: originally, I tracked cardinality as well as the name of an attribute. I'm leaving the code here as I suspect something may come of it later. Currently, though, cardinality is not used. """ def __init__(self, name, defaultValueST = None): self.name = name # self.cardinality = REQUIRED ## If they specified name="value", store the template here # self.defaultValueST = defaultValueST def __eq__(self, other): if not isinstance(other, FormalArgument): return False if self.name != other.name: return False # only check if there is a default value; that's all if ( (self.defaultValueST is not None and other.defaultValueST is None) or (self.defaultValueST is None and other.defaultValueST is not None) ): return False return True def __neq__(self, other): return not self.__eq__(other) def __str__(self): if self.defaultValueST: return self.name + '=' + str(self.defaultValueST) return self.name __repr__ = __str__ stringtemplate3-3.1/stringtemplate3/writers.py0000444000175000001440000002204110756132626020565 0ustar pinkusers # [The "BSD licence"] # Copyright (c) 2003-2006 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # from stringtemplate3.utils import deprecated class AttributeRenderer(object): """ This interface describes an object that knows how to format or otherwise render an object appropriately. Usually this is used for locale changes for objects such as Date and floating point numbers... You can either have an object that is sensitive to the locale or have a different object per locale. Each template may have a renderer for each object type or can default to the group's renderer or the super group's renderer if the group doesn't have one. The toString(Object,String) method is used when the user uses the format option: $o; format="f"$. It checks the formatName and applies the appropriate formatting. If the format string passed to the renderer is not recognized then simply call toString(). """ def __init__(self): pass def toString(self, o, formatName=None): pass class StringTemplateWriter(object): """ Generic StringTemplate output writer filter. Literals and the elements of expressions are emitted via write(). Separators are emitted via writeSeparator() because they must be handled specially when wrapping lines (we don't want to wrap in between an element and it's separator). """ NO_WRAP = -1 def __init__(self): pass def pushIndentation(self, indent): raise NotImplementedError def popIndentation(self): raise NotImplementedError def pushAnchorPoint(self): raise NotImplementedError def popAnchorPoint(self): raise NotImplementedError def setLineWidth(self, lineWidth): raise NotImplementedError def write(self, str, wrap=None): """ Write the string and return how many actual chars were written. With autoindentation and wrapping, more chars than length(str) can be emitted. No wrapping is done unless wrap is not None """ raise NotImplementedError def writeWrapSeparator(self, wrap): """ Because we might need to wrap at a non-atomic string boundary (such as when we wrap in between template applications ]}; wrap>) we need to expose the wrap string writing just like for the separator. """ raise NotImplementedError def writeSeparator(self, str): """ Write a separator. Same as write() except that a \n cannot be inserted before emitting a separator. """ raise NotImplementedError class AutoIndentWriter(StringTemplateWriter): """ Essentially a char filter that knows how to auto-indent output by maintaining a stack of indent levels. I set a flag upon newline and then next nonwhitespace char resets flag and spits out indention. The indent stack is a stack of strings so we can repeat original indent not just the same number of columns (don't have to worry about tabs vs spaces then). Anchors are char positions (tabs won't work) that indicate where all future wraps should justify to. The wrap position is actually the larger of either the last anchor or the indentation level. This is a filter on a Writer. It may be screwed up for '\r' '\n' on PC's. """ def __init__(self, out): StringTemplateWriter.__init__(self) ## stack of indents self.indents = [None] # start with no indent ## Stack of integer anchors (char positions in line) self.anchors = [] self.out = out self.atStartOfLine = True ## Track char position in the line (later we can think about tabs). # Indexed from 0. We want to keep charPosition <= lineWidth. # This is the position we are *about* to write not the position # last written to. self.charPosition = 0 self.lineWidth = self.NO_WRAP self.charPositionOfStartOfExpr = 0 @deprecated def setLineWidth(self, lineWidth): self.lineWidth = lineWidth def pushIndentation(self, indent): """ Push even blank (null) indents as they are like scopes; must be able to pop them back off stack. """ self.indents.append(indent) def popIndentation(self): return self.indents.pop(-1) def getIndentationWidth(self): return sum(len(ind) for ind in self.indents if ind is not None) def pushAnchorPoint(self): self.anchors.append(self.charPosition) def popAnchorPoint(self): self.anchors.pop(-1) def write(self, text, wrap=None): """ Write out a string literal or attribute expression or expression element. If doing line wrap, then check wrap before emitting this str. If at or beyond desired line width then emit a \n and any indentation before spitting out this str. """ assert isinstance(text, basestring), repr(text) n = 0 if wrap is not None: n += self.writeWrapSeparator(wrap) for c in text: if c == '\n': self.atStartOfLine = True self.charPosition = -1 # set so the write below sets to 0 else: if self.atStartOfLine: n += self.indent() self.atStartOfLine = False n += 1 self.out.write(c) self.charPosition += 1 return n def writeWrapSeparator(self, wrap): n = 0 # if want wrap and not already at start of line (last char was \n) # and we have hit or exceeded the threshold if ( self.lineWidth != self.NO_WRAP and not self.atStartOfLine and self.charPosition >= self.lineWidth ): # ok to wrap # Walk wrap string and look for A\nB. Spit out A\n # then spit indent or anchor, whichever is larger # then spit out B for c in wrap: if c == '\n': n += 1 self.out.write(c) #atStartOfLine = true; self.charPosition = 0 indentWidth = self.getIndentationWidth() try: lastAnchor = self.anchors[-1] except IndexError: # no anchors saved lastAnchor = 0 if lastAnchor > indentWidth: # use anchor not indentation n += self.indent(lastAnchor) else: # indent is farther over than last anchor, ignore anchor n += self.indent() # continue writing any chars out else: # write A or B part n += 1 self.out.write(c) self.charPosition += 1 return n def writeSeparator(self, text): return self.write(text) def indent(self, spaces=None): if spaces is None: n = 0 for ind in self.indents: if ind is not None: n += len(ind) self.out.write(ind) self.charPosition += n return n else: self.out.write(' ' * spaces) self.charPosition += spaces return spaces ## Just pass through the text # class NoIndentWriter(AutoIndentWriter): def __init__(self, out): super(NoIndentWriter, self).__init__(out) def write(self, str, wrap=None): self.out.write(str) return len(str) stringtemplate3-3.1/stringtemplate3/templates.py0000444000175000001440000015061210756132624021070 0ustar pinkusers # [The "BSD licence"] # Copyright (c) 2003-2006 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import sys import traceback from StringIO import StringIO from copy import copy import antlr from stringtemplate3.language import ( FormalArgument, UNKNOWN_ARGS, ChunkToken, ASTExpr, StringTemplateAST, TemplateParser, ActionLexer, ActionParser, ConditionalExpr, NewlineRef, StringTemplateToken, ) from stringtemplate3.writers import StringTemplateWriter from stringtemplate3.utils import deprecated import stringtemplate3 class STAttributeList(list): """ Just an alias for list, but this way I can track whether a list is something ST created or it's an incoming list. """ pass class Aggregate(object): """ An automatically created aggregate of properties. I often have lists of things that need to be formatted, but the list items are actually pieces of data that are not already in an object. I need ST to do something like: Ter=3432 Tom=32234 .... using template: $items:{$attr.name$=$attr.type$$ This example will call getName() on the objects in items attribute, but what if they aren't objects? I have perhaps two parallel arrays instead of a single array of objects containing two fields. One solution is allow dictionaries to be handled like properties so that it.name would fail getName() but then see that it's a dictionary and do it.get('name') instead. This very clean approach is espoused by some, but the problem is that it's a hole in my separation rules. People can put the logic in the view because you could say: 'go get bob's data' in the view: Bob's Phone: $db.bob.phone$ A view should not be part of the program and hence should never be able to go ask for a specific person's data. After much thought, I finally decided on a simple solution. I've added setAttribute variants that pass in multiple property values, with the property names specified as part of the name using a special attribute name syntax: 'name.{propName1,propName2,...'. This object is a special kind of dictionary that hopefully prevents people from passing a subclass or other variant that they have created as it would be a loophole. Anyway, the ASTExpr.getObjectProperty() method looks for Aggregate as a special case and does a get() instead of getPropertyName. """ def __init__(self, master): self.properties = {} self.master = master ## Allow StringTemplate to add values, but prevent the end # user from doing so. # def __setitem__(self, propName, propValue): # Instead of relying on data hiding, we check the type of the # master of this aggregate. if isinstance(self.master, StringTemplate): self.properties[propName] = propValue else: raise AttributeError def get(self, propName, default=None): # Instead of relying on data hiding, we check the type of the # master of this aggregate. if isinstance(self.master, StringTemplate): return self.properties.get(propName, default) raise AttributeError def __getitem__(self, propName): # Instead of relying on data hiding, we check the type of the # master of this aggregate. if isinstance(self.master, StringTemplate): if self.properties.has_key(propName): return self.properties[propName] return None raise AttributeError def __contains__(self, propName): # Instead of relying on data hiding, we check the type of the # master of this aggregate. if isinstance(self.master, StringTemplate): return self.properties.has_key(propName) raise AttributeError def __str__(self): return str(self.properties) # <@r()> REGION_IMPLICIT = 1 # <@r>...<@end> REGION_EMBEDDED = 2 # @t.r() ::= "..." defined manually by coder REGION_EXPLICIT = 3 ANONYMOUS_ST_NAME = "anonymous" ## incremental counter for templates IDs templateCounter = 0 def getNextTemplateCounter(): global templateCounter templateCounter += 1 return templateCounter def resetTemplateCounter(): """ reset the template ID counter to 0; def that testing routine can access but not really of interest to the user. """ global templateCounter templateCounter = 0 class StringTemplate(object): """ A StringTemplate is a "document" with holes in it where you can stick values. StringTemplate breaks up your template into chunks of text and attribute expressions. StringTemplate< ignores everything outside of attribute expressions, treating it as just text to spit out when you call StringTemplate.toString(). """ ## Either: # Create a blank template with no pattern and no attributes # Or: # Create an anonymous template. It has no name just # chunks (which point to self anonymous template) and attributes. # Or: # Create an anonymous template with no name, but with a group # Or: # Create a template # def __init__(self, template=None, group=None, lexer=None, attributes=None): self.referencedAttributes = None ## What's the name of self template? self.name = ANONYMOUS_ST_NAME self.templateID = getNextTemplateCounter() ## Enclosing instance if I'm embedded within another template. # IF-subtemplates are considered embedded as well. self._enclosingInstance = None ## A list of embedded templates self.embeddedInstances = None ## If self template is an embedded template such as when you apply # a template to an attribute, then the arguments passed to self # template represent the argument context--a set of values # computed by walking the argument assignment list. For example, # would result in an # argument context of:[item=name], [foo="x"] for self # template. This template would be the bold() template and # the enclosingInstance would point at the template that held # that template call. When you want to get # an attribute value, you first check the attributes for the # 'self' template then the arg context then the enclosingInstance # like resolving variables in pascal-like language with nested # procedures. # # With multi-valued attributes such as # attribute "i" is set to 1..n. self.argumentContext = None ## If self template is embedded in another template, the arguments # must be evaluated just before each application when applying # template to a list of values. The "it" attribute must change # with each application so that $names:bold(item=it)$ works. If # you evaluate once before starting the application loop then it # has a single fixed value. Eval.g saves the AST rather than evaluating # before invoking applyListOfAlternatingTemplates(). Each iteration # of a template application to a multi-valued attribute, these args # are re-evaluated with an initial context of:[it=...], [i=...]. self.argumentsAST = None ## When templates are defined in a group file format, the attribute # list is provided including information about attribute cardinality # such as present, optional, ... When self information is available, # rawSetAttribute should do a quick existence check as should the # invocation of other templates. So if you ref bold(item="foo") but # item is not defined in bold(), then an exception should be thrown. # When actually rendering the template, the cardinality is checked. # This is a {str:FormalArgument} dictionary. self.formalArgumentKeys = None self.formalArguments = UNKNOWN_ARGS ## How many formal arguments to this template have default values # specified? self.numberOfDefaultArgumentValues = 0 ## Normally, formal parameters hide any attributes inherited from the # enclosing template with the same name. This is normally what you # want, but makes it hard to invoke another template passing in all # the data. Use notation now: to say "pass in # all data". Works great. Can also say self.passThroughAttributes = False ## What group originally defined the prototype for self template? # This affects the set of templates I can refer to. super.t() must # always refer to the super of the original group. # # group base; # t ::= "base"; # # group sub; # t ::= "super.t()2" # # group subsub; # t ::= "super.t()3" self.nativeGroup = None ## This template was created as part of what group? Even if this # template was created from a prototype in a supergroup, its group # will be the subgroup. That's the way polymorphism works. if group is not None: assert isinstance(group, StringTemplateGroup) self.group = group else: self.group = StringTemplateGroup(name='defaultGroup', rootDir='.') if lexer is not None: self.group.templateLexerClass = lexer ## If this template is defined within a group file, what line number? self._groupFileLine = None ## Where to report errors self.listener = None ## The original, immutable pattern/language (not really used again # after initial "compilation", setup/parsing). self.pattern = None ## Map an attribute name to its value(s). These values are set by # outside code via st[name] = value. StringTemplate is like self in # that a template is both the "class def" and "instance". When you # create a StringTemplate or setTemplate, the text is broken up into # chunks (i.e., compiled down into a series of chunks that can be # evaluated later). # You can have multiple. self.attributes = None ## A Map that allows people to register a renderer for # a particular kind of object to be displayed in this template. This # overrides any renderer set for this template's group. # # Most of the time this map is not used because the StringTemplateGroup # has the general renderer map for all templates in that group. # Sometimes though you want to override the group's renderers. self.attributeRenderers = None ## A list of alternating string and ASTExpr references. # This is compiled when the template is loaded/defined and walked to # write out a template instance. self.chunks = None ## If someone refs <@r()> in template t, an implicit # # @t.r() ::= "" # # is defined, but you can overwrite this def by defining your # own. We need to prevent more than one manual def though. Between # this var and isEmbeddedRegion we can determine these cases. self.regionDefType = None ## Does this template come from a <@region>...<@end> embedded in # another template? self._isRegion = False ## Set of implicit and embedded regions for this template */ self.regions = set() if template is not None: assert isinstance(template, basestring) self.template = template if attributes is not None: assert isinstance(attributes, dict) self.attributes = attributes def dup(self, fr, to): """ Make the 'to' template look exactly like the 'from' template except for the attributes. This is like creating an instance of a class in that the executable code is the same (the template chunks), but the instance data is blank (the attributes). Do not copy the enclosingInstance pointer since you will want self template to eval in a context different from the examplar. """ to.attributeRenderers = fr.attributeRenderers to.pattern = copy(fr.pattern) to.chunks = copy(fr.chunks) to.formalArgumentKeys = copy(fr.formalArgumentKeys) to.formalArguments = copy(fr.formalArguments) to.numberOfDefaultArgumentValues = fr.numberOfDefaultArgumentValues to.name = copy(fr.name) to.nativeGroup = fr.nativeGroup to.group = fr.group to.listener = copy(fr.listener) to.regions = fr.regions to._isRegion = fr._isRegion to.regionDefTyep = fr.regionDefType def getInstanceOf(self): """ Make an instance of self template; it contains an exact copy of everything (except the attributes and enclosing instance pointer). So the new template refers to the previously compiled chunks of self template but does not have any attribute values. """ if self.nativeGroup is not None: # create a template using the native group for this template # but it's "group" is set to this.group by dup after creation so # polymorphism still works. t = self.nativeGroup.createStringTemplate() else: t = self.group.createStringTemplate() self.dup(self, t) return t def getEnclosingInstance(self): return self._enclosingInstance def setEnclosingInstance(self, enclosingInstance): if self == self._enclosingInstance: raise AttributeError('cannot embed template ' + str(self.name) + ' in itself') # set the parent for this template self._enclosingInstance = enclosingInstance # make the parent track self template as an embedded template if enclosingInstance: self._enclosingInstance.addEmbeddedInstance(self) enclosingInstance = property(getEnclosingInstance, setEnclosingInstance) getEnclosingInstance = deprecated(getEnclosingInstance) setEnclosingInstance = deprecated(setEnclosingInstance) def getOutermostEnclosingInstance(self): if self.enclosingInstance is not None: return self.enclosingInstance.getOutermostEnclosingInstance() return self def addEmbeddedInstance(self, embeddedInstance): if not self.embeddedInstances: self.embeddedInstances = [] self.embeddedInstances.append(embeddedInstance) @deprecated def getArgumentContext(self): return self.argumentContext @deprecated def setArgumentContext(self, ac): self.argumentContext = ac @deprecated def getArgumentsAST(self): return self.argumentsAST @deprecated def setArgumentsAST(self, argumentsAST): self.argumentsAST = argumentsAST @deprecated def getName(self): return self.name @deprecated def setName(self, name): self.name = name def getOutermostName(self): if self.enclosingInstance is not None: return self.enclosingInstance.getOutermostName() return self.name @deprecated def getGroup(self): return self.group @deprecated def setGroup(self, group): self.group = group @deprecated def getNativeGroup(self): return self.nativeGroup @deprecated def setNativeGroup(self, group): self.nativeGroup = group def getGroupFileLine(self): """Return the outermost template's group file line number""" if self.enclosingInstance is not None: return self.enclosingInstance.getGroupFileLine() return self._groupFileLine def setGroupFileLine(self, groupFileLine): self._groupFileLine = groupFileLine groupFileLine = property(getGroupFileLine, setGroupFileLine) getGroupFileLine = deprecated(getGroupFileLine) setGroupFileLine = deprecated(setGroupFileLine) def setTemplate(self, template): self.pattern = template self.breakTemplateIntoChunks() def getTemplate(self): return self.pattern template = property(getTemplate, setTemplate) getTemplate = deprecated(getTemplate) setTemplate = deprecated(setTemplate) def setErrorListener(self, listener): self.listener = listener def getErrorListener(self): if not self.listener: return self.group.errorListener return self.listener errorListener = property(getErrorListener, setErrorListener) getErrorListener = deprecated(getErrorListener) setErrorListener = deprecated(setErrorListener) def reset(self): # just throw out table and make new one self.attributes = {} def setPredefinedAttributes(self): # only do self method so far in lint mode if not stringtemplate3.lintMode: return def removeAttribute(self, name): del self.attributes[name] __delitem__ = removeAttribute def setAttribute(self, name, *values): """ Set an attribute for self template. If you set the same attribute more than once, you get a multi-valued attribute. If you send in a StringTemplate object as a value, its enclosing instance (where it will inherit values from) is set to 'self'. This would be the normal case, though you can set it back to None after this call if you want. If you send in a List plus other values to the same attribute, they all get flattened into one List of values. This will be a new list object so that incoming objects are not altered. If you send in an array, it is converted to a List. Works with arrays of objects and arrays of:int,float,double. """ if len(values) == 0: return if len(values) == 1: value = values[0] if value is None or name is None: return if '.' in name: raise ValueError("cannot have '.' in attribute names") if self.attributes is None: self.attributes = {} if isinstance(value, StringTemplate): value.enclosingInstance = self elif (isinstance(value, (list, tuple)) and not isinstance(value, STAttributeList)): # convert to STAttributeList value = STAttributeList(value) # convert plain collections # get exactly in this scope (no enclosing) o = self.attributes.get(name, None) if o is None: # new attribute self.rawSetAttribute(self.attributes, name, value) return # it will be a multi-value attribute if isinstance(o, STAttributeList): # already a list made by ST v = o elif isinstance(o, list): # existing attribute is non-ST List # must copy to an ST-managed list before adding new attribute v = STAttributeList() v.extend(o) self.rawSetAttribute(self.attributes, name, v) # replace attribute w/list else: # non-list second attribute, must convert existing to ArrayList v = STAttributeList() # make list to hold multiple values # make it point to list now self.rawSetAttribute(self.attributes, name, v) # replace attribute w/list v.append(o) # add previous single-valued attribute if isinstance(value, list): # flatten incoming list into existing if v != value: # avoid weird cyclic add v.extend(value) else: v.append(value) else: ## Create an aggregate from the list of properties in aggrSpec and # fill with values from values array. # aggrSpec = name aggrName, properties = self.parseAggregateAttributeSpec(aggrSpec) if not values or len(properties) == 0: raise ValueError('missing properties or values for \'' + aggrSpec + '\'') if len(values) != len(properties): raise IndexError('number of properties in \'' + aggrSpec + '\' != number of values') aggr = Aggregate(self) for i, value in enumerate(values): if isinstance(value, StringTemplate): value.setEnclosingInstance(self) #else: # value = AST.Expr.convertArrayToList(value) property_ = properties[i] aggr[property_] = value self.setAttribute(aggrName, aggr) __setitem__ = setAttribute def parseAggregateAttributeSpec(self, aggrSpec): """ Split "aggrName.{propName1,propName2" into list [propName1,propName2] and the aggrName. Space is allowed around ',' """ dot = aggrSpec.find('.') if dot <= 0: raise ValueError('invalid aggregate attribute format: ' + aggrSpec) aggrName = aggrSpec[:dot].strip() propString = aggrSpec[dot+1:] propString = [ p.strip() for p in propString.split('{',2)[-1].split('}',2)[0].split(',') ] return aggrName, propString def rawSetAttribute(self, attributes, name, value): """ Map a value to a named attribute. Throw KeyError if the named attribute is not formally defined in self's specific template and a formal argument list exists. """ if self.formalArguments != UNKNOWN_ARGS and \ not self.hasFormalArgument(name): # a normal call to setAttribute with unknown attribute raise KeyError("no such attribute: " + name + " in template context " + self.enclosingInstanceStackString) if value is not None: attributes[name] = value elif isinstance(value, list) or \ isinstance(value, dict) or \ isinstance(value, set): attributes[name] = value def rawSetArgumentAttribute(self, embedded, attributes, name, value): """ Argument evaluation such as foo(x=y), x must be checked against foo's argument list not this's (which is the enclosing context). So far, only eval.g uses arg self as something other than "this". """ if embedded.formalArguments != UNKNOWN_ARGS and \ not embedded.hasFormalArgument(name): raise KeyError("template " + embedded.name + " has no such attribute: " + name + " in template context " + self.enclosingInstanceStackString) if value: attributes[name] = value elif isinstance(value, list) or \ isinstance(value, dict) or \ isinstance(value, set): attributes[name] = value def write(self, out): """ Walk the chunks, asking them to write themselves out according to attribute values of 'self.attributes'. This is like evaluating or interpreting the StringTemplate as a program using the attributes. The chunks will be identical (point at same list) for all instances of self template. """ if self.group.debugTemplateOutput: self.group.emitTemplateStartDebugString(self, out) n = 0 self.setPredefinedAttributes() self.setDefaultArgumentValues() if self.chunks: i = 0 while i < len(self.chunks): a = self.chunks[i] chunkN = a.write(self, out) # expr-on-first-line-with-no-output NEWLINE => NEWLINE if ( chunkN == 0 and i == 0 and i + 1 < len(self.chunks) and isinstance(self.chunks[i+1], NewlineRef) ): # skip next NEWLINE i += 2 # skip *and* advance! continue # NEWLINE expr-with-no-output NEWLINE => NEWLINE # Indented $...$ have the indent stored with the ASTExpr # so the indent does not come out as a StringRef if (not chunkN) and (i-1) >= 0 and \ isinstance(self.chunks[i-1], NewlineRef) and \ (i+1) < len(self.chunks) and \ isinstance(self.chunks[i+1], NewlineRef): #sys.stderr.write('found pure \\n blank \\n pattern\n') i += 1 # make it skip over the next chunk, the NEWLINE n += chunkN i += 1 if self.group.debugTemplateOutput: self.group.emitTemplateStopDebugString(self, out) if stringtemplate3.lintMode: self.checkForTrouble() return n def get(self, this, attribute): """ Resolve an attribute reference. It can be in four possible places: 1. the attribute list for the current template 2. if self is an embedded template, somebody invoked us possibly with arguments--check the argument context 3. if self is an embedded template, the attribute list for the enclosing instance (recursively up the enclosing instance chain) 4. if nothing is found in the enclosing instance chain, then it might be a map defined in the group or the its supergroup etc... Attribute references are checked for validity. If an attribute has a value, its validity was checked before template rendering. If the attribute has no value, then we must check to ensure it is a valid reference. Somebody could reference any random value like $xyz$ formal arg checks before rendering cannot detect self--only the ref can initiate a validity check. So, if no value, walk up the enclosed template tree again, this time checking formal parameters not attributes dictionary. The formal definition must exist even if no value. To avoid infinite recursion in str(), we have another condition to check regarding attribute values. If your template has a formal argument, foo, then foo will hide any value available from "above" in order to prevent infinite recursion. This method is not static so people can override its functionality. """ if not this: return None if stringtemplate3.lintMode: this.trackAttributeReference(attribute) # is it here? o = None if this.attributes and this.attributes.has_key(attribute): o = this.attributes[attribute] return o # nope, check argument context in case embedded if not o: argContext = this.argumentContext if argContext and argContext.has_key(attribute): o = argContext[attribute] return o if (not o) and \ (not this.passThroughAttributes) and \ this.hasFormalArgument(attribute): # if you've defined attribute as formal arg for self # template and it has no value, do not look up the # enclosing dynamic scopes. This avoids potential infinite # recursion. return None # not locally defined, check enclosingInstance if embedded if (not o) and this.enclosingInstance: #sys.stderr.write('looking for ' + self.getName() + '.' + \ # str(attribute) + ' in super [=' + \ # this.enclosingInstance.getName() + ']\n') valueFromEnclosing = self.get(this.enclosingInstance, attribute) if not valueFromEnclosing: self.checkNullAttributeAgainstFormalArguments(this, attribute) o = valueFromEnclosing # not found and no enclosing instance to look at elif (not o) and (not this.enclosingInstance): # It might be a map in the group or supergroup... o = this.group.getMap(attribute) return o def getAttribute(self, name): return self.get(self, name) __getitem__ = getAttribute def breakTemplateIntoChunks(self): """ Walk a template, breaking it into a list of chunks: Strings and actions/expressions. """ #sys.stderr.write('parsing template: ' + str(self.pattern) + '\n') if not self.pattern: return try: # instead of creating a specific template lexer, use # an instance of the class specified by the user. # The default is DefaultTemplateLexer. # The only constraint is that you use an ANTLR lexer # so I can use the special ChunkToken. lexerClass = self.group.templateLexerClass chunkStream = lexerClass(StringIO(self.pattern)) chunkStream.this = self chunkStream.setTokenObjectClass(ChunkToken) chunkifier = TemplateParser.Parser(chunkStream) chunkifier.template(self) except Exception, e: name = "" outerName = self.getOutermostName() if self.name: name = self.name if outerName and not name == outerName: name = name + ' nested in ' + outerName self.error('problem parsing template \'' + name + '\' ', e) def parseAction(self, action): lexer = ActionLexer.Lexer(StringIO(str(action))) parser = ActionParser.Parser(lexer, self) parser.setASTNodeClass(StringTemplateAST) lexer.setTokenObjectClass(StringTemplateToken) a = None try: options = parser.action() tree = parser.getAST() if tree: if tree.getType() == ActionParser.CONDITIONAL: a = ConditionalExpr(self, tree) else: a = ASTExpr(self, tree, options) except antlr.RecognitionException, re: self.error('Can\'t parse chunk: ' + str(action), re) except antlr.TokenStreamException, tse: self.error('Can\'t parse chunk: ' + str(action), tse) return a @deprecated def getTemplateID(self): return self.templateID @deprecated def getAttributes(self): return self.attributes @deprecated def setAttributes(self, attributes): self.attributes = attributes ## Get a list of the strings and subtemplates and attribute # refs in a template. @deprecated def getChunks(self): return self.chunks def addChunk(self, e): if not self.chunks: self.chunks = [] self.chunks.append(e) # ---------------------------------------------------------------------------- # F o r m a l A r g S t u f f # ---------------------------------------------------------------------------- @deprecated def getFormalArgumentKeys(self): return self.formalArgumentKeys @deprecated def getFormalArguments(self): return self.formalArguments @deprecated def setFormalArguments(self, args): self.formalArguments = args def setDefaultArgumentValues(self): """ Set any default argument values that were not set by the invoking template or by setAttribute directly. Note that the default values may be templates. Their evaluation context is the template itself and, hence, can see attributes within the template, any arguments, and any values inherited by the template. Default values are stored in the argument context rather than the template attributes table just for consistency's sake. """ if not self.numberOfDefaultArgumentValues: return if not self.argumentContext: self.argumentContext = {} if self.formalArguments != UNKNOWN_ARGS: argNames = self.formalArgumentKeys for argName in argNames: # use the default value then arg = self.formalArguments[argName] if arg.defaultValueST: existingValue = self.getAttribute(argName) if not existingValue: # value unset? # if no value for attribute, set arg context # to the default value. We don't need an instance # here because no attributes can be set in # the arg templates by the user. self.argumentContext[argName] = arg.defaultValueST def lookupFormalArgument(self, name): """ From self template upward in the enclosing template tree, recursively look for the formal parameter. """ if not self.hasFormalArgument(name): if self.enclosingInstance: arg = self.enclosingInstance.lookupFormalArgument(name) else: arg = None else: arg = self.getFormalArgument(name) return arg def getFormalArgument(self, name): return self.formalArguments[name] def hasFormalArgument(self, name): return self.formalArguments.has_key(name) def defineEmptyFormalArgumentList(self): self.formalArgumentKeys = [] self.formalArguments = {} def defineFormalArgument(self, names, defaultValue = None): if not names: return if isinstance(names, basestring): name = names if defaultValue: self.numberOfDefaultArgumentValues += 1 a = FormalArgument(name, defaultValue) if self.formalArguments == UNKNOWN_ARGS: self.formalArguments = {} self.formalArgumentKeys = [name] self.formalArguments[name] = a elif isinstance(names, list): for name in names: a = FormalArgument(name, defaultValue) if self.formalArguments == UNKNOWN_ARGS: self.formalArgumentKeys = [] self.formalArguments = {} self.formalArgumentKeys.append(name) self.formalArguments[name] = a @deprecated def setPassThroughAttributes(self, passThroughAttributes): """ Normally if you call template y from x, y cannot see any attributes of x that are defined as formal parameters of y. Setting this passThroughAttributes to true, will override that and allow a template to see through the formal arg list to inherited values. """ self.passThroughAttributes = passThroughAttributes @deprecated def setAttributeRenderers(self, renderers): """ Specify a complete map of what object classes should map to which renderer objects. """ self.attributeRenderers = renderers def registerRenderer(self, attributeClassType, renderer): """ Register a renderer for all objects of a particular type. This overrides any renderer set in the group for this class type. """ if not self.attributeRenderers: self.attributeRenderers = {} self.attributeRenderers[attributeClassType] = renderer def getAttributeRenderer(self, attributeClassType): """ What renderer is registered for this attributeClassType for this template. If not found, the template's group is queried. """ renderer = None if self.attributeRenderers is not None: renderer = self.attributeRenderers.get(attributeClassType, None) if renderer is not None: # found it return renderer # we have no renderer overrides for the template or none for class arg # check parent template if we are embedded if self.enclosingInstance is not None: return self.enclosingInstance.getAttributeRenderer(attributeClassType) # else check group return self.group.getAttributeRenderer(attributeClassType) # ---------------------------------------------------------------------------- # U t i l i t y R o u t i n e s # ---------------------------------------------------------------------------- def warning(self, msg): if self.errorListener is not None: self.errorListener.warning(msg) else: sys.stderr.write('StringTemplate: warning: ' + msg) def error(self, msg, e = None): if self.errorListener is not None: self.errorListener.error(msg, e) elif e: sys.stderr.write('StringTemplate: error: ' + msg + ': ' + str(e)) traceback.print_exc() else: sys.stderr.write('StringTemplate: error: ' + msg) def trackAttributeReference(self, name): """ Indicates that 'name' has been referenced in self template. """ if not self.referencedAttributes: self.referencedAttributes = [] if not name in self.referencedAttributes: self.referencedAttributes.append(name) @classmethod def isRecursiveEnclosingInstance(cls, st): """ Look up the enclosing instance chain (and include self) to see if st is a template already in the enclosing instance chain. """ if not st: return False p = st.enclosingInstance if p == st: # self-recursive return True # now look for indirect recursion while p: if p == st: return True p = p.enclosingInstance return False def getEnclosingInstanceStackTrace(self): buf = StringIO() seen = {} p = self while p: if hash(p) in seen: buf.write(p.templateDeclaratorString) buf.write(" (start of recursive cycle)\n...") break seen[hash(p)] = p buf.write(p.templateDeclaratorString) if p.attributes: buf.write(", attributes=[") i = 0 for attrName in p.attributes.keys(): if i > 0: buf.write(", ") i += 1 buf.write(attrName) o = p.attributes[attrName] if isinstance(o, StringTemplate): buf.write('=<' + o.name + '()@') buf.write(str(o.templateID) + '>') elif isinstance(o, list): buf.write("=List[..") n = 0 for st in o: if isinstance(st, StringTemplate): if n > 0: buf.write(", ") n += 1 buf.write('<' + st.name + '()@') buf.write(str(st.templateID) + '>') buf.write("..]") buf.write(']') if p.referencedAttributes: buf.write(', references=') buf.write(p.referencedAttributes) buf.write('>\n') p = p.enclosingInstance # if self.enclosingInstance: # buf.write(enclosingInstance.getEnclosingInstanceStackTrace()) return buf.getvalue() enclosingInstanceStackTrace = property(getEnclosingInstanceStackTrace) getEnclosingInstanceStackTrace = deprecated(getEnclosingInstanceStackTrace) def getTemplateDeclaratorString(self): return '<' + self.name + '(' + str(self.formalArgumentKeys) + \ ')' + '@' + str(self.templateID) + '>' templateDeclaratorString = property(getTemplateDeclaratorString) getTemplateDeclaratorString = deprecated(getTemplateDeclaratorString) def getTemplateHeaderString(self, showAttributes): if showAttributes and self.attributes is not None: return self.name + str(self.attributes.keys()) return self.name def checkNullAttributeAgainstFormalArguments(self, this, attribute): """ A reference to an attribute with no value, must be compared against the formal parameter to see if it exists; if it exists all is well, but if not, throw an exception. Don't do the check if no formal parameters exist for self template ask enclosing. """ if this.formalArguments == UNKNOWN_ARGS: # bypass unknown arg lists if this.enclosingInstance: self.checkNullAttributeAgainstFormalArguments( this.enclosingInstance, attribute) else: formalArg = this.lookupFormalArgument(attribute) if not formalArg: raise KeyError('no such attribute: ' + str(attribute) + ' in template context ' + self.enclosingInstanceStackString) def checkForTrouble(self): """ Executed after evaluating a template. For now, checks for setting of attributes not reference. """ # we have table of set values and list of values referenced # compare, looking for SET BUT NOT REFERENCED ATTRIBUTES if not self.attributes: return for name in self.attributes.keys(): if self.referencedAttributes and \ not name in self.referencedAttributes: self.warning(self.name + ': set but not used: ' + name) # can do the reverse, but will have lots of False warnings :( def getEnclosingInstanceStackString(self): """ If an instance of x is enclosed in a y which is in a z, return a String of these instance names in order from topmost to lowest; here that would be "[z y x]". """ names = [] p = self while p: names.append(p.name) p = p.enclosingInstance names.reverse() s = '[' while names: s += names[0] if len(names) > 1: s += ' ' names = names[1:] return s + ']' enclosingInstanceStackString = property(getEnclosingInstanceStackString) getEnclosingInstanceStackString = deprecated(getEnclosingInstanceStackString) def isRegion(self): return self._isRegion def setIsRegion(self, isRegion): self._isRegion = isRegion def addRegionName(self, name): self.regions.add(name) def containsRegionName(self, name): return name in self.regions @deprecated def getRegionDefType(self): return self.regionDefType @deprecated def setRegionDefType(self, regionDefType): self.regionDefType = regionDefType def toDebugString(self): buf = StringIO() buf.write('template-' + self.getTemplateDeclaratorString() + ': ') buf.write('chunks=') if self.chunks: buf.write(str(self.chunks)) buf.write('attributes=[') if self.attributes: n = 0 for name in self.attributes.keys(): if n > 0: buf.write(',') buf.write(name + '=') value = self.attributes[name] if isinstance(value, StringTemplate): buf.write(value.toDebugString()) else: buf.write(value) n += 1 buf.write(']') retval = buf.getvalue() buf.close() return retval def toStructureString(self, indent=0): """ Don't print values, just report the nested structure with attribute names. Follow (nest) attributes that are templates only. """ buf = StringIO() buf.write(' '*indent) # indent buf.write(self.name) buf.write(str(self.attributes.keys())) # FIXME: errr.. that's correct? buf.write(":\n") if self.attributes is not None: attrNames = self.attributes.keys() for name in attrNames: value = self.attributes[name] if isinstance(value, StringTemplate): # descend buf.write(value.toStructureString(indent+1)) else: if isinstance(value, list): for o in value: if isinstance(o, StringTemplate): # descend buf.write(o.toStructureString(indent+1)) elif isinstance(value, dict): for o in value.values(): if isinstance(o, StringTemplate): # descend buf.write(o.toStructureString(indent+1)) return buf.getvalue() def getDOTForDependencyGraph(self, showAttributes): """ Generate a DOT file for displaying the template enclosure graph; e.g., digraph prof { "t1" -> "t2" "t1" -> "t3" "t4" -> "t5" } """ structure = ( "digraph StringTemplateDependencyGraph {\n" + "node [shape=$shape$, $if(width)$width=$width$,$endif$" + " $if(height)$height=$height$,$endif$ fontsize=$fontsize$];\n" + "$edges:{e|\"$e.src$\" -> \"$e.trg$\"\n}$" + "}\n" ) graphST = StringTemplate(structure) edges = {} self.getDependencyGraph(edges, showAttributes) # for each source template for src, targetNodes in edges.iteritems(): # for each target template for trg in targetNodes: graphST.setAttribute("edges.{src,trg}", src, trg) graphST.setAttribute("shape", "none") graphST.setAttribute("fontsize", "11") graphST.setAttribute("height", "0") # make height return graphST def getDependencyGraph(self, edges, showAttributes): """ Get a list of n->m edges where template n contains template m. The map you pass in is filled with edges: key->value. Useful for having DOT print out an enclosing template graph. It finds all direct template invocations too like but not indirect ones like <(name)()>. Ack, I just realized that this is done statically and hence cannot see runtime arg values on statically included templates. Hmm...someday figure out to do this dynamically as if we were evaluating the templates. There will be extra nodes in the tree because we are static like method and method[...] with args. """ srcNode = self.getTemplateHeaderString(showAttributes) if self.attributes is not None: for name, value in self.attributes.iteritems(): if isinstance(value, StringTemplate): targetNode = value.getTemplateHeaderString(showAttributes) self.putToMultiValuedMap(edges, srcNode, targetNode) value.getDependencyGraph(edges, showAttributes) # descend else: if isinstance(value, list): for o in value: if isinstance(o, StringTemplate): targetNode = o.getTemplateHeaderString(showAttributes) self.putToMultiValuedMap(edges, srcNode, targetNode) o.getDependencyGraph(edges, showAttributes) # descend elif isinstance(value, dict): for o in value.values(): if isinstance(o, StringTemplate): targetNode = o.getTemplateHeaderString(showAttributes) self.putToMultiValuedMap(edges, srcNode, targetNode) o.getDependencyGraph(edges, showAttributes) # descend # look in chunks too for template refs for chunk in self.chunks: if not isinstance(chunk, ASTExpr): continue from stringtemplate3.language.ActionEvaluator import INCLUDE tree = chunk.getAST() includeAST = antlr.CommonAST( antlr.CommonToken(INCLUDE,"include") ) for t in tree.findAllPartial(includeAST): templateInclude = t.getFirstChild().getText() #System.out.println("found include "+templateInclude); self.putToMultiValuedMap(edges, srcNode, templateInclude) group = self.getGroup() if group is not None: st = group.getInstanceOf(templateInclude) # descend into the reference template st.getDependencyGraph(edges, showAttributes) def putToMultiValuedMap(self, map, key, value): """Manage a hash table like it has multiple unique values.""" try: map[key].append(value) except KeyError: map[key] = [value] def printDebugString(self, out=sys.stderr): out.write('template-' + self.name + ':\n') out.write('chunks=' + str(self.chunks)) if not self.attributes: return out.write("attributes=[") n = 0 for name in self.attributes.keys(): if n > 0: out.write(',') value = self.attributes[name] if isinstance(value, StringTemplate): out.write(name + '=') value.printDebugString() else: if isinstance(value, list): i = 0 for o in value: out.write(name + '[' + i + '] is ' + o.__class__.__name__ + '=') if isinstance(o, StringTemplate): o.printDebugString() else: out.write(o) i += 1 else: out.write(name + '=' + value + '\n') n += 1 out.write("]\n") def toString(self, lineWidth=StringTemplateWriter.NO_WRAP): # Write the output to a StringIO out = StringIO(u'') wr = self.group.getStringTemplateWriter(out) wr.lineWidth = lineWidth try: self.write(wr) except IOError, io: self.error("Got IOError writing to writer" + \ str(wr.__class__.__name__)) # reset so next toString() does not wrap; normally this is a new writer # each time, but just in case they override the group to reuse the # writer. wr.lineWidth = StringTemplateWriter.NO_WRAP return out.getvalue() __str__ = toString # initialize here, because of cyclic imports from stringtemplate3.groups import StringTemplateGroup StringTemplateGroup.NOT_FOUND_ST = StringTemplate() ASTExpr.MAP_KEY_VALUE = StringTemplate() stringtemplate3-3.1/stringtemplate3/utils.py0000444000175000001440000000054310756132626020231 0ustar pinkusersimport warnings def deprecated(func): def wrap(*args, **kwargs): warnings.warn( 'Use of this method is deprecated, use property instead', DeprecationWarning, stacklevel=2 ) return func(*args, **kwargs) wrap.__name__ = func.__name__ wrap.__doc__ = func.__doc__ return wrap stringtemplate3-3.1/stringtemplate3/groups.py0000444000175000001440000007516310756132623020417 0ustar pinkusers # [The "BSD licence"] # Copyright (c) 2003-2006 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import sys import traceback import imp import time from StringIO import StringIO import antlr from stringtemplate3.language import ( AngleBracketTemplateLexer, DefaultTemplateLexer, GroupLexer, GroupParser, ) from stringtemplate3.utils import deprecated from stringtemplate3.errors import ( DEFAULT_ERROR_LISTENER ) from stringtemplate3.templates import ( StringTemplate, REGION_IMPLICIT ) from stringtemplate3.writers import AutoIndentWriter from stringtemplate3.interfaces import StringTemplateGroupInterface DEFAULT_EXTENSION = '.st' ## Used to indicate that the template doesn't exist. # We don't have to check disk for it; we know it's not there. # Set later to work around cyclic class definitions NOT_FOUND_ST = None class StringTemplateGroup(object): """ Manages a group of named mutually-referential StringTemplate objects. Currently the templates must all live under a directory so that you can reference them as foo.st or gutter/header.st. To refresh a group of templates, just create a new StringTemplateGroup and start pulling templates from there. Or, set the refresh interval. Use getInstanceOf(template-name) to get a string template to fill in. The name of a template is the file name minus ".st" ending if present unless you name it as you load it. You can use the group file format also to define a group of templates (this works better for code gen than for html page gen). You must give a Reader to the ctor for it to load the group; this is general and distinguishes it from the ctors for the old-style "load template files from the disk". 10/2005 I am adding a StringTemplateGroupLoader concept so people can define supergroups within a group and have it load that group automatically. """ ## Track all groups by name; maps name to StringTemplateGroup nameToGroupMap = {} ## Track all interfaces by name; maps name to StringTemplateGroupInterface nameToInterfaceMap = {} ## If a group file indicates it derives from a supergroup, how do we # find it? Shall we make it so the initial StringTemplateGroup file # can be loaded via this loader? Right now we pass a Reader to ctor # to distinguish from the other variety. groupLoader = None ## You can set the lexer once if you know all of your groups use the # same separator. If the instance has templateLexerClass set # then it is used as an override. defaultTemplateLexerClass = DefaultTemplateLexer.Lexer def __init__(self, name=None, rootDir=None, lexer=None, file=None, errors=None, superGroup=None): ## What is the group name # self.name = None ## Maps template name to StringTemplate object # self.templates = {} ## Maps map names to HashMap objects. This is the list of maps # defined by the user like typeInitMap ::= ["int":"0"] # self.maps = {} ## How to pull apart a template into chunks? self._templateLexerClass = None ## Under what directory should I look for templates? If None, # to look into the CLASSPATH for templates as resources. # self.rootDir = None ## Are we derived from another group? Templates not found in this # group will be searched for in the superGroup recursively. self._superGroup = None ## Keep track of all interfaces implemented by this group. self.interfaces = [] ## When templates are files on the disk, the refresh interval is used # to know when to reload. When a Reader is passed to the ctor, # it is a stream full of template definitions. The former is used # for web development, but the latter is most likely used for source # code generation for translators; a refresh is unlikely. Anyway, # I decided to track the source of templates in case such info is useful # in other situations than just turning off refresh interval. I just # found another: don't ever look on the disk for individual templates # if this group is a group file...immediately look into any super group. # If not in the super group, report no such template. # self.templatesDefinedInGroupFile = False ## Normally AutoIndentWriter is used to filter output, but user can # specify a new one. # self.userSpecifiedWriter = None self.debugTemplateOutput = False ## The set of templates to ignore when dumping start/stop debug strings self.noDebugStartStopStrings = None ## A Map that allows people to register a renderer for # a particular kind of object to be displayed for any template in this # group. For example, a date should be formatted differently depending # on the locale. You can set Date.class to an object whose # str() method properly formats a Date attribute # according to locale. Or you can have a different renderer object # for each locale. # # These render objects are used way down in the evaluation chain # right before an attribute's str() method would normally be # called in ASTExpr.write(). # self.attributeRenderers = None ## Where to report errors. All string templates in this group # use this error handler by default. if errors is not None: self.listener = errors else: self.listener = DEFAULT_ERROR_LISTENER ## How long before tossing out all templates in seconds. # default: no refreshing from disk # self.refreshInterval = sys.maxint/1000 self.lastCheckedDisk = 0L if name is not None: assert isinstance(name, basestring) self.name = name assert rootDir is None or isinstance(rootDir, basestring) self.rootDir = rootDir self.lastCheckedDisk = time.time() StringTemplateGroup.nameToGroupMap[self.name] = self self.templateLexerClass = lexer assert superGroup is None or isinstance(superGroup, StringTemplateGroup) self.superGroup = superGroup if file is not None: assert hasattr(file, 'read') self.templatesDefinedInGroupFile = True if lexer is not None: self.templateLexerClass = lexer else: self.templateLexerClass = AngleBracketTemplateLexer.Lexer assert superGroup is None or isinstance(superGroup, StringTemplateGroup) self.superGroup = superGroup self.parseGroup(file) assert self.name is not None StringTemplateGroup.nameToGroupMap[self.name] = self self.verifyInterfaceImplementations() def getTemplateLexerClass(self): """ What lexer class to use to break up templates. If not lexer set for this group, use static default. """ if self._templateLexerClass is not None: return self._templateLexerClass return self.defaultTemplateLexerClass def setTemplateLexerClass(self, lexer): if isinstance(lexer, basestring): try: self._templateLexerClass = { 'default': DefaultTemplateLexer.Lexer, 'angle-bracket': AngleBracketTemplateLexer.Lexer, }[lexer] except KeyError: raise ValueError('Unknown lexer id %r' % lexer) elif isinstance(lexer, type) and issubclass(lexer, antlr.CharScanner): self._templateLexerClass = lexer elif lexer is not None: raise TypeError( "Lexer must be string or lexer class, got %r" % type(lexer).__name__ ) templateLexerClass = property(getTemplateLexerClass, setTemplateLexerClass) getTemplateLexerClass = deprecated(getTemplateLexerClass) setTemplateLexerClass = deprecated(setTemplateLexerClass) @deprecated def getName(self): return self.name @deprecated def setName(self, name): self.name = name def setSuperGroup(self, superGroup): if superGroup is None or isinstance(superGroup, StringTemplateGroup): self._superGroup = superGroup elif isinstance(superGroup, basestring): # Called by group parser when ": supergroupname" is found. # This method forces the supergroup's lexer to be same as lexer # for this (sub) group. superGroupName = superGroup superGroup = StringTemplateGroup.nameToGroupMap.get( superGroupName, None) if superGroup is not None: # we've seen before; just use it self.superGroup = superGroup else: # else load it using this group's template lexer superGroup = self.loadGroup( superGroupName, lexer=self.templateLexerClass) if superGroup is not None: StringTemplateGroup.nameToGroupMap[superGroup] = superGroup self._superGroup = superGroup elif self.groupLoader is None: self.listener.error("no group loader registered", None) else: raise TypeError( "Need StringTemplateGroup or string, got %s" % type(superGroup).__name__ ) def getSuperGroup(self): return self._superGroup superGroup = property(getSuperGroup, setSuperGroup) getSuperGroup = deprecated(getSuperGroup) setSuperGroup = deprecated(setSuperGroup) def getGroupHierarchyStackString(self): """Walk up group hierarchy and show top down to this group""" groupNames = [] p = self while p is not None: groupNames.insert(0, p.name) p = p.superGroup return '[' + ' '.join(groupNames) + ']' @deprecated def getRootDir(self): return self.rootDir @deprecated def setRootDir(self, rootDir): self.rootDir = rootDir def implementInterface(self, interface): """ Indicate that this group implements this interface. Load if necessary if not in the nameToInterfaceMap. """ if isinstance(interface, StringTemplateGroupInterface): self.interfaces.append(interface) else: interfaceName = interface interface = self.nameToInterfaceMap.get(interfaceName, None) if interface is not None: # we've seen before; just use it self.interfaces.append(interface) return # else load it interface = self.loadInterface(interfaceName) if interface is not None: self.nameToInterfaceMap[interfaceName] = interface self.interfaces.append(interface) elif self.groupLoader is None: self.listener.error("no group loader registered", None) ## StringTemplate object factory; each group can have its own. def createStringTemplate(self): return StringTemplate() def getInstanceOf(self, name, enclosingInstance=None, attributes=None): """ A support routine that gets an instance of name knowing which ST encloses it for error messages. """ assert isinstance(name, basestring) assert enclosingInstance is None or isinstance(enclosingInstance, StringTemplate) assert attributes is None or isinstance(attributes, dict) st = self.lookupTemplate(name, enclosingInstance) if st is not None: st = st.getInstanceOf() if attributes is not None: st.attributes = attributes return st return None def getEmbeddedInstanceOf(self, name, enclosingInstance): assert isinstance(name, basestring) assert enclosingInstance is None or isinstance(enclosingInstance, StringTemplate) st = None # TODO: seems like this should go into lookupTemplate if name.startswith("super."): # for super.foo() refs, ensure that we look at the native # group for the embedded instance not the current evaluation # group (which is always pulled down to the original group # from which somebody did group.getInstanceOf("foo"); st = enclosingInstance.nativeGroup.getInstanceOf( name, enclosingInstance ) else: st = self.getInstanceOf(name, enclosingInstance) # make sure all embedded templates have the same group as enclosing # so that polymorphic refs will start looking at the original group st.group = self st.enclosingInstance = enclosingInstance return st ## Get the template called 'name' from the group. If not found, # attempt to load. If not found on disk, then try the superGroup # if any. If not even there, then record that it's # NOT_FOUND so we don't waste time looking again later. If we've gone # past refresh interval, flush and look again. # # If I find a template in a super group, copy an instance down here def lookupTemplate(self, name, enclosingInstance=None): assert isinstance(name, basestring) assert enclosingInstance is None or isinstance(enclosingInstance, StringTemplate) if name.startswith('super.'): if self.superGroup: dot = name.find('.') name = name[dot+1:] return self.superGroup.lookupTemplate(name, enclosingInstance) raise ValueError(self.name + ' has no super group; ' + 'invalid template: ' + name) self.checkRefreshInterval() st = self.templates.get(name, None) if not st: # not there? Attempt to load if not self.templatesDefinedInGroupFile: # only check the disk for individual template st = self.loadTemplateFromBeneathRootDir(self.getFileNameFromTemplateName(name)) if (not st) and self.superGroup: # try to resolve in super group st = self.superGroup.getInstanceOf(name) # make sure that when we inherit a template, that it's # group is reset; it's nativeGroup will remain where it was if st is not None: st.group = self if st: # found in superGroup # insert into this group; refresh will allow super # to change it's def later or this group to add # an override. self.templates[name] = st else: # not found; remember that this sucker doesn't exist self.templates[name] = StringTemplateGroup.NOT_FOUND_ST context = "" if enclosingInstance is not None: context = ( "; context is "+ enclosingInstance.enclosingInstanceStackString ) hier = self.getGroupHierarchyStackString() context += "; group hierarchy is "+hier raise ValueError( "Can't load template " + self.getFileNameFromTemplateName(name) + context ) elif st is StringTemplateGroup.NOT_FOUND_ST: return None return st def checkRefreshInterval(self): if self.templatesDefinedInGroupFile: return if self.refreshInterval == 0 or \ (time.time() - self.lastCheckedDisk) >= \ self.refreshInterval: # throw away all pre-compiled references self.templates = {} self.lastCheckedDisk = time.time() def loadTemplate(self, name, src): if isinstance(src, basestring): template = None try: br = open(src, 'r') try: template = self.loadTemplate(name, br) finally: br.close() # FIXME: eek, that's ugly except Exception, e: raise return template elif hasattr(src, 'readlines'): buf = src.readlines() # strip newlines etc.. from front/back since filesystem # may add newlines etc... pattern = str().join(buf).strip() if not pattern: self.error("no text in template '"+name+"'") return None return self.defineTemplate(name, pattern) raise TypeError( 'loadTemplate should be called with a file or filename' ) ## Load a template whose name is derived from the template filename. # If there is a rootDir, try to load the file from there. # def loadTemplateFromBeneathRootDir(self, fileName): template = None name = self.getTemplateNameFromFileName(fileName) # if no rootDir, try to load as a resource in CLASSPATH # In the Python case that is of course the sys.path if not self.rootDir: try: br, pathName, descr = imp.find_module(name) except ImportError: br = None if br is None: return None try: try: template = self.loadTemplate(name, br) except IOError, ioe: self.error("Problem reading template file: "+fileName, ioe) finally: try: br.close() except IOError, ioe2: self.error('Cannot close template file: ' + pathName, ioe2) return template # load via rootDir template = self.loadTemplate(name, self.rootDir + '/' + fileName) return template ## (def that people can override behavior; not a general # purpose method) # def getFileNameFromTemplateName(self, templateName): return templateName + DEFAULT_EXTENSION ## Convert a filename relativePath/name.st to relativePath/name. # (def that people can override behavior; not a general # purpose method) # def getTemplateNameFromFileName(self, fileName): name = fileName suffix = name.rfind(DEFAULT_EXTENSION) if suffix >= 0: name = name[:suffix] return name ## Define an examplar template; precompiled and stored # with no attributes. Remove any previous definition. # def defineTemplate(self, name, template): if name is not None and '.' in name: raise ValueError("cannot have '.' in template names") st = self.createStringTemplate() st.name = name st.group = self st.nativeGroup = self st.template = template st.errorListener = self.listener self.templates[name] = st return st def defineRegionTemplate(self, enclosingTemplate, regionName, template, type): """Track all references to regions <@foo>...<@end> or <@foo()>.""" if isinstance(enclosingTemplate, StringTemplate): enclosingTemplateName = self.getMangledRegionName( enclosingTemplate.getOutermostName(), regionName ) enclosingTemplate.getOutermostEnclosingInstance().addRegionName(regionName) else: enclosingTemplateName = self.getMangledRegionName(enclosingTemplate, regionName) regionST = self.defineTemplate(enclosingTemplateName, template) regionST.setIsRegion(True) regionST.regionDefType = type return regionST def defineImplicitRegionTemplate(self, enclosingTemplate, name): """ Track all references to regions <@foo()>. We automatically define as @enclosingtemplate.foo() ::= "" You cannot set these manually in the same group; you have to subgroup to override. """ return self.defineRegionTemplate( enclosingTemplate, name, "", REGION_IMPLICIT ) def getMangledRegionName(self, enclosingTemplateName, name): """ The 'foo' of t() ::= '<@foo()>' is mangled to 'region__t__foo' """ return "region__"+enclosingTemplateName+"__"+name def getUnMangledTemplateName(self, mangledName): """ Return "t" from "region__t__foo" """ return mangledName[len("region__"):mangledName.rindex("__")] ## Make name and alias for target. Replace any previous def of name# def defineTemplateAlias(self, name, target): targetST = self.getTemplateDefinition(target) if not targetST: self.error('cannot alias ' + name + ' to undefined template: ' + target) return None self.templates[name] = targetST return targetST def isDefinedInThisGroup(self, name): st = self.templates.get(name, None) if st is not None: if st.isRegion(): # don't allow redef of @t.r() ::= "..." or <@r>...<@end> if st.regionDefType == REGION_IMPLICIT: return False return True return False ## Get the ST for 'name' in this group only # def getTemplateDefinition(self, name): if self.templates.has_key(name): return self.templates[name] ## Is there *any* definition for template 'name' in this template # or above it in the group hierarchy? # def isDefined(self, name): try: return self.lookupTemplate(name) is not None except ValueError: return False def parseGroup(self, r): try: lexer = GroupLexer.Lexer(r) parser = GroupParser.Parser(lexer) parser.group(self) # sys.stderr.write("read group\n" + str(self)) except "foo", e: # FIXME: Exception, e: name = "" if self.name: name = self.name self.error('problem parsing group ' + name + ': ' + str(e), e) def verifyInterfaceImplementations(self): """verify that this group satisfies its interfaces""" for interface in self.interfaces: missing = interface.getMissingTemplates(self) mismatched = interface.getMismatchedTemplates(self) if missing: self.error( "group " + self.name + " does not satisfy interface " + interface.name + ": missing templates [" + ', '.join(["'%s'" % m for m in missing]) + ']' ) if mismatched: self.error( "group " + self.name + " does not satisfy interface " + interface.name + ": mismatched arguments on these templates [" + ', '.join(["'%s'" % m for m in mismatched]) + ']' ) @deprecated def getRefreshInterval(self): return self.refreshInterval ## How often to refresh all templates from disk. This is a crude # mechanism at the moment--just tosses everything out at this # frequency. Set interval to 0 to refresh constantly (no caching). # Set interval to a huge number like MAX_INT to have no refreshing # at all (DEFAULT); it will cache stuff. @deprecated def setRefreshInterval(self, refreshInterval): self.refreshInterval = refreshInterval def setErrorListener(self, listener): self.listener = listener def getErrorListener(self): return self.listener errorListener = property(getErrorListener, setErrorListener) getErrorListener = deprecated(getErrorListener) setErrorListener = deprecated(setErrorListener) ## Specify a StringTemplateWriter implementing class to use for # filtering output def setStringTemplateWriter(self, c): self.userSpecifiedWriter = c ## return an instance of a StringTemplateWriter that spits output to w. # If a writer is specified, use it instead of the default. def getStringTemplateWriter(self, w): stw = None if self.userSpecifiedWriter: try: stw = self.userSpecifiedWriter(w) except RuntimeError, e: #FIXME Exception, e: self.error('problems getting StringTemplateWriter', e) if not stw: stw = AutoIndentWriter(w) return stw ## Specify a complete map of what object classes should map to which # renderer objects for every template in this group (that doesn't # override it per template). @deprecated def setAttributeRenderers(self, renderers): self.attributeRenderers = renderers ## Register a renderer for all objects of a particular type for all # templates in this group. # def registerRenderer(self, attributeClassType, renderer): if not self.attributeRenderers: self.attributeRenderers = {} self.attributeRenderers[attributeClassType] = renderer ## What renderer is registered for this attributeClassType for # this group? If not found, as superGroup if it has one. # def getAttributeRenderer(self, attributeClassType): if not self.attributeRenderers: if not self.superGroup: return None # no renderers and no parent? Stop. # no renderers; consult super group return self.superGroup.getAttributeRenderer(attributeClassType) if self.attributeRenderers.has_key(attributeClassType): return self.attributeRenderers[attributeClassType] elif self.superGroup is not None: # no renderer registered for this class, check super group return self.superGroup.getAttributeRenderer(attributeClassType) return None def getMap(self, name): if not self.maps: if not self.superGroup: return None return self.superGroup.getMap(name) m = None if self.maps.has_key(name): m = self.maps[name] if (not m) and self.superGroup: m = self.superGroup.getMap(name) return m def defineMap(self, name, mapping): """ Define a map for this group; not thread safe...do not keep adding these while you reference them. """ self.maps[name] = mapping @classmethod def registerGroupLoader(cls, loader): cls.groupLoader = loader @classmethod def registerDefaultLexer(cls, lexerClass): cls.defaultTemplateLexerClass = lexerClass @classmethod def loadGroup(cls, name, superGroup=None, lexer=None): if cls.groupLoader is not None: return cls.groupLoader.loadGroup(name, superGroup, lexer) return None @classmethod def loadInterface(cls, name): if cls.groupLoader is not None: return cls.groupLoader.loadInterface(name) return None def error(self, msg, e = None): if self.listener: self.listener.error(msg, e) else: sys.stderr.write('StringTemplate: ' + msg + ': ' + e + '\n') traceback.print_exc() def getTemplateNames(self): return self.templates.keys() def emitDebugStartStopStrings(self, emit): """ Indicate whether ST should emit ... strings for debugging around output for templates from this group. """ self.debugTemplateOutput = emit def doNotEmitDebugStringsForTemplate(self, templateName): if self.noDebugStartStopStrings is None: self.noDebugStartStopStrings = set() self.noDebugStartStopStrings.add(templateName) def emitTemplateStartDebugString(self, st, out): if (self.noDebugStartStopStrings is None or st.name not in self.noDebugStartStopStrings ): out.write("<"+st.name+">") def emitTemplateStopDebugString(self, st, out): if (self.noDebugStartStopStrings is None or st.name not in self.noDebugStartStopStrings ): out.write("") def toString(self, showTemplatePatterns=True): buf = StringIO() buf.write('group ' + str(self.name) + ';\n') sortedNames = self.templates.keys() sortedNames.sort() for tname in sortedNames: st = self.templates[tname] if st != StringTemplateGroup.NOT_FOUND_ST: args = st.formalArguments.keys() args.sort() buf.write(str(tname) + '(' + ",".join(args) + ')') if showTemplatePatterns: buf.write(' ::= <<' + str(st.template) + '>>\n') else: buf.write('\n') retval = buf.getvalue() buf.close() return retval __str__ = toString stringtemplate3-3.1/stringtemplate3/__init__.py0000444000175000001440000000105710756132622020625 0ustar pinkusers __all__ = ['language'] __version__ = "3.1" ## track probable issues like setting attribute that is not referenced. # Set to true to make StringTemplate check your work as it evaluates # templates. Problems are sent to error listener. Currently warns when # you set attributes that are not used. lintMode = False from stringtemplate3.errors import * from stringtemplate3.writers import * from stringtemplate3.templates import * from stringtemplate3.groups import * from stringtemplate3.interfaces import * from stringtemplate3.grouploaders import * stringtemplate3-3.1/stringtemplate3/grouploaders.py0000444000175000001440000001364010756132622021575 0ustar pinkusers # [The "BSD licence"] # Copyright (c) 2003-2006 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import sys import os import traceback import codecs from stringtemplate3.utils import deprecated from stringtemplate3.groups import StringTemplateGroup from stringtemplate3.interfaces import StringTemplateGroupInterface from stringtemplate3.language import AngleBracketTemplateLexer class StringTemplateGroupLoader(object): """ When group files derive from another group, we have to know how to load that group and its supergroups. This interface also knows how to load interfaces """ def loadGroup(self, groupName, superGroup=None, lexer=None): """ Load the group called groupName from somewhere. Return null if no group is found. Groups with region definitions must know their supergroup to find templates during parsing. Specify the template lexer to use for parsing templates. If null, it assumes angle brackets <...>. """ raise NotImplementedError def loadInterface(self, interfaceName): """ Load the interface called interfaceName from somewhere. Return null if no interface is found. """ raise NotImplementedError class PathGroupLoader(StringTemplateGroupLoader): """ A brain dead loader that looks only in the directory(ies) you specify in the ctor. You may specify the char encoding. """ def __init__(self, dir=None, errors=None): """ Pass a single dir or multiple dirs separated by colons from which to load groups/interfaces. """ StringTemplateGroupLoader.__init__(self) ## List of ':' separated dirs to pull groups from self.dirs = dir.split(':') self.errors = errors ## How are the files encoded (ascii, UTF8, ...)? # You might want to read UTF8 for example on an ascii machine. self.fileCharEncoding = sys.getdefaultencoding() def loadGroup(self, groupName, superGroup=None, lexer=None): if lexer is None: lexer = AngleBracketTemplateLexer.Lexer try: fr = self.locate(groupName+".stg") if fr is None: self.error("no such group file "+groupName+".stg") return None try: return StringTemplateGroup( file=fr, lexer=lexer, errors=self.errors, superGroup=superGroup ) finally: fr.close() except IOError, ioe: self.error("can't load group "+groupName, ioe) return None def loadInterface(self, interfaceName): try: fr = self.locate(interfaceName+".sti") if fr is None: self.error("no such interface file "+interfaceName+".sti") return None try: return StringTemplateGroupInterface(fr, self.errors) finally: fr.close() except (IOError, OSError), ioe: self.error("can't load interface "+interfaceName, ioe) return None def locate(self, name): """Look in each directory for the file called 'name'.""" for dir in self.dirs: path = os.path.join(dir, name) if os.path.isfile(path): fr = open(path, 'r') # FIXME: something breaks, when stream return unicode if self.fileCharEncoding is not None: fr = codecs.getreader(self.fileCharEncoding)(fr) return fr return None @deprecated def getFileCharEncoding(self): return self.fileCharEncoding @deprecated def setFileCharEncoding(self, fileCharEncoding): self.fileCharEncoding = fileCharEncoding def error(self, msg, exc=None): if self.errors is not None: self.errors.error(msg, exc) else: sys.stderr.write("StringTemplate: "+msg+"\n") if exc is not None: traceback.print_exc() class CommonGroupLoader(PathGroupLoader): """ Subclass o PathGroupLoader that also works, if the package is packaged in a zip file. FIXME: this is not yet implemented, behaviour is identical to PathGroupLoader! """ # FIXME: this needs to be overridden! def locate(self, name): """Look in each directory for the file called 'name'.""" return PathGroupLoader.locate(self, name) stringtemplate3-3.1/stringtemplate3/errors.py0000444000175000001440000000423510756132622020403 0ustar pinkusers # [The "BSD licence"] # Copyright (c) 2003-2006 Terence Parr # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. The name of the author may not be used to endorse or promote products # derived from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES # OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, # INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT # NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF # THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import sys import traceback ## Lets you specify where errors, warnings go. class StringTemplateErrorListener(object): def error(self, msg, e): raise NotImplementedError def warning(self, msg): raise NotImplementedError class DefaultStringTemplateErrorListener(StringTemplateErrorListener): def __init__(self, output=None): StringTemplateErrorListener.__init__(self) self.output = output or sys.stderr def error(self, msg, exc): self.output.write(msg + '\n') if exc is not None: traceback.print_exc(file=self.output) def warning(self, msg): self.output.write(msg + '\n') DEFAULT_ERROR_LISTENER = DefaultStringTemplateErrorListener() stringtemplate3-3.1/CHANGES.txt0000444000175000001440000000073110756132622015176 0ustar pinkusers2008-02-17: Release 3.1 - Resolve ST-34. Super group loading was not using template lexer (<...> or $...$) according to subclass. Updated group loader interface. - Improved error message for "template not found" - $first(list).prop$ was not allowed. Fixed and added a unit test. 2007-12-03: Release 3.1b1 - Initial beta release of V3.1. - Lots of API breaking changes, please read the wiki for more information: stringtemplate3-3.1/setup.py0000444000175000001440000000263410756132622015103 0ustar pinkusers#!/usr/bin/env python from distutils.core import setup setup( name="stringtemplate3", version="3.1", description="A powerful template engine with strict model-view separation", long_description=""" ST (StringTemplate) is a template engine for generating source code, web pages, emails, or any other formatted text output. ST is particularly good at multi-targeted code generators, multiple site skins, and internationalization/localization. It evolved over years of effort developing jGuru.com. ST also generates this website and powers the ANTLR v3 code generator. Its distinguishing characteristic is that it strictly enforces model-view separation unlike other engines. """, classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Topic :: Software Development :: Libraries :: Python Modules', 'Topic :: Text Processing', ], author="Terence Parr / Marq Kole / Benjamin Niemann", author_email="parrt AT antlr.org / marq.kole AT philips.com / pink@odahoda.de", maintainer="Benjamin Niemann", maintainer_email="pink@odahoda.de", url="http://www.stringtemplate.org/", license="BSD", platform="any", packages=['stringtemplate3', 'stringtemplate3.language'], ) stringtemplate3-3.1/LICENSE.txt0000444000175000001440000000262010756132622015207 0ustar pinkusers[The "BSD licence"] Copyright (c) 2003-2008 Terence Parr All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.